INFRASTRUCTURE · ANDROID LOCAL AISpec & prototype · active validation

Harnex

Name: Harnex
Author: Daniele Moltisanti

Your local AI harness for Android: one gateway, shared on-device LLM runtime, model plane, and control plane so Android teams can build local AI features without rebuilding infrastructure inside every app.

Android API 26+GGUF + llama.cppKotlin + JNI/C++MIT

Explore architecture ↓

Harnex architecture showing the app, gateway and local inference engine

Core Gateway Pattern: App → IPC → Gateway → EngineGGUF / llama.cpp

Project Mission

Make local AI on Android explicit, measurable and reusable.

Running LLMs on-device shouldn't mean re-implementing model loading, memory limits, and runtime bindings in every single app. Harnex abstracts on-device inference into an explicit system service.

ExplicitNo opaque black boxes

MeasurableTokens/sec & RAM tracking

ReusableOne gateway for N apps

The Challenge

Why on-device AI needs a Gateway layer

Building local LLM features on mobile today comes with structural friction that impedes product reliability.

Duplicated Engine Setup

Every app bundles its own runtime binaries, inflating APK size and multiplying maintenance overhead.

Uncontrolled Memory Spikes

On-device LLMs consume gigabytes of RAM. Without centralized management, apps crash silently under OOM conditions.

No Standard IPC Protocol

Integrating local AI into existing background services or third-party apps requires custom ad-hoc protocols.

Opaque Performance Metrics

Measuring prompt evaluation speed, generation latency, and thermal throttling requires dedicated telemetry.

Concept

Think of Harnex as the local AI layer of an Android device.

Just as Android provides system services for location, camera, or network, Harnex provides a system gateway for local LLM execution.

Unified Model Repository

Store and manage GGUF quantized models in a centralized device location shared across applications.

High-Performance C++ Runtime

Direct bindings to llama.cpp via JNI with hardware acceleration for ARM NEON and Vulkan GPU execution.

Standardized AIDL & IPC

Expose thread-safe Android Binder interfaces so any app can stream tokens without runtime lock-in.

Under the hood

Start simple. Then go technical.

The architecture is intentionally layered so the product surface, runtime contracts, model plane, native backend and observability can evolve without leaking implementation details into every client.

Android appproduct / client

LocalLlmClientpublic boundary

RuntimeOrchestratorlifecycle + scheduling

Model resolutionapp + use case

Model storeverified GGUF

InferenceBackendreplaceable backend

llama.cppJNI / C++17

Full technical architecture

Developer boundary

Client code stays small on purpose.

Applications depend on a neutral client and explicit identifiers rather than calling JNI directly. The target gateway keeps this interaction while moving backend ownership behind the host service.

val prepared = client.prepare(applicationId, useCaseId)
check(prepared.ready)

val sessionId = client.createSession(applicationId, useCaseId)
val handle = client.generate(
    request = generationRequest(sessionId, applicationId, useCaseId),
    listener = generationListener,
)

handle.cancel()

Connected console

Five product surfaces. One inference foundation.

Harnex connects runtime controls, model management, interactive testing, diagnostics and system settings into a unified mobile console.

Overview

Runtime readiness, device resources and next valid action.

Playground

Prompt, configure, stream and cancel inference locally.

Models

Import, download, verify, select and remove GGUF artifacts.

Diagnostics

Runs, health, resources, logs and benchmarks.

Settings

Privacy, storage, build information and developer controls.

Security Boundary: Current boundary: the runtime is still embedded and in-process inside Harnex. Other Android apps cannot connect to a shared Binder/AIDL service yet. The gateway is the target architecture, not a current capability.

Design values

Infrastructure designed for decisions, not demos.

The product is organized around the qualities that make local AI useful after the first successful generation.

Local-first control

Inference runs on-device and normal telemetry excludes prompts and generated output.

Reuse instead of duplication

The gateway vision centralizes model storage and resource ownership so multiple authorized apps can reuse infrastructure.

Evaluation before confidence

Latency, throughput, memory, thermal state, health and benchmarks make model choices inspectable.

Guardrails that enable delivery

Bindings, integrity, cancellation, failures and cleanup are explicit rather than hidden runtime assumptions.

Open, replaceable boundaries

Product contracts stay independent from llama.cpp internals, Android UI and transport details.

Shared operational visibility

The target gateway adds one privacy-conscious control plane across local AI applications.

EVIDENCE & LIMITS

The runtime foundation is integrated; the shared gateway is still the target architecture.

The page separates what exists in the connected console today from the physical-device and cross-application evidence still required before stronger platform claims are justified.

Evidence & maturity

Useful today. Still an engineering and validation project.

Harnex should not be presented as a production-ready Android inference platform yet. The current value is a connected console and tested foundation that de-risks the final gateway architecture.

Integrated today

Embedded GGUF inference path
Streaming and cancellation
Model import, verification and lifecycle
Connected console surfaces
Telemetry, health, resources and benchmarks

Still blocking production claims

Representative physical-device GGUF evidence
Memory and thermal validation on target devices
Packaging and production-readiness evidence
Cross-application Binder/AIDL host is still future work

EVIDENCE BOUNDARY

Integrated capability is not yet production-readiness evidence.

Harnex currently integrates embedded GGUF inference, streaming, cancellation, model lifecycle and diagnostic surfaces. Representative device evidence and the shared Binder/AIDL host remain necessary before describing it as a production Android inference platform.

Supported today

Embedded GGUF inference path is integrated
Streaming, cancellation and model lifecycle are implemented
Runtime health, resources and benchmark surfaces are connected
Client and backend boundaries are explicit in the current architecture

Not claimed here

Representative physical-device memory and thermal validation
Production packaging and reliability evidence
Cross-application Binder/AIDL host

STATUS & NEXT

Prove the data plane, stabilize the contract, then move ownership.

The roadmap deliberately validates the difficult runtime and model-management foundations before adding cross-application IPC and shared resource ownership.

Strategy

From console to gateway.

The sequence deliberately proves difficult runtime and model-management foundations before introducing cross-application IPC and shared ownership.

1 · Local AI Console

Prove the data plane

Distribution, storage, inference, cancellation, cleanup, diagnostics and real-device behavior.

2 · Stable client boundary

Stabilize the contract

Keep explicit application/use-case identity and backend-neutral domain contracts.

3 · Local AI Gateway

Move ownership

Protected Binder/AIDL transport plus a lightweight Android client integration.

4 · Multi-app platform

Unlock shared value

Shared models, RAM coordination, scheduling, health and monitoring across authorized apps.

CONNECTED SYSTEM

How the Android local-AI layer is evaluated

BUILD · INFRASTRUCTUREHarnexRuntime, models and diagnostics on-device

MEASURE · PERFORMANCEPerformance LabLatency, throughput and resource-fit evidenceMEASURE · NETWORKTraffic Monitoring AndroidNetwork-boundary observability on Android

Open-source · MIT

Make local AI explicit, measurable and reusable on Android.

Harnex is part of a broader mission: move beyond isolated AI experiments and build product infrastructure that can be understood, evaluated and scaled deliberately.

Explore Harnex on GitHub ↗Back to Daniele Moltisanti Portfolio ↗