Routing between a local model and a remote one
Routing between a local and a hosted model needs a decision you can audit. Route before attempting, record the path on every result, and watch the proportion.
All posts
One inference API, two platform runtimes
Designing an expect/actual seam so platform differences stay on one side. What the shared declaration must not promise, and where the abstraction leaks.
How to know your on-device model got worse
A tiny eval suite that fails the build, the budget that bounds it, and why a verdict needs a third value meaning neither pass nor fail but undetermined.
Wiring ML Kit GenAI into a Compose screen without blocking it
Wiring an on-device GenAI capability into Compose is mostly state design. Model the availability and failure states before the happy path, not after.
Shipping a model in your APK without breaking the size budget
Bundling a model trades download size for a guarantee. The decision turns on first-run experience, and the budget needs a gate rather than good intentions.
Updating a model without shipping an app release
Shipping a model outside the store release means versioning, integrity, rollback and a census of what is actually running — and a human gate on the promote.
An AI feature that still works on a plane
Reading offline is the easy half. The write path is where it breaks: stale cold starts, unresolved conflicts, unorchestrated retries and torn pages.
The on-device privacy argument, stated honestly
Running the model locally is not a privacy guarantee — it is a precondition for one. Only a mechanism that refuses by default makes the boundary real.
Prompt injection does not go away because the model is local
A denylist of three dangerous commands was bypassed by a fourth. Why injection defence has to be allowlist-shaped, and what on-device quietly takes away.
Quantization has a quality cliff, and you can see it
Quantization degrades non-linearly, so an average score hides the cliff. Find it with a suite that fails per-case, and keep the per-case results, not the mean.
Streaming tokens into Compose without recomposing the world
Streaming output updates state far faster than a normal screen. Narrow what reads the changing value, and let the smallest possible scope recompose.
How do you test a screen whose text is generated?
Snapshot tests die the moment output varies per run. Asserting invariants instead of strings, the vacuous-check trap, and a verdict that means could-not-determine.