The tech
There is no trick.
The model is on the phone.
Every answer, translation and image is computed by a neural network whose weights sit in your device's storage and run on its own silicon. No request goes out because there is nothing on the other end to receive it.
What actually runs
The engine is llama.cpp with the ggml-hexagon backend — both open source, MIT licensed, and shipped inside the app. On a Snapdragon phone that backend drives the Hexagon NPU directly; where there is no supported NPU the same engine runs on the CPU and the app says so honestly rather than pretending.
Models are GGUF files quantised to Q4_0 — the format the Hexagon path and ARM's i8mm instructions both prefer. You download the one you want once, from Hugging Face, and it stays yours: no re-validation, no licence check, no phoning home to keep working.
Nothing proprietary from Qualcomm is redistributed. The NPU path uses llama.cpp's own Hexagon skeletons, which is why the app can be given away without a redistribution licence hanging over it.
Measured — S25 Ultra, 8 Elite
- Qwen3.5 4B, NPU prefill: 213–223 tok/s
- The same model on CPU prefill: 58.2 tok/s
- Qwen 2B on the open NPU backend: 700 tok/s prefill
- Gemma 4 E4B, NPU prefill: 285.3 tok/s
- Decode, 4B: 12.9–13.9 tok/s
- Decode, 0.8B on CPU: 64–67 tok/s
Where the work happens
A few centimetres from where you typed it.
What we learned measuring it
The NPU wins prefill, not decode. Reading your prompt is matmul-heavy and that is exactly what the Hexagon units are for — 700 tok/s against 48 on the CPU for a 2B model. Writing the answer back is different: every token walks the entire weight set through memory, so decode lands at the phone's memory bandwidth no matter which chip does the work.
Model size is the lever, not the backend. A 0.8B model on the CPU decodes nearly five times faster than a 4B on the NPU, and the ratio tracks the weight-set sizes to within a few percent. That is why the apps let you pick your model instead of choosing one for you: on a phone, the honest speed control is how big a brain you loaded.
We publish the numbers that are unflattering too. The measurements above are single-device runs, not a benchmark suite, and they move a few percent with heat. When an app cannot verify that the NPU took the model, it tells you it is on the CPU rather than claiming a win it did not get.
What leaves the device
Two things, both of which you start: the model download (a file request to Hugging Face) and, in Offline Notebook only, fetching a web page whose address you typed. That is the entire list. No conversation, document, image, memory or usage statistic is transmitted anywhere, because the apps contain no code that could send one.
The full statement, with the permission table, is the privacy policy.
Open source, credited
llama.cpp, ggml and the rest ship with their licence texts readable inside every app, offline, under About → Open source licences. Each release is checked against its own package: a binary with no attribution fails the build.