Tempo9 measurements
Measured on one Mac.
Published with the method.
English · 中文版
Four questions matter for a local inference runtime: how long agents wait under load, how memory grows with context, whether tool calls remain correct, and whether vision leaves enough GPU capacity for the language model.
Multiple agents
Four sessions, shorter waits for the first token
Continuous batching admits new requests into work already in flight. Paged KV allocates cache only for tokens that exist, while the prefix cache reuses a shared system prompt or tool history. The measurement below isolates the delay visible to each agent: time to first token.
Four agents · lower is better
Every Tempo9 range ends before the matching comparison range
Ranges across two to three runs per cell; twelve requests per run. Axis: seconds to first token.
The 10K prompt approximates the system prompt OpenClaw sends on each turn. Both engines read the same GGUF and keep their default prefix caches. Concurrent decode throughput is omitted because it has not yet met the reproducibility bar for publication.
Source report: results_p13_serving_retest.md
Long context
Declared capacity is not preallocated capacity
Tempo9 allocates Paged KV as tokens arrive. Increasing the declared context limit from 8K to 128K therefore leaves startup memory at roughly 0.55 GB. When cold pages are offloaded, KV resides in GPU-accessible unified memory or on SSD; there is no separate host-memory staging tier.
KV + working memory · lower is better
From 8K to 128K, Tempo9 startup memory stays nearly flat
Declared capacity at startup
Scale: 0–4 GB
Memory after tokens arrive
Scale: 0–2.2 GB
The panels use different scales. Left: declared capacity before use. Right: memory after tokens become resident.
The chart includes KV and working memory and excludes file-backed model weights. Tokens that become resident still consume memory; paging removes the up-front reservation for unused capacity.
Source report: results_p4b_boundary.md
Tool calling
Accuracy first, generation speed second
The BFCL measurement exercises the service path an agent actually uses: request parsing, tool definitions, constrained generation, and returned tool arguments through the OpenAI-compatible HTTP endpoint.
Category breakdown
| BFCL v4 category | Tempo9 |
|---|---|
| simple (Python) | 95.25% |
| multiple | 94.50% |
| parallel | 89.00% |
| parallel multiple | 83.50% |
| Non-Live irrelevance | 84.17% |
Structured output with speculative decoding
200+ token tool-call JSON · one stream
104.9 tok/s under the structured-output condition
Decode throughput with TTFT removed. Speculation is opt-in and remains off by default.
The official BFCL v4 evaluator scored all 3,641 completed single-turn cases. The throughput result is scoped to structured JSON; the same speculative setting produces no net gain on prose.
Source reports: results_p11_bfcl_fixed.md · results_p6_mtp.md
From LLM to VLM
The same image, with vision on the NPU or GPU
Tempo9 can place the vision transformer on the Neural Engine while the language model continues on the GPU. This comparison keeps the image and language-model workload fixed and changes only where vision runs.
Text baseline · same image · placement only
With an image, NPU vision leaves 2.8× more LLM decode throughput
The text-only row is a baseline. The 2.8× comparison is NPU versus GPU placement for the same image.
The memory system also lets Qwen3-VL-30B-A3B complete an image question on this 24 GB machine. For a smaller model that both engines can run, the head-to-head result is deliberately mixed:
| Gemma 4 12B · one image | Tempo9 | llama.cpp |
|---|---|---|
| Image encoding | 28 ms | 131 ms |
| Answer decode | 28.2 tok/s | 33.9 tok/s |
The NPU result is a system-level placement result,
not a claim that the Neural Engine is faster than the GPU in isolation. Both
engines in the Gemma comparison read the same GGUF and mmproj.
Source reports: results_p7_power.md · results_p4_p5_tickets_vlm.md
Method and provenance
What these numbers describe
Machine
16-inch MacBook Pro, Apple M5 Pro, 24 GB unified memory, macOS Tahoe 26.5. Every result on this page comes from this machine.
System settings
iogpu.wired_limit_mb remains at the stock value. No engine receives a larger wired-memory allowance.
Main model
Qwen3.5-35B-A3B q3km, 15.98 GiB. Head-to-head sections use the same GGUF file in both engines.
Reading the page
Each section answers a different question. Throughput, latency, memory, and accuracy values should not be compared across workloads.
Engine revisions, comparator versions, scripts, raw outputs, invalidation rules, and superseded runs are retained in the benchmark repository. The repository will open with the first public release.