Chinese and English notes from building a local inference runtime.

I write about measurements, false starts, and the hypotheses the numbers killed. A result without its script, its error bars, and its failed alternatives is a claim, not a result.

• The economics of speculative decoding: +41% on 9B, break-even on 35B
  2026-09-02 · English · Drafting is cheap and verification is expensive, but the marginal cost of verification decides whether speculation pays.
• Put the Neural Engine to work: why the vision tower should not run on the GPU
  2026-09-01 · English · Every Apple Silicon Mac has an NPU, yet most local inference runtimes leave it idle. Moving the vision tower there keeps the GPU available for the language model.
• Local inference was designed for one person typing. Agents do not work that way.
  2026-08-30 · English · Concurrency, paged KV, tool calls, multimodal execution, and the hypotheses the numbers killed.