Run Gemma 4 on Your Mac with Tempo9

2026-10-27

English · 中文版

You can run Gemma 4 locally on Apple Silicon with Tempo9 using a GGUF file and an OpenAI-compatible API. This quick guide covers installation, model download, and inference, followed by a short benchmark against llama.cpp on an M5 Pro with 24 GB unified memory.

1. Install Tempo9

brew install thinkspread/tap/tempo9

2. Download Gemma 4

Choose the smaller E2B or the larger E4B. These commands download Q4_K_M GGUF builds from Unsloth:

python3 -m pip install -U huggingface_hub
mkdir -p ~/models/gemma4-e
cd ~/models/gemma4-e

# E2B
hf download unsloth/gemma-4-E2B-it-GGUF \
  gemma-4-E2B-it-Q4_K_M.gguf --local-dir .

# E4B (optional)
hf download unsloth/gemma-4-E4B-it-GGUF \
  gemma-4-E4B-it-Q4_K_M.gguf --local-dir .

3. Start the local server

For E2B:

tempo9 --gguf ~/models/gemma4-e/gemma-4-E2B-it-Q4_K_M.gguf

To run E4B, replace the filename with gemma-4-E4B-it-Q4_K_M.gguf. The server listens on localhost:11435 by default.

4. Make an API request

Tempo9 provides an OpenAI-compatible Chat Completions endpoint:

curl http://localhost:11435/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemma4",
    "messages": [
      {"role": "user", "content": "Explain speculative decoding in simple terms."}
    ],
    "max_tokens": 256
  }'

5. A short benchmark against llama.cpp

On the same M5 Pro, we compared Tempo9 1.2.1 and llama.cpp 0.7.0, loading identical Q4_K_M GGUF files. These measurements apply to the stated setup, not every model or device.

MeasurementTempo9 1.2.1llama.cpp 0.7.0
E2B single stream135–140 tok/s117–119 tok/s
E4B single stream75–76 tok/s68–69 tok/s
E2B, 16-client aggregate1,029–1,079 tok/s637–644 tok/s
Gemma 4 E2B: aggregate tok/s llama.cpp 0.7.0 Tempo9 1.2.1 N concurrent clients, 256 greedy tokens each; higher is better; two interleaved rounds. 4 clients Tempo9 1.2.1, 4 clients: 370–381 370–381 llama.cpp 0.7.0, 4 clients: 178–179 178–179 8 clients Tempo9 1.2.1, 8 clients: 631–667 631–667 llama.cpp 0.7.0, 8 clients: 328–330 328–330 16 clients Tempo9 1.2.1, 16 clients: 1029–1079 1029–1079 llama.cpp 0.7.0, 16 clients: 637–644 637–644

In this test, Tempo9 was roughly 10–16% faster in a single stream, with approximately 1.6× the aggregate throughput at 16 clients.

Two areas of optimization matter here: Gemma 4–specific attention execution and lower overhead in long-context prefill and KV-cache management. This post focuses on using the engine, rather than its internal implementation.

Benchmark notes

Tested on macOS 26.6.2, M5 Pro 24 GB, Tempo9 1.2.1 (engine 935b18586), and llama.cpp 0.7.0 (620d2225c). Single-stream tests generated 256 tokens. The concurrent test used 16 independent clients and server-reported completion-token counts. Streaming event counts matched reported completion tokens in the single-stream runs, although that check is not a token-boundary proof.

For full test procedures, see the benchmark script and published test prompt.

For installation and updates, see Tempo9 Releases.