Apple Silicon LLM benchmarks
Benchmarked 7 Oct 2026
We compare uzu, MLX, llama.cpp, MTPLX, oMLX, and Splash locally on the same Apple device. Choose your device, then click a result to open the model's download and setup page.
How we measureOutput speedThe speed at which text appears on screen We use our own dataset mixture, which is derived from traces of several well-known public evals. The sampling is done with default sampling parameters from the original model's README (e.g. T=0.95, top-k=20). The datasets we use are as follows: STEM: AppWorld, BrowseComp+, DABStep, DeepSWE, TerminalBench 2.1, TerminalBench 3, TerminalBench Science, HLE math and a 'make a game' prompt. Prose: UPHELD, NCP-Bench, UTILMEM, CoSER, HLE humanities, WritingBench, MT-Bench and a 'write a story' prompt. Other: FinFirst, LTB, LilyBench, ASCII Bench The averaging weights are equal per dataset, so stem/prose/other weights are 9/8/4, and each dataset contributes an equal part to the final tok/s estimate.
uzu is the fastest in 12 of 12 models on this device.
tok/sHigher is better
LFM2.51.2B Thinking · 4bit
LFM2.51.2B Instruct · 4bit
LFM2.51.2B Thinking · 8bit
LFM2.51.2B Instruct · 8bit
Qwen 3.52B · 4bit
LFM2.52.6B · 4bit
Qwen 3.52B · 8bit
Qwen 3.59B · 4bit
LFM2.52.6B · 8bit
Qwen 3.54B · 4bit
Qwen 3.59B · 8bit
Qwen 3.54B · 8bit
For uzu, we use Mirai-M, Mirai-L as 4-bit and 8-bit references. For llama.cpp we select the closest quantized checkpoints based on KL divergence as measured by our KL eval tool. For mlx we always use mlx-community checkpoints.
Input speedHow fast the model processes the prompt Uzu uses native prefill_tokens_per_second; MTPLX uses prompt_tps; llama.cpp uses prompt_per_second; MLX divides prompt-token count by wall-clock time to the first token.
uzu is the fastest in 12 of 12 models on this device.
tok/sHigher is betterLFM2.51.2B Thinking · 8bit
LFM2.51.2B Instruct · 8bit
LFM2.51.2B Instruct · 4bit
LFM2.51.2B Thinking · 4bit
Qwen 3.52B · 8bit
LFM2.52.6B · 8bit
Qwen 3.52B · 4bit
LFM2.52.6B · 4bit
Qwen 3.54B · 8bit
Qwen 3.54B · 4bit
Qwen 3.59B · 8bit
Qwen 3.59B · 4bit
For uzu, we use Mirai-M, Mirai-L as 4-bit and 8-bit references. For llama.cpp we select the closest quantized checkpoints based on KL divergence as measured by our KL eval tool. For mlx we always use mlx-community checkpoints.
Resident memoryHow much RAM the model occupies while running Uzu reports session.peak_memory_usage; MLX and MTPLX use native peak-memory counters; llama.cpp uses process resident memory.
uzu leads in 11 of 12 models on this device.
GiBLower is betterLFM2.51.2B Instruct · 4bit
LFM2.51.2B Thinking · 4bit
LFM2.51.2B Instruct · 8bit
LFM2.51.2B Thinking · 8bit
Qwen 3.52B · 4bit
LFM2.52.6B · 4bit
Qwen 3.52B · 8bit
Qwen 3.54B · 4bit
LFM2.52.6B · 8bit
Qwen 3.54B · 8bit
Qwen 3.59B · 4bit
Qwen 3.59B · 8bit
For uzu, we use Mirai-M, Mirai-L as 4-bit and 8-bit references. For llama.cpp we select the closest quantized checkpoints based on KL divergence as measured by our KL eval tool. For mlx we always use mlx-community checkpoints.
Quantization quality
How we measure
Each configuration is run three times using the same prompt, which is approximately 1,357 input tokens long (the exact number may vary depending on the tokenizer and chat template). The maximum output length is set to 4,096 tokens. Before each iteration, the runner waits until both CPU and GPU temperatures are below 60°C.
uzu is run via its CLI. MLX is run through mlx-lm. llama.cpp is run through llama-cpp-python for autoregressive runs and through the llama.cpp server for speculative decoding runs. All engines use the same sampling parameters where supported, otherwise, they use their respective client defaults.