I havewith
1
Run a recommended model on your Mac
Models quantized by Mirai and ranked for Apple M5 Max with 128 GB. Every command on this page uses the model shown at the top. Pick another row to switch.Qwen 3.8 27B Mirai-MMirai Medium. 4-bit asymmetric integer weights, group size 64, quantization-aware distillation. Comparable to Q4_K_M in size, lower KL divergence.
Best fit- SizeDownload size of the checkpoint.
- 15.6 GB
- MemoryPeak resident memory while generating, measured with uzu on Apple M5 Max 128GB.
- 15.7 GB
- OutputMeasured on Apple M5 Max 128GB.
- 106 tok/s
Run it in Terminal
brew install miraimirai --model trymirai/Qwen3.8-27B-MAlso recommended
All available models ↗- Size
- 15.6 GB
- Memory
- 15.7 GB
- Output
- 116 tok/s
- Size
- 16.6 GB
- Memory
- 16.1 GB
- Output
- 82 tok/s
- Size
- 6.0 GB
- Memory
- 6.2 GB
- Output
- 211 tok/s
2
Compare uzu with other engines for this model
Same model, same Mac, same prompt. uzu against MLX, MTPLX and llama.cpp on output speed, input speed and resident memory. Follows the model and Mac selected at the top of the page.Qwen 3.8 27B Mirai-M
Apple M5 Max 128GB
106 tok/s
higher is better ↑
Benchmarked 11 Oct 2026
Qwen 3.8 27B Mirai-M
Apple M5 Max 128GB
1259 tok/s
higher is better ↑
Benchmarked 11 Oct 2026
Qwen 3.8 27B Mirai-M
Apple M5 Max 128GB
15.66 GB
lower is better ↓
Benchmarked 11 Oct 2026
Benchmarked 11 Oct 2026
Speculative decodingA small draft model proposes 16 to 32 tokens; uzu verifies them in one pass. Code is easy to predict, chat is harder, so gains depend on the task.Qwen 3.8 27B · Mirai-M · Apple M5 Max 128GBHow it works ↗
benchmarks-mixture
106 tok/s
61 tok/s
43 tok/s
116 tok/s
uzuMTPLXoMLXSplash
Quantization qualityQuality loss from quantization, measured as mean KL divergence to the bfloat16 model over 100 Lmsys-Chat prompts. Lower is better. Noise floor is about 0.0008.Qwen 3.8 27BLearn more ↗
MiraiUnsloth / llama.cppMLXMTPLX↑ KL divergence, log · → checkpoint size, GB
Mirai-MMirai Medium. 4-bit asymmetric integer weights, group size 64, quantization-aware distillation. Comparable to Q4_K_M in size, lower KL divergence. · 4-bitMirai-LMirai Large. 8-bit symmetric integer weights with bfloat16 scales, group size 64. Closest to the original model. · 8-bit, below the noise floor of this test
3
Integrate this model with SDK
The same engine as the CLI and the Mac app, embedded in your app. Download a model once, then stream replies with a few lines.1 · Choose a framework
2 · Install
https://github.com/trymirai/uzu3 · Download the model and stream a reply
import Foundationimport Uzu public func runChat() async throws { let engineConfig = EngineConfig.create() let engine = try await Engine.create(config: engineConfig) guard let model = try await engine.model(identifier: "trymirai/Qwen3.8-27B-M") else { return } for try await update in try await engine.download(model: model).iterator() { print(String(format: "\r\u{001B}[2KDownload progress: %.2f%%", update.progress() * 100), terminator: "") fflush(stdout) } print() let messages = [ ChatMessage.system().withText(text: "You are a helpful assistant"), ChatMessage.user().withText(text: "Tell me a short, funny story about a robot") ] let session = try await engine.chat(model: model, config: .create()) let stream = await session.replyWithStream(input: messages, config: .create()) var message: ChatMessage? = nil for try await update in stream.iterator() { switch update { case .replies(let replies): let reply = replies.last message = reply?.message print("Generated tokens: \(reply?.stats.tokensCountOutput ?? 0)") case .error(let error): print("Error: \(error)") case .toolResults: break } } print("Reasoning: \(message?.reasoning() ?? "empty")") print("Text: \(message?.text() ?? "empty")")} 4
Try this model in the Mac app
A native macOS app on top of uzu. The same download also installs the mirai CLI.New chat
5
Run a coding agent on this model
mirai server exposes the selected model as an OpenAI-compatible API on localhost. Point any harness at it.1 · Start the server
mirai server --model trymirai/Qwen3.8-27B-M# OpenAI-compatible API on http://127.0.0.1:8000/v1POST /v1/chat/completionsStreaming when stream: true. Honors temperature, top_p, top_k and max_tokens.GET /v1/modelsLists the loaded model so harnesses can discover it.no API keyAny value is accepted. Most clients still require the field to be set.Change host or port with
--host and --port.2 · Point your agent at it
npm install -g --ignore-scripts @earendil-works/pi-coding-agent~/.pi/agent/models.json
{ "providers": { "mirai": { "baseUrl": "http://127.0.0.1:8000/v1", "api": "openai-completions", "apiKey": "none", "compat": { "supportsDeveloperRole": false, "supportsReasoningEffort": false }, "models": [ { "id": "trymirai/Qwen3.8-27B-M", "name": "Qwen3.8-27B-M (local)", "reasoning": true, "contextWindow": 32768, "maxTokens": 8192 } ] } }}3 · Start it
pi --provider mirai --model trymirai/Qwen3.8-27B-MAny agent that speaks the OpenAI API works the same way: base URL, any key, the model id above.
6
More from Mirai Labs
The code, the research behind the numbers on this page, and the rest of Mirai Labs.GitHub
Research
7