Qwen 3.5 4B Mirai-M

trymirai/Qwen3.5-4B-M

Vendor: Alibaba
Quantization: Mirai-M
Parameters: 4B
Size: 2.3 GB

$ brew install mirai$ mirai --model trymirai/Qwen3.5-4B-M

Benchmarks

0.87 s

lower is better ↓

0.87 s

uzu

0.94 s

mlx

0.93 s

llamacpp

141 t/s

higher is better ↑

141 t/s

uzu

144 t/s

mlx

91 t/s

llamacpp

2.67 GB

lower is better ↓

2.67 GB

3.58 GB

2.56 GB

0.31.2 llama.cpp0.3.23/1,357 input tokens/345 output tokens

Benchmarked 7 Aug 2026

Integrate with SDK

Choose framework

Run the following command to install Mirai SDK

https://github.com/trymirai/uzu-swift

Apply code

1import Foundation2import Uzu34public func runChat() async throws {5    let engineConfig = EngineConfig.create()6    let engine = try await Engine.create(config: engineConfig)7    8    guard let model = try await engine.model(identifier: "alibaba:qwen3.5:0.8b:mirai:mirai-m:4") else {9        return10    }11    for try await update in try await engine.download(model: model).iterator() {12        print(String(format: "\r\u{001B}[2KDownload progress: %.2f%%", update.progress() * 100), terminator: "")13        fflush(stdout)14    }15    print()16    17    let messages = [18        ChatMessage.system().withText(text: "You are a helpful assistant"),19        ChatMessage.user().withText(text: "Tell me a short, funny story about a robot")20    ]21    let session = try await engine.chat(model: model, config: .create())22    let stream = await session.replyWithStream(input: messages, config: .create())23    var message: ChatMessage? = nil24    for try await update in stream.iterator() {25        switch update {26        case .replies(let replies):27            let reply = replies.last28            message = reply?.message29            print("Generated tokens: \(reply?.stats.tokensCountOutput ?? 0)")30        case .error(let error):31            print("Error: \(error)")32        }33    }34    print("Reasoning: \(message?.reasoning() ?? "empty")")35    print("Text: \(message?.text() ?? "empty")")36}37

1Choose framework

2Run the following command to install Mirai SDK

https://github.com/trymirai/uzu-swift

3Apply code

1	import Foundation
2	import Uzu
3
4	public func runChat() async throws {
5	let engineConfig = EngineConfig.create()
6	let engine = try await Engine.create(config: engineConfig)
7
8	guard let model = try await engine.model(identifier: "alibaba:qwen3.5:0.8b:mirai:mirai-m:4") else {
9	return
10	}
11	for try await update in try await engine.download(model: model).iterator() {
12	print(String(format: "\r\u{001B}[2KDownload progress: %.2f%%", update.progress() * 100), terminator: "")
13	fflush(stdout)
14	}
15	print()
16
17	let messages = [
18	ChatMessage.system().withText(text: "You are a helpful assistant"),
19	ChatMessage.user().withText(text: "Tell me a short, funny story about a robot")
20	]
21	let session = try await engine.chat(model: model, config: .create())
22	let stream = await session.replyWithStream(input: messages, config: .create())
23	var message: ChatMessage? = nil
24	for try await update in stream.iterator() {
25	switch update {
26	case .replies(let replies):
27	let reply = replies.last
28	message = reply?.message
29	print("Generated tokens: \(reply?.stats.tokensCountOutput ?? 0)")
30	case .error(let error):
31	print("Error: \(error)")
32	}
33	}
34	print("Reasoning: \(message?.reasoning() ?? "empty")")
35	print("Text: \(message?.text() ?? "empty")")
36	}
37

Details

Mirai's Qwen3.5-4B Medium Quantization

A state-of-the-art quantization that runs at about 84 tok/s on Apple M4 Pro while staying under 2.53 GB resident memory. Read more in our blog post.

Quickstart

If you are on macOS, the easiest way is to install the `mirai` Homebrew package and then run the CLI:

brew install mirai
mirai --model trymirai/Qwen3.5-4B-M

Currently only Apple silicon inference is supported. If you want to build things from source, read this overview.

Model page | API Docs

Method

Mirai Medium uses 4-bit asymmetric integer quantization with 4-bit zero points and bf16 scales, plus block-diagonal Random Hadamard Transforms with block size 32 to suppress outliers. The checkpoint is produced with Post-Training Quantization, using YAQA with additional stabilization heuristics, followed by a lightweight Quantization-Aware Distillation step for further quality recovery.

Benchmarks & Evaluations

In our reasoning-enabled MMLU-Pro evaluation with an 81,920-token limit, Mirai-M reaches 75.9% accuracy, compared with 77.7% for the unquantized BF16 Qwen/Qwen3.5-4B baseline under the same setting.

Device	Generation	Prefill
Apple A18 Pro	19 tok/s	152 tok/s
Apple M1	21 tok/s	146 tok/s
Apple M2	32 tok/s	208 tok/s
Apple M2 Pro	62 tok/s	388 tok/s
Apple M3 Max	117 tok/s	1,375 tok/s
Apple M4	38 tok/s	422 tok/s
Apple M4 Pro	84 tok/s	815 tok/s
Apple M4 Max	146 tok/s	1,583 tok/s

Citation

If you find our work helpful, feel free to give us a cite.

bibtex

@misc{mirai-quant,
    title  = {{Mirai Quantization}: Redefining the speed-quality frontier for local LLMs on Apple silicon},
    author = {Artur Chakhvadze and Ryan Mathieu and Roman Knyazhitskiy and Nikolai Voinilenko and Chen-Chen Yeh and Artur Mullakhmetov and Eugene Bokhan and others},
    note   = {In collaboration with others at Mirai Labs},
    month  = {June},
    year   = {2026},
    url    = {https://trymirai.com/blog/quantization}
}

Explore all local models