Qwen 3

Qwen/Qwen3-14B-MLX-8bit

Vendor: Alibaba
Quantization: MLX 8-bit
Parameters: 14B
Size: 14.2 GB

$ brew install mirai$ mirai --model Qwen/Qwen3-14B-MLX-8bit

Benchmarks

2.74 s

lower is better ↓

2.74 s

uzu

3.16 s

mlx

2.74 s

llamacpp

31 t/s

higher is better ↑

31 t/s

uzu

31 t/s

mlx

25 t/s

llamacpp

15.68 GB

lower is better ↓

15.68 GB

16.02 GB

17.70 GB

0.31.2 llama.cpp0.3.23/1,312 input tokens/512 output tokens

Benchmarked 7 Aug 2026

Integrate with SDK

Choose framework

Run the following command to install Mirai SDK

https://github.com/trymirai/uzu-swift

Apply code

1import Foundation2import Uzu34public func runChat() async throws {5    let engineConfig = EngineConfig.create()6    let engine = try await Engine.create(config: engineConfig)7    8    guard let model = try await engine.model(identifier: "alibaba:qwen3.5:0.8b:mirai:mirai-m:4") else {9        return10    }11    for try await update in try await engine.download(model: model).iterator() {12        print(String(format: "\r\u{001B}[2KDownload progress: %.2f%%", update.progress() * 100), terminator: "")13        fflush(stdout)14    }15    print()16    17    let messages = [18        ChatMessage.system().withText(text: "You are a helpful assistant"),19        ChatMessage.user().withText(text: "Tell me a short, funny story about a robot")20    ]21    let session = try await engine.chat(model: model, config: .create())22    let stream = await session.replyWithStream(input: messages, config: .create())23    var message: ChatMessage? = nil24    for try await update in stream.iterator() {25        switch update {26        case .replies(let replies):27            let reply = replies.last28            message = reply?.message29            print("Generated tokens: \(reply?.stats.tokensCountOutput ?? 0)")30        case .error(let error):31            print("Error: \(error)")32        }33    }34    print("Reasoning: \(message?.reasoning() ?? "empty")")35    print("Text: \(message?.text() ?? "empty")")36}37

1Choose framework

2Run the following command to install Mirai SDK

https://github.com/trymirai/uzu-swift

3Apply code

1	import Foundation
2	import Uzu
3
4	public func runChat() async throws {
5	let engineConfig = EngineConfig.create()
6	let engine = try await Engine.create(config: engineConfig)
7
8	guard let model = try await engine.model(identifier: "alibaba:qwen3.5:0.8b:mirai:mirai-m:4") else {
9	return
10	}
11	for try await update in try await engine.download(model: model).iterator() {
12	print(String(format: "\r\u{001B}[2KDownload progress: %.2f%%", update.progress() * 100), terminator: "")
13	fflush(stdout)
14	}
15	print()
16
17	let messages = [
18	ChatMessage.system().withText(text: "You are a helpful assistant"),
19	ChatMessage.user().withText(text: "Tell me a short, funny story about a robot")
20	]
21	let session = try await engine.chat(model: model, config: .create())
22	let stream = await session.replyWithStream(input: messages, config: .create())
23	var message: ChatMessage? = nil
24	for try await update in stream.iterator() {
25	switch update {
26	case .replies(let replies):
27	let reply = replies.last
28	message = reply?.message
29	print("Generated tokens: \(reply?.stats.tokensCountOutput ?? 0)")
30	case .error(let error):
31	print("Error: \(error)")
32	}
33	}
34	print("Reasoning: \(message?.reasoning() ?? "empty")")
35	print("Text: \(message?.text() ?? "empty")")
36	}
37

Details

An 8-bit quantized version of Qwen3-14B, optimized for Apple Silicon via the MLX framework. This model belongs to the Qwen3 generation of large language models from the Qwen team, delivering strong reasoning, multilingual, and agentic capabilities in a format tailored for efficient local inference on Mac hardware.

Key Features

Dual-Mode Reasoning: Qwen3 introduces seamless switching between a thinking mode (for complex math, coding, and logical reasoning) and a non-thinking mode (for fast, general-purpose dialogue) — all within a single model. Users can toggle modes via `enable_thinking` or inline `/think` and `/no_think` commands during conversation.
14.8B Parameters with a 40-layer transformer architecture using Grouped Query Attention (40 Q heads, 8 KV heads). Natively supports 32,768-token contexts, extensible to 131,072 tokens via YaRN rope scaling.
Multilingual Proficiency: Supports over 100 languages and dialects with strong instruction-following and translation capabilities.
Agent & Tool Use: Excels at structured tool calling in both thinking and non-thinking modes, with first-class support through the Qwen-Agent framework and MCP-compatible tool configurations.
Human Preference Alignment: Tuned for creative writing, role-playing, multi-turn dialogue, and nuanced instruction following.

Architecture & Provenance

Based on Qwen3-14B-Base, this variant has been quantized to 8-bit precision for the MLX ecosystem. It requires `mlx_lm ≥ 0.25.2` and `transformers ≥ 4.52.4`. The model is released under the Apache 2.0 license.

Ideal Use Cases

Local inference on Apple Silicon Macs
Complex reasoning tasks in math, coding, and logic
Multilingual chat and translation
Agentic workflows with external tool integration
Rapid prototyping without cloud dependencies

Explore all local models