Qwen/Qwen3-14B-MLX-4bit
Run locally on Apple devices with Mirai
- Type
- local
- From
- Alibaba
- Quantization
- MLX 4-bit
- Parameters
- 14B
- Size
- 7.3 GB
- Source
A 4-bit quantized version of Qwen3-14B, optimized for Apple Silicon via the MLX framework. This model brings the full capabilities of Qwen's latest-generation language model to Mac hardware with significantly reduced memory requirements.
What It Is
Qwen3-14B-MLX-4bit is a causal language model with 14.8 billion parameters (13.2B non-embedding), built on a 40-layer transformer architecture with grouped-query attention (40 Q heads, 8 KV heads). It supports a native context length of 32,768 tokens, extendable to 131,072 tokens via YaRN rope scaling. The base model, Qwen3-14B, was developed by the Qwen team at Alibaba and underwent both pretraining and post-training stages before being quantized to 4-bit precision for efficient MLX inference.
Key Capabilities
- Dual Thinking Modes: Seamlessly switch between a *thinking mode* for complex reasoning tasks (math, code, logic) and a *non-thinking mode* for fast, general-purpose conversation — all within a single model. Thinking mode wraps internal reasoning in `<think>...</think>` blocks before delivering a final answer.
- Strong Reasoning: Surpasses both QwQ (in thinking mode) and Qwen2.5 Instruct (in non-thinking mode) on mathematics, code generation, and commonsense reasoning benchmarks.
- Agent & Tool Use: First-class support for tool calling and agentic workflows, compatible with frameworks like Qwen-Agent and MCP server configurations.
- Multilingual: Supports 100+ languages and dialects, with robust multilingual instruction-following and translation capabilities.
- Human Preference Alignment: Tuned for creative writing, role-playing, multi-turn dialogue, and precise instruction following.
Usage Notes
Requires `mlx_lm ≥ 0.25.2` and `transformers ≥ 4.52.4`. Recommended sampling parameters differ by mode — thinking mode works best with Temperature 0.6 and TopP 0.95, while non-thinking mode favors Temperature 0.7 and TopP 0.8. Greedy decoding should be avoided in thinking mode.
Licensed under Apache 2.0.
spm https://github.com/trymirai/uzu.git
| 1 | import Uzu |
| 2 | |
| 3 | public func runChat() async throws { |
| 4 | let engineConfig = EngineConfig.create() |
| 5 | let engine = try await Engine.create(config: engineConfig) |
| 6 | |
| 7 | guard let model = try await engine.model(identifier: "Qwen/Qwen3-14B-MLX-4bit") else { |
| 8 | return |
| 9 | } |
| 10 | for try await update in try await engine.download(model: model).iterator() { |
| 11 | print("Download progress: \(update.progress())") |
| 12 | } |
| 13 | |
| 14 | let messages = [ |
| 15 | ChatMessage.system().withText(text: "You are a helpful assistant"), |
| 16 | ChatMessage.user().withText(text: "Tell me a short, funny story about a robot") |
| 17 | ] |
| 18 | let session = try await engine.chat(model: model, config: .create()) |
| 19 | let stream = await session.replyWithStream(input: messages, config: .create()) |
| 20 | var message: ChatMessage? = nil |
| 21 | for try await update in stream.iterator() { |
| 22 | switch update { |
| 23 | case .replies(let replies): |
| 24 | message = replies.last?.message |
| 25 | case .error(let error): |
| 26 | print("Error: \(error)") |
| 27 | } |
| 28 | } |
| 29 | print("Text: \(message?.text() ?? "empty")") |
| 30 | } |