Qwen/Qwen3-4B-MLX-4bit
Run locally on Apple devices with Mirai
- Type
- local
- From
- Alibaba
- Quantization
- MLX 4-bit
- Parameters
- 4B
- Size
- 2.0 GB
- Source
Qwen3-4B-MLX-4bit is a 4-bit quantized version of the Qwen3-4B language model, optimized for efficient inference on Apple Silicon using the MLX framework. Part of the latest Qwen3 generation from Alibaba's Qwen team, this model brings advanced reasoning and conversational capabilities to local Mac-based workflows.
Key Capabilities
Qwen3-4B introduces a distinctive dual-mode architecture that supports seamless switching between a thinking mode — designed for complex reasoning, math, and coding tasks — and a non-thinking mode for fast, general-purpose dialogue. This toggle can be controlled at the API level or dynamically within a conversation using `/think` and `/no_think` user commands.
Despite its compact 4.0B parameter size (3.6B non-embedding), the model punches above its weight across several dimensions:
- Reasoning: Enhanced logical, mathematical, and code generation performance surpassing prior Qwen2.5 instruct models in non-thinking mode.
- Multilingual support: Coverage of 100+ languages and dialects, with strong translation and multilingual instruction-following ability.
- Agent and tool use: First-class support for external tool integration via frameworks like Qwen-Agent, including MCP server configuration and built-in tools such as a code interpreter.
- Human preference alignment: Improved creative writing, role-playing, and multi-turn dialogue quality.
Architecture Details
- Type: Causal language model (decoder-only)
- Layers: 36
- Attention: Grouped-query attention (32 Q heads, 8 KV heads)
- Context Length: 32,768 tokens natively; up to 131,072 tokens with YaRN RoPE scaling
- Quantization: 4-bit (MLX format)
Use Cases
This model is well suited for developers and researchers running inference locally on Apple Silicon Macs who need a responsive, lightweight model with strong reasoning and multilingual capabilities. It pairs naturally with `mlx_lm` (≥ 0.25.2) for straightforward local generation. Licensed under Apache 2.0.
spm https://github.com/trymirai/uzu.git
| 1 | import Uzu |
| 2 | |
| 3 | public func runChat() async throws { |
| 4 | let engineConfig = EngineConfig.create() |
| 5 | let engine = try await Engine.create(config: engineConfig) |
| 6 | |
| 7 | guard let model = try await engine.model(identifier: "Qwen/Qwen3-4B-MLX-4bit") else { |
| 8 | return |
| 9 | } |
| 10 | for try await update in try await engine.download(model: model).iterator() { |
| 11 | print("Download progress: \(update.progress())") |
| 12 | } |
| 13 | |
| 14 | let messages = [ |
| 15 | ChatMessage.system().withText(text: "You are a helpful assistant"), |
| 16 | ChatMessage.user().withText(text: "Tell me a short, funny story about a robot") |
| 17 | ] |
| 18 | let session = try await engine.chat(model: model, config: .create()) |
| 19 | let stream = await session.replyWithStream(input: messages, config: .create()) |
| 20 | var message: ChatMessage? = nil |
| 21 | for try await update in stream.iterator() { |
| 22 | switch update { |
| 23 | case .replies(let replies): |
| 24 | message = replies.last?.message |
| 25 | case .error(let error): |
| 26 | print("Error: \(error)") |
| 27 | } |
| 28 | } |
| 29 | print("Text: \(message?.text() ?? "empty")") |
| 30 | } |