Qwen/Qwen3-1.7B-MLX-8bit
Run locally on Apple devices with Mirai
- Type
- local
- From
- Alibaba
- Quantization
- MLX 8-bit
- Parameters
- 1.7B
- Size
- 1.7 GB
- Source
Qwen3-1.7B-MLX-8bit is an 8-bit quantized version of Qwen's third-generation 1.7B-parameter language model, optimized for Apple Silicon via the MLX framework. It delivers the full capabilities of Qwen3 in a compact, efficient package suited for on-device inference on Mac hardware.
Architecture & Specifications
Built on a causal language model architecture, Qwen3-1.7B features 28 layers with grouped-query attention (16 Q heads, 8 KV heads) and supports a context length of 32,768 tokens. The base model contains 1.7B total parameters (1.4B non-embedding), making it one of the most lightweight entries in the Qwen3 lineup. The 8-bit MLX quantization further reduces memory footprint while preserving quality.
Thinking and Non-Thinking Modes
A standout feature of Qwen3 is its ability to seamlessly switch between thinking mode — where the model reasons step-by-step through complex math, logic, and coding problems — and non-thinking mode, which provides fast, direct responses for general conversation. This can be toggled via `enable_thinking` in the chat template, or dynamically controlled mid-conversation using `/think` and `/no_think` tags in user messages.
Key Capabilities
- Reasoning: Enhanced performance on mathematics, code generation, and commonsense reasoning tasks compared to prior Qwen generations.
- Agent & Tool Use: Strong integration with external tools and MCP-based workflows, supported natively through Qwen-Agent.
- Multilingual: Supports over 100 languages and dialects, with robust instruction-following and translation abilities.
- Conversational Quality: Improved alignment for creative writing, role-playing, multi-turn dialogue, and instruction following.
Ideal Use Cases
This model is well-suited for developers building lightweight, on-device AI applications on macOS — particularly those needing a balance of reasoning depth and fast inference. It requires `mlx_lm` ≥ 0.25.2 and `transformers` ≥ 4.52.4. Released under the Apache 2.0 license by the Qwen Team at Alibaba.
spm https://github.com/trymirai/uzu.git
| 1 | import Uzu |
| 2 | |
| 3 | public func runChat() async throws { |
| 4 | let engineConfig = EngineConfig.create() |
| 5 | let engine = try await Engine.create(config: engineConfig) |
| 6 | |
| 7 | guard let model = try await engine.model(identifier: "Qwen/Qwen3-1.7B-MLX-8bit") else { |
| 8 | return |
| 9 | } |
| 10 | for try await update in try await engine.download(model: model).iterator() { |
| 11 | print("Download progress: \(update.progress())") |
| 12 | } |
| 13 | |
| 14 | let messages = [ |
| 15 | ChatMessage.system().withText(text: "You are a helpful assistant"), |
| 16 | ChatMessage.user().withText(text: "Tell me a short, funny story about a robot") |
| 17 | ] |
| 18 | let session = try await engine.chat(model: model, config: .create()) |
| 19 | let stream = await session.replyWithStream(input: messages, config: .create()) |
| 20 | var message: ChatMessage? = nil |
| 21 | for try await update in stream.iterator() { |
| 22 | switch update { |
| 23 | case .replies(let replies): |
| 24 | message = replies.last?.message |
| 25 | case .error(let error): |
| 26 | print("Error: \(error)") |
| 27 | } |
| 28 | } |
| 29 | print("Text: \(message?.text() ?? "empty")") |
| 30 | } |