Qwen/Qwen3-1.7B-MLX-4bit

Name: Qwen/Qwen3-1.7B-MLX-4bit
Author: Alibaba

Run locally on Apple devices with Mirai

Type: local
From: Alibaba
Quantization: MLX 4-bit
Parameters: 1.7B
Size: 883.2 MB
Source

A 4-bit quantized version of Qwen3-1.7B optimized for Apple Silicon via the MLX framework. Part of Alibaba's third-generation Qwen language model family, this compact model brings advanced reasoning and conversational capabilities to local Mac-based inference with a minimal memory footprint.

Key Features

Qwen3-1.7B-MLX-4bit inherits the standout capabilities of the Qwen3 series in a lightweight package:

Dual-mode reasoning: Seamlessly switch between a thinking mode (for complex math, coding, and logical reasoning) and a non-thinking mode (for fast, general-purpose dialogue) within a single model. Thinking mode wraps intermediate reasoning in `<think>...</think>` blocks before delivering a final answer.
Multilingual fluency: Supports over 100 languages and dialects, with strong multilingual instruction-following and translation performance.
Agent and tool-calling support: Designed for agentic workflows, including integration with external tools and MCP-compatible servers via frameworks like Qwen-Agent.
Soft switching: Users can toggle thinking behavior mid-conversation using `/think` and `/no_think` commands in prompts, enabling fine-grained control over response style per turn.

Architecture

Type: Causal language model (dense)
Parameters: 1.7B total (1.4B non-embedding)
Layers: 28
Attention: Grouped-Query Attention (16 Q heads, 8 KV heads)
Context length: 32,768 tokens
Quantization: 4-bit (MLX)
Base model: Qwen3-1.7B-Base (pretrained + post-trained)

Use Cases

This model is well-suited for on-device inference on Apple Silicon Macs — ideal for developers and researchers who want local, privacy-preserving access to a capable small language model. It handles creative writing, role-playing, multi-turn dialogue, code generation, and tool-augmented tasks. Compatible with `mlx_lm` (≥ 0.25.2) and `transformers` (≥ 4.52.4).

1	import Uzu
2
3	public func runChat() async throws {
4	let engineConfig = EngineConfig.create()
5	let engine = try await Engine.create(config: engineConfig)
6
7	guard let model = try await engine.model(identifier: "Qwen/Qwen3-1.7B-MLX-4bit") else {
8	return
9	}
10	for try await update in try await engine.download(model: model).iterator() {
11	print("Download progress: \(update.progress())")
12	}
13
14	let messages = [
15	ChatMessage.system().withText(text: "You are a helpful assistant"),
16	ChatMessage.user().withText(text: "Tell me a short, funny story about a robot")
17	]
18	let session = try await engine.chat(model: model, config: .create())
19	let stream = await session.replyWithStream(input: messages, config: .create())
20	var message: ChatMessage? = nil
21	for try await update in stream.iterator() {
22	switch update {
23	case .replies(let replies):
24	message = replies.last?.message
25	case .error(let error):
26	print("Error: \(error)")
27	}
28	}
29	print("Text: \(message?.text() ?? "empty")")
30	}

Key Features

Architecture

Use Cases

Other local models from Alibaba