Qwen/Qwen3-0.6B-MLX-4bit

Name: Qwen/Qwen3-0.6B-MLX-4bit
Author: Alibaba

Run locally on Apple devices with Mirai

Type: local
From: Alibaba
Quantization: MLX 4-bit
Parameters: 600M
Size: 313.3 MB
Source

A compact, 4-bit quantized version of Qwen3-0.6B optimized for Apple Silicon via the MLX framework. Published by the Qwen team, this model brings the latest-generation Qwen3 capabilities to efficient on-device inference on Mac hardware.

Key Features

Qwen3-0.6B-MLX-4bit inherits the standout capabilities of the Qwen3 family in a remarkably small footprint:

Dual-mode reasoning: Seamlessly switch between a thinking mode (for step-by-step logical reasoning, math, and code) and a non-thinking mode (for fast, general-purpose dialogue) — all within a single model. Users can toggle behavior via `enable_thinking` or inline `/think` and `/no_think` tags in conversation.
Multilingual support: Covers 100+ languages and dialects with strong instruction-following and translation performance.
Agent & tool-calling capabilities: Designed for integration with external tools, compatible with frameworks like Qwen-Agent, and capable of handling complex agentic workflows.

Architecture

Type: Causal language model (dense transformer)
Parameters: 0.6B total (0.44B non-embedding)
Layers: 28, with grouped-query attention (16 Q heads, 8 KV heads)
Context length: 32,768 tokens
Quantization: 4-bit (MLX format)
Base model: Qwen3-0.6B-Base, with full pretraining and post-training

Intended Use

This model is ideal for developers building lightweight, privacy-friendly applications on Apple Silicon — think local chatbots, coding assistants, creative writing tools, or multilingual agents — where low latency and minimal memory consumption matter. It requires `mlx_lm` ≥ 0.25.2 and `transformers` ≥ 4.52.4.

1	import Uzu
2
3	public func runChat() async throws {
4	let engineConfig = EngineConfig.create()
5	let engine = try await Engine.create(config: engineConfig)
6
7	guard let model = try await engine.model(identifier: "Qwen/Qwen3-0.6B-MLX-4bit") else {
8	return
9	}
10	for try await update in try await engine.download(model: model).iterator() {
11	print("Download progress: \(update.progress())")
12	}
13
14	let messages = [
15	ChatMessage.system().withText(text: "You are a helpful assistant"),
16	ChatMessage.user().withText(text: "Tell me a short, funny story about a robot")
17	]
18	let session = try await engine.chat(model: model, config: .create())
19	let stream = await session.replyWithStream(input: messages, config: .create())
20	var message: ChatMessage? = nil
21	for try await update in stream.iterator() {
22	switch update {
23	case .replies(let replies):
24	message = replies.last?.message
25	case .error(let error):
26	print("Error: \(error)")
27	}
28	}
29	print("Text: \(message?.text() ?? "empty")")
30	}

Key Features

Architecture

Intended Use

Provenance

Other local models from Alibaba