Qwen/Qwen3-14B-MLX-8bit

Name: Qwen/Qwen3-14B-MLX-8bit
Author: Alibaba

Run locally on Apple devices with Mirai

Type: local
From: Alibaba
Quantization: MLX 8-bit
Parameters: 14B
Size: 14.2 GB
Source

An 8-bit quantized version of Qwen3-14B, optimized for Apple Silicon via the MLX framework. This model belongs to the Qwen3 generation of large language models from the Qwen team, delivering strong reasoning, multilingual, and agentic capabilities in a format tailored for efficient local inference on Mac hardware.

Key Features

Dual-Mode Reasoning: Qwen3 introduces seamless switching between a thinking mode (for complex math, coding, and logical reasoning) and a non-thinking mode (for fast, general-purpose dialogue) — all within a single model. Users can toggle modes via `enable_thinking` or inline `/think` and `/no_think` commands during conversation.
14.8B Parameters with a 40-layer transformer architecture using Grouped Query Attention (40 Q heads, 8 KV heads). Natively supports 32,768-token contexts, extensible to 131,072 tokens via YaRN rope scaling.
Multilingual Proficiency: Supports over 100 languages and dialects with strong instruction-following and translation capabilities.
Agent & Tool Use: Excels at structured tool calling in both thinking and non-thinking modes, with first-class support through the Qwen-Agent framework and MCP-compatible tool configurations.
Human Preference Alignment: Tuned for creative writing, role-playing, multi-turn dialogue, and nuanced instruction following.

Architecture & Provenance

Based on Qwen3-14B-Base, this variant has been quantized to 8-bit precision for the MLX ecosystem. It requires `mlx_lm ≥ 0.25.2` and `transformers ≥ 4.52.4`. The model is released under the Apache 2.0 license.

1	import Uzu
2
3	public func runChat() async throws {
4	let engineConfig = EngineConfig.create()
5	let engine = try await Engine.create(config: engineConfig)
6
7	guard let model = try await engine.model(identifier: "Qwen/Qwen3-14B-MLX-8bit") else {
8	return
9	}
10	for try await update in try await engine.download(model: model).iterator() {
11	print("Download progress: \(update.progress())")
12	}
13
14	let messages = [
15	ChatMessage.system().withText(text: "You are a helpful assistant"),
16	ChatMessage.user().withText(text: "Tell me a short, funny story about a robot")
17	]
18	let session = try await engine.chat(model: model, config: .create())
19	let stream = await session.replyWithStream(input: messages, config: .create())
20	var message: ChatMessage? = nil
21	for try await update in stream.iterator() {
22	switch update {
23	case .replies(let replies):
24	message = replies.last?.message
25	case .error(let error):
26	print("Error: \(error)")
27	}
28	}
29	print("Text: \(message?.text() ?? "empty")")
30	}

Key Features

Architecture & Provenance

Ideal Use Cases

Other local models from Alibaba