Qwen 3

Name: Qwen/Qwen3-8B
Author: Alibaba

Qwen/Qwen3-8B

Vendor: Alibaba
Parameters: 8B
Size: 15.3 GB

$ brew install mirai$ mirai --model Qwen/Qwen3-8B

Benchmarks

1.37 s

lower is better ↓

1.37 s

uzu

1.56 s

mlx

1.44 s

llamacpp

32 t/s

higher is better ↑

32 t/s

uzu

29 t/s

mlx

29 t/s

llamacpp

16.20 GB

lower is better ↓

16.20 GB

18.99 GB

15.49 GB

0.31.2 llama.cpp0.3.23/1,312 input tokens/512 output tokens

Benchmarked 7 Aug 2026

Integrate with SDK

Choose framework

Run the following command to install Mirai SDK

https://github.com/trymirai/uzu-swift

Apply code

1import Foundation2import Uzu34public func runChat() async throws {5    let engineConfig = EngineConfig.create()6    let engine = try await Engine.create(config: engineConfig)7    8    guard let model = try await engine.model(identifier: "alibaba:qwen3.5:0.8b:mirai:mirai-m:4") else {9        return10    }11    for try await update in try await engine.download(model: model).iterator() {12        print(String(format: "\r\u{001B}[2KDownload progress: %.2f%%", update.progress() * 100), terminator: "")13        fflush(stdout)14    }15    print()16    17    let messages = [18        ChatMessage.system().withText(text: "You are a helpful assistant"),19        ChatMessage.user().withText(text: "Tell me a short, funny story about a robot")20    ]21    let session = try await engine.chat(model: model, config: .create())22    let stream = await session.replyWithStream(input: messages, config: .create())23    var message: ChatMessage? = nil24    for try await update in stream.iterator() {25        switch update {26        case .replies(let replies):27            let reply = replies.last28            message = reply?.message29            print("Generated tokens: \(reply?.stats.tokensCountOutput ?? 0)")30        case .error(let error):31            print("Error: \(error)")32        }33    }34    print("Reasoning: \(message?.reasoning() ?? "empty")")35    print("Text: \(message?.text() ?? "empty")")36}37

1Choose framework

2Run the following command to install Mirai SDK

https://github.com/trymirai/uzu-swift

3Apply code

1	import Foundation
2	import Uzu
3
4	public func runChat() async throws {
5	let engineConfig = EngineConfig.create()
6	let engine = try await Engine.create(config: engineConfig)
7
8	guard let model = try await engine.model(identifier: "alibaba:qwen3.5:0.8b:mirai:mirai-m:4") else {
9	return
10	}
11	for try await update in try await engine.download(model: model).iterator() {
12	print(String(format: "\r\u{001B}[2KDownload progress: %.2f%%", update.progress() * 100), terminator: "")
13	fflush(stdout)
14	}
15	print()
16
17	let messages = [
18	ChatMessage.system().withText(text: "You are a helpful assistant"),
19	ChatMessage.user().withText(text: "Tell me a short, funny story about a robot")
20	]
21	let session = try await engine.chat(model: model, config: .create())
22	let stream = await session.replyWithStream(input: messages, config: .create())
23	var message: ChatMessage? = nil
24	for try await update in stream.iterator() {
25	switch update {
26	case .replies(let replies):
27	let reply = replies.last
28	message = reply?.message
29	print("Generated tokens: \(reply?.stats.tokensCountOutput ?? 0)")
30	case .error(let error):
31	print("Error: \(error)")
32	}
33	}
34	print("Reasoning: \(message?.reasoning() ?? "empty")")
35	print("Text: \(message?.text() ?? "empty")")
36	}
37

Details

Qwen3-8B is a dense 8.2 billion parameter causal language model from Alibaba's Qwen team, representing the latest generation in the Qwen series. Built on the Qwen3-8B-Base and refined through both pretraining and post-training, it introduces a distinctive dual-mode architecture that sets it apart from typical instruction-tuned models.

Thinking and Non-Thinking Modes

Qwen3-8B's headline feature is seamless switching between thinking mode and non-thinking mode within a single model. In thinking mode, the model engages in explicit chain-of-thought reasoning — ideal for complex math, code generation, and logical problem-solving — wrapping its internal reasoning in `<think>...</think>` blocks before delivering a final answer. In non-thinking mode, it operates as a streamlined conversational model akin to Qwen2.5-Instruct, prioritizing efficiency and directness. Users can toggle between modes via API parameters or even inline `/think` and `/no_think` commands within a conversation.

Architecture & Context

Parameters: 8.2B total (6.95B non-embedding)
Layers: 36, with Grouped Query Attention (32 Q heads, 8 KV heads)
Context Length: 32,768 tokens natively, extensible to 131,072 tokens via YaRN RoPE scaling
License: Apache 2.0

Key Capabilities

Reasoning: Surpasses QwQ (thinking mode) and Qwen2.5-Instruct (non-thinking mode) on math, code, and commonsense reasoning benchmarks.
Agent & Tool Use: Strong function-calling and tool integration abilities, compatible with MCP configurations and the Qwen-Agent framework for complex agentic workflows.
Multilingual: Supports over 100 languages and dialects with robust multilingual instruction-following and translation.
Human Preference Alignment: Excels in creative writing, role-playing, multi-turn dialogue, and instruction following.

Deployment

Qwen3-8B is compatible with Hugging Face Transformers (v4.51.0+), vLLM, SGLang, Ollama, LM Studio, llama.cpp, and other popular inference frameworks, making it straightforward to deploy as an OpenAI-compatible API or run locally.

Explore all local models