Skip to main content
Alibaba

Qwen3.5 9B

trymirai/Qwen3.5-9B-Dash on Hugging Face
Vendor
Alibaba
Quantization
Mirai-DASH
Parameters
9B
Size
2.8 GB
$ brew install mirai$ mirai --model trymirai/Qwen3.5-9B-Dash

Integrate with SDK

1
Choose framework
2
Run the following command to install Mirai SDK
https://github.com/trymirai/uzu
3
Apply code
1import Foundation2import Uzu3​4public func runChat() async throws {5    let engineConfig = EngineConfig.create()6    let engine = try await Engine.create(config: engineConfig)7    8    guard let model = try await engine.model(identifier: "trymirai/Qwen3.5-9B-Dash") else {9        return10    }11    for try await update in try await engine.download(model: model).iterator() {12        print(String(format: "\r\u{001B}[2KDownload progress: %.2f%%", update.progress() * 100), terminator: "")13        fflush(stdout)14    }15    print()16    17    let messages = [18        ChatMessage.system().withText(text: "You are a helpful assistant"),19        ChatMessage.user().withText(text: "Tell me a short, funny story about a robot")20    ]21    let session = try await engine.chat(model: model, config: .create())22    let stream = await session.replyWithStream(input: messages, config: .create())23    var message: ChatMessage? = nil24    for try await update in stream.iterator() {25        switch update {26        case .replies(let replies):27            let reply = replies.last28            message = reply?.message29            print("Generated tokens: \(reply?.stats.tokensCountOutput ?? 0)")30        case .error(let error):31            print("Error: \(error)")32        case .toolResults:33            break34        }35    }36    print("Reasoning: \(message?.reasoning() ?? "empty")")37    print("Text: \(message?.text() ?? "empty")")38}

Details

A packed 2-bit Dash quantization of Qwen3.5-9B with INT8 attention key/value rows, mixed-precision vocabulary tables, and an adapted 2-bit speculator with a learned confidence head.

Mirai Labs

KL divergence versus language-model weight size

Evaluated with romakl on its default set: 20 conversations and 14,399 teacher-generated continuation tokens. This checkpoint: KL 0.1256530; top-1 agreement 88.98%. The plot shows checkpoints within ±30% in token-generating weight size and ¼–4× this checkpoint's KL; arrows mark MLX results outside the window. Vision, MTP and speculator weights are excluded from size. No tested GGUF checkpoint falls within this size window.

Quickstart

If you are on macOS, the easiest way is to install the mirai Homebrew package and then run the CLI:

sh
brew install mirai
mirai --model trymirai/Qwen3.5-9B-Dash

Currently only Apple silicon inference is supported. If you want to build things from source, read this overview.

Learn more →


Method

Dash stores packed 2-bit MLX weights with Hadamard-128 transforms for its transformer projections. Attention key/value rows use INT8. Each vocabulary table combines 245,824 2-bit rows with group size 128 and 2,496 INT8 rows. Quantization scales and affine biases are bfloat16. The vocabulary tables and tokenizer use the same reordered token IDs. This checkpoint is the packed kvs-dash model at quantization-aware training step 4,000.

Speculative decoding

The speculator/ directory contains a six-layer DFlash draft model, a one-layer Weaver model for tree construction, and a learned confidence head for speculative decoding. The DFlash and Weaver projections use 2-bit MLX quantization with group size 128 and Hadamard-32 input/output transforms.

The Dash speculator and confidence head are adapted to this packed Dash checkpoint. Its mask token is aligned with the reordered tokenizer.

Citation

If you find our work helpful, feel free to give us a cite.

bibtex
@misc{mirai-quant,
    title  = {{Mirai Quantization}: Redefining the speed-quality frontier for local LLMs on Apple silicon},
    author = {Artur Chakhvadze and Ryan Mathieu and Roman Knyazhitskiy and Nikolai Voinilenko and Chen-Chen Yeh and Artur Mullakhmetov and Eugene Bokhan and others},
    note   = {In collaboration with others at Mirai Labs},
    month  = {June},
    year   = {2026},
    url    = {https://trymirai.com/blog/quantization}
}

Original model

This is a quantized version of Qwen/Qwen3.5-9B. For architecture details, intended use, evaluations, and limitations, see the original model card.

Explore all local models