Skip to main content
Alibaba

Qwen3.8 27B

trymirai/Qwen3.8-27B-XS on Hugging Face
Vendor
Alibaba
Quantization
Mirai-XS
Parameters
27B
Size
7.2 GB
$ brew install mirai$ mirai --model trymirai/Qwen3.8-27B-XS

Integrate with SDK

1
Choose framework
2
Run the following command to install Mirai SDK
https://github.com/trymirai/uzu
3
Apply code
1import Foundation2import Uzu3​4public func runChat() async throws {5    let engineConfig = EngineConfig.create()6    let engine = try await Engine.create(config: engineConfig)7    8    guard let model = try await engine.model(identifier: "trymirai/Qwen3.8-27B-XS") else {9        return10    }11    for try await update in try await engine.download(model: model).iterator() {12        print(String(format: "\r\u{001B}[2KDownload progress: %.2f%%", update.progress() * 100), terminator: "")13        fflush(stdout)14    }15    print()16    17    let messages = [18        ChatMessage.system().withText(text: "You are a helpful assistant"),19        ChatMessage.user().withText(text: "Tell me a short, funny story about a robot")20    ]21    let session = try await engine.chat(model: model, config: .create())22    let stream = await session.replyWithStream(input: messages, config: .create())23    var message: ChatMessage? = nil24    for try await update in stream.iterator() {25        switch update {26        case .replies(let replies):27            let reply = replies.last28            message = reply?.message29            print("Generated tokens: \(reply?.stats.tokensCountOutput ?? 0)")30        case .error(let error):31            print("Error: \(error)")32        case .toolResults:33            break34        }35    }36    print("Reasoning: \(message?.reasoning() ?? "empty")")37    print("Text: \(message?.text() ?? "empty")")38}

Details

An extra-small, high-quality quantization of Qwen3.8-27B prepared for efficient local inference on Apple silicon.

Mirai Labs

Evaluation pending

Quickstart

If you are on macOS, the easiest way is to install the mirai Homebrew package and then run the CLI:

sh
brew install mirai
mirai --model trymirai/Qwen3.8-27B-XS

Currently only Apple silicon inference is supported. If you want to build things from source, read this overview.


Method

Mirai Extra-Small uses QTIP-style trellis-coded quantization with 8, 7 or 6 transition bits per group of four weights, chosen per matrix by measured cost (about 2 bits per weight). Kronecker-structured random rotations are used to reduce weight outliers. The input embedding uses a 2-bit D4 lattice code, and the output head uses 4-bit integers with group size 64. The checkpoint was fit against a calibration set with reasoning and agent text, then prepared with two rounds of quantization-aware distillation and a long-context round that retrains only the row and norm scales on the model's own long agent sessions.

Citation

If you find our work helpful, feel free to give us a cite.

bibtex
@misc{mirai-quant,
    title  = {{Mirai Quantization}: Redefining the speed-quality frontier for local LLMs on Apple silicon},
    author = {Artur Chakhvadze and Ryan Mathieu and Roman Knyazhitskiy and Nikolai Voinilenko and Chen-Chen Yeh and Artur Mullakhmetov and Eugene Bokhan and others},
    note   = {In collaboration with others at Mirai Labs},
    month  = {June},
    year   = {2026},
    url    = {https://trymirai.com/blog/quantization}
}

Original model

This is a quantized version of Qwen/Qwen3.8-27B. For architecture details, intended use, evaluations, and limitations, see the original model card.

Explore all local models