Gemma-3

Name: mlx-community/gemma-3-1b-it-8bit
Author: Google

mlx-community/gemma-3-1b-it-8bit

Vendor: Google
Quantization: MLX 8-bit
Parameters: 1B
Size: 1.0 GB

$ brew install mirai$ mirai --model mlx-community/gemma-3-1b-it-8bit

Benchmarks

0.21 s

lower is better ↓

0.21 s

uzu

0.31 s

mlx

0.20 s

llamacpp

252 t/s

higher is better ↑

252 t/s

uzu

250 t/s

mlx

183 t/s

llamacpp

1.22 GB

lower is better ↓

1.22 GB

18.99 GB

1.46 GB

0.31.2 llama.cpp0.3.23/1,343 input tokens/382 output tokens

Benchmarked 7 Aug 2026

Integrate with SDK

Choose framework

Run the following command to install Mirai SDK

https://github.com/trymirai/uzu-swift

Apply code

1import Foundation2import Uzu34public func runChat() async throws {5    let engineConfig = EngineConfig.create()6    let engine = try await Engine.create(config: engineConfig)7    8    guard let model = try await engine.model(identifier: "alibaba:qwen3.5:0.8b:mirai:mirai-m:4") else {9        return10    }11    for try await update in try await engine.download(model: model).iterator() {12        print(String(format: "\r\u{001B}[2KDownload progress: %.2f%%", update.progress() * 100), terminator: "")13        fflush(stdout)14    }15    print()16    17    let messages = [18        ChatMessage.system().withText(text: "You are a helpful assistant"),19        ChatMessage.user().withText(text: "Tell me a short, funny story about a robot")20    ]21    let session = try await engine.chat(model: model, config: .create())22    let stream = await session.replyWithStream(input: messages, config: .create())23    var message: ChatMessage? = nil24    for try await update in stream.iterator() {25        switch update {26        case .replies(let replies):27            let reply = replies.last28            message = reply?.message29            print("Generated tokens: \(reply?.stats.tokensCountOutput ?? 0)")30        case .error(let error):31            print("Error: \(error)")32        }33    }34    print("Reasoning: \(message?.reasoning() ?? "empty")")35    print("Text: \(message?.text() ?? "empty")")36}37

1Choose framework

2Run the following command to install Mirai SDK

https://github.com/trymirai/uzu-swift

3Apply code

1	import Foundation
2	import Uzu
3
4	public func runChat() async throws {
5	let engineConfig = EngineConfig.create()
6	let engine = try await Engine.create(config: engineConfig)
7
8	guard let model = try await engine.model(identifier: "alibaba:qwen3.5:0.8b:mirai:mirai-m:4") else {
9	return
10	}
11	for try await update in try await engine.download(model: model).iterator() {
12	print(String(format: "\r\u{001B}[2KDownload progress: %.2f%%", update.progress() * 100), terminator: "")
13	fflush(stdout)
14	}
15	print()
16
17	let messages = [
18	ChatMessage.system().withText(text: "You are a helpful assistant"),
19	ChatMessage.user().withText(text: "Tell me a short, funny story about a robot")
20	]
21	let session = try await engine.chat(model: model, config: .create())
22	let stream = await session.replyWithStream(input: messages, config: .create())
23	var message: ChatMessage? = nil
24	for try await update in stream.iterator() {
25	switch update {
26	case .replies(let replies):
27	let reply = replies.last
28	message = reply?.message
29	print("Generated tokens: \(reply?.stats.tokensCountOutput ?? 0)")
30	case .error(let error):
31	print("Error: \(error)")
32	}
33	}
34	print("Reasoning: \(message?.reasoning() ?? "empty")")
35	print("Text: \(message?.text() ?? "empty")")
36	}
37

Details

A compact, instruction-tuned language model optimized for Apple Silicon via the MLX framework. This is an 8-bit quantized conversion of Google's Gemma 3 1B IT, produced by the MLX Community using mlx-lm v0.21.6.

Overview

Gemma 3 1B is part of Google's Gemma family of lightweight, open-weight language models. The "IT" (instruction-tuned) variant is fine-tuned for conversational and instruction-following tasks, making it well-suited for chatbot applications, text generation, and general-purpose language understanding at a small footprint.

This MLX conversion brings the model into Apple's MLX ecosystem, enabling efficient on-device inference on Mac hardware with M-series chips. The 8-bit quantization significantly reduces memory usage compared to the full-precision original while retaining strong task performance — ideal for local, low-latency deployments.

Key Details

Architecture: Gemma 3 (1B parameters)
Variant: Instruction-tuned (chat-ready with built-in chat template)
Quantization: 8-bit
Framework: MLX (via `mlx-lm`)
Base Model: `google/gemma-3-1b-it`
License: Gemma (Google usage license; access requires acknowledgment)

Use Cases

On-device text generation and chat on Apple Silicon Macs
Lightweight assistant or copilot prototyping
Edge deployment scenarios where memory and compute are constrained
Rapid experimentation with a small but capable instruction-following model

Strengths

The 1B parameter size makes this one of the smallest models in the Gemma 3 lineup, striking a balance between capability and resource efficiency. Combined with 8-bit quantization and MLX optimization, it delivers fast inference with a minimal memory footprint — a practical choice for developers building local-first applications on macOS.

Explore all local models