mlx-community/gemma-3-1b-it-8bit
Run locally on Apple devices with Mirai
- Type
- local
- From
- Quantization
- MLX 8-bit
- Parameters
- 1B
- Size
- 1.0 GB
- Source
A compact, instruction-tuned language model optimized for Apple Silicon via the MLX framework. This is an 8-bit quantized conversion of Google's Gemma 3 1B IT, produced by the MLX Community using mlx-lm v0.21.6.
Overview
Gemma 3 1B is part of Google's Gemma family of lightweight, open-weight language models. The "IT" (instruction-tuned) variant is fine-tuned for conversational and instruction-following tasks, making it well-suited for chatbot applications, text generation, and general-purpose language understanding at a small footprint.
This MLX conversion brings the model into Apple's MLX ecosystem, enabling efficient on-device inference on Mac hardware with M-series chips. The 8-bit quantization significantly reduces memory usage compared to the full-precision original while retaining strong task performance — ideal for local, low-latency deployments.
Key Details
- Architecture: Gemma 3 (1B parameters)
- Variant: Instruction-tuned (chat-ready with built-in chat template)
- Quantization: 8-bit
- Framework: MLX (via `mlx-lm`)
- Base Model: `google/gemma-3-1b-it`
- License: Gemma (Google usage license; access requires acknowledgment)
Use Cases
- On-device text generation and chat on Apple Silicon Macs
- Lightweight assistant or copilot prototyping
- Edge deployment scenarios where memory and compute are constrained
- Rapid experimentation with a small but capable instruction-following model
Strengths
The 1B parameter size makes this one of the smallest models in the Gemma 3 lineup, striking a balance between capability and resource efficiency. Combined with 8-bit quantization and MLX optimization, it delivers fast inference with a minimal memory footprint — a practical choice for developers building local-first applications on macOS.
spm https://github.com/trymirai/uzu.git
| 1 | import Uzu |
| 2 | |
| 3 | public func runChat() async throws { |
| 4 | let engineConfig = EngineConfig.create() |
| 5 | let engine = try await Engine.create(config: engineConfig) |
| 6 | |
| 7 | guard let model = try await engine.model(identifier: "mlx-community/gemma-3-1b-it-8bit") else { |
| 8 | return |
| 9 | } |
| 10 | for try await update in try await engine.download(model: model).iterator() { |
| 11 | print("Download progress: \(update.progress())") |
| 12 | } |
| 13 | |
| 14 | let messages = [ |
| 15 | ChatMessage.system().withText(text: "You are a helpful assistant"), |
| 16 | ChatMessage.user().withText(text: "Tell me a short, funny story about a robot") |
| 17 | ] |
| 18 | let session = try await engine.chat(model: model, config: .create()) |
| 19 | let stream = await session.replyWithStream(input: messages, config: .create()) |
| 20 | var message: ChatMessage? = nil |
| 21 | for try await update in stream.iterator() { |
| 22 | switch update { |
| 23 | case .replies(let replies): |
| 24 | message = replies.last?.message |
| 25 | case .error(let error): |
| 26 | print("Error: \(error)") |
| 27 | } |
| 28 | } |
| 29 | print("Text: \(message?.text() ?? "empty")") |
| 30 | } |