mlx-community/gemma-3-1b-it-4bit

Name: mlx-community/gemma-3-1b-it-4bit
Author: Google

Run locally on Apple devices with Mirai

Type: local
From: Google
Quantization: MLX 4-bit
Parameters: 1B
Size: 568.8 MB
Source

A 4-bit quantized version of Google's Gemma 3 1B Instruct model, converted to the Apple MLX framework for efficient on-device inference on Apple Silicon hardware.

Origin & Architecture

This model is a community conversion of google/gemma-3-1b-it, part of Google's Gemma 3 family of lightweight language models. At 1 billion parameters, it sits at the compact end of the Gemma lineup — designed for fast, resource-friendly text generation while retaining strong instruction-following capabilities. The 4-bit quantization further reduces memory footprint and accelerates inference, making it well-suited for local deployment on Mac laptops and desktops.

Key Details

Base model: Google Gemma 3 1B Instruct
Quantization: 4-bit (via `mlx-lm` v0.21.6)
Framework: Apple MLX
Task: Text generation (instruction-tuned)

Use Cases

This model is a strong fit for developers and researchers who want a lightweight, responsive chat or instruction-following model running natively on Apple Silicon — no cloud dependency required. Typical applications include:

Local chatbots and assistants
Quick prototyping of text generation pipelines
On-device summarization, rewriting, and Q&A
Educational exploration of LLM behavior

Considerations

The model is compact by design; for tasks demanding deeper reasoning or broader world knowledge, larger Gemma variants may be more appropriate. Access requires agreeing to Google's Gemma usage license on Hugging Face.

Explore all local models

Choose framework

Run the following command to install Mirai SDK

spm https://github.com/trymirai/uzu.git

Apply code

1import Uzu23public func runChat() async throws {4    let engineConfig = EngineConfig.create()5    let engine = try await Engine.create(config: engineConfig)67    guard let model = try await engine.model(identifier: "mlx-community/gemma-3-1b-it-4bit") else {8        return9    }10    for try await update in try await engine.download(model: model).iterator() {11        print("Download progress: \(update.progress())")12    }1314    let messages = [15        ChatMessage.system().withText(text: "You are a helpful assistant"),16        ChatMessage.user().withText(text: "Tell me a short, funny story about a robot")17    ]18    let session = try await engine.chat(model: model, config: .create())19    let stream = await session.replyWithStream(input: messages, config: .create())20    var message: ChatMessage? = nil21    for try await update in stream.iterator() {22        switch update {23        case .replies(let replies):24            message = replies.last?.message25        case .error(let error):26            print("Error: \(error)")27        }28    }29    print("Text: \(message?.text() ?? "empty")")30}

1Choose framework

2Run the following command to install Mirai SDK

spm https://github.com/trymirai/uzu.git

3Apply code

1	import Uzu
2
3	public func runChat() async throws {
4	let engineConfig = EngineConfig.create()
5	let engine = try await Engine.create(config: engineConfig)
6
7	guard let model = try await engine.model(identifier: "mlx-community/gemma-3-1b-it-4bit") else {
8	return
9	}
10	for try await update in try await engine.download(model: model).iterator() {
11	print("Download progress: \(update.progress())")
12	}
13
14	let messages = [
15	ChatMessage.system().withText(text: "You are a helpful assistant"),
16	ChatMessage.user().withText(text: "Tell me a short, funny story about a robot")
17	]
18	let session = try await engine.chat(model: model, config: .create())
19	let stream = await session.replyWithStream(input: messages, config: .create())
20	var message: ChatMessage? = nil
21	for try await update in stream.iterator() {
22	switch update {
23	case .replies(let replies):
24	message = replies.last?.message
25	case .error(let error):
26	print("Error: \(error)")
27	}
28	}
29	print("Text: \(message?.text() ?? "empty")")
30	}