mlx-community/gemma-3-27b-it-8bit
Run locally on Apple devices with Mirai
- Type
- local
- From
- Quantization
- MLX 8-bit
- Parameters
- 27B
- Size
- 26.8 GB
- Source
An 8-bit quantized conversion of Google's Gemma 3 27B Instruct model, optimized for Apple Silicon via the MLX framework. This community conversion was produced using mlx-vlm v0.1.18 and is hosted by the mlx-community organization on Hugging Face.
Model Overview
Gemma 3 27B IT is a 27-billion-parameter instruction-tuned model from Google, built on the Gemma 3 architecture. It supports multimodal image-text-to-text tasks — meaning it can accept both images and text as input and generate text responses. This makes it suitable for visual question answering, image captioning, document understanding, and general-purpose instruction following.
The 8-bit quantization significantly reduces the memory footprint compared to the full-precision model, making it feasible to run locally on high-end Apple Silicon machines (e.g., M2 Ultra, M3 Max, M4 Max/Ultra with sufficient unified memory).
Key Details
- Base model: `google/gemma-3-27b-pt` (pretrained), instruction-tuned as `google/gemma-3-27b-it`
- Format: MLX-native weights, 8-bit quantized
- Pipeline: Image-text-to-text (multimodal)
- Framework: Compatible with `mlx-vlm` for inference on macOS
Use Cases
- Visual reasoning: Describe, analyze, or answer questions about images
- Instruction following: General conversational and task-completion capabilities at the 27B parameter scale
- Local inference: Run a powerful multimodal model on-device without cloud dependencies
Access
This model is gated under Google's Gemma usage license. You must review and acknowledge the license on Hugging Face before downloading. Refer to the original model card for full details on capabilities, training data, and intended use.
spm https://github.com/trymirai/uzu.git
| 1 | import Uzu |
| 2 | |
| 3 | public func runChat() async throws { |
| 4 | let engineConfig = EngineConfig.create() |
| 5 | let engine = try await Engine.create(config: engineConfig) |
| 6 | |
| 7 | guard let model = try await engine.model(identifier: "mlx-community/gemma-3-27b-it-8bit") else { |
| 8 | return |
| 9 | } |
| 10 | for try await update in try await engine.download(model: model).iterator() { |
| 11 | print("Download progress: \(update.progress())") |
| 12 | } |
| 13 | |
| 14 | let messages = [ |
| 15 | ChatMessage.system().withText(text: "You are a helpful assistant"), |
| 16 | ChatMessage.user().withText(text: "Tell me a short, funny story about a robot") |
| 17 | ] |
| 18 | let session = try await engine.chat(model: model, config: .create()) |
| 19 | let stream = await session.replyWithStream(input: messages, config: .create()) |
| 20 | var message: ChatMessage? = nil |
| 21 | for try await update in stream.iterator() { |
| 22 | switch update { |
| 23 | case .replies(let replies): |
| 24 | message = replies.last?.message |
| 25 | case .error(let error): |
| 26 | print("Error: \(error)") |
| 27 | } |
| 28 | } |
| 29 | print("Text: \(message?.text() ?? "empty")") |
| 30 | } |