mlx-community/gemma-3-27b-it-4bit
Run locally on Apple devices with Mirai
- Type
- local
- From
- Quantization
- MLX 4-bit
- Parameters
- 27B
- Size
- 14.2 GB
- Source
A 4-bit quantized, MLX-optimized conversion of Google's Gemma 3 27B Instruct model, prepared by the MLX Community for efficient inference on Apple Silicon hardware. The conversion was performed using mlx-vlm v0.1.18.
Overview
Gemma 3 27B IT is Google's instruction-tuned large language model built on the Gemma 3 architecture, featuring 27 billion parameters. This variant has been quantized to 4-bit precision and reformatted for the MLX framework, Apple's machine learning library designed to take full advantage of the unified memory and GPU capabilities of M-series chips.
Multimodal Capabilities
Classified under the image-text-to-text pipeline, this model supports vision-language tasks — accepting both images and text as input and generating text responses. This makes it well suited for:
- Image captioning and description
- Visual question answering
- Document and diagram interpretation
- General instruction-following and conversational AI
Why 4-Bit Quantization?
The 4-bit quantization dramatically reduces the memory footprint of the full 27B-parameter model, making it feasible to run locally on consumer Apple hardware such as MacBook Pros and Mac Studios with sufficient unified memory. Despite the reduced precision, 4-bit variants of large models typically retain strong performance across a wide range of tasks.
Provenance
- Base model: google/gemma-3-27b-pt
- Instruction-tuned source: google/gemma-3-27b-it
- Conversion tool: mlx-vlm
- License: Gemma (requires agreement to Google's usage license on Hugging Face)
This model is a strong choice for developers and researchers seeking a powerful, multimodal instruction-tuned model that runs natively and efficiently on Apple Silicon without relying on cloud infrastructure.
spm https://github.com/trymirai/uzu.git
| 1 | import Uzu |
| 2 | |
| 3 | public func runChat() async throws { |
| 4 | let engineConfig = EngineConfig.create() |
| 5 | let engine = try await Engine.create(config: engineConfig) |
| 6 | |
| 7 | guard let model = try await engine.model(identifier: "mlx-community/gemma-3-27b-it-4bit") else { |
| 8 | return |
| 9 | } |
| 10 | for try await update in try await engine.download(model: model).iterator() { |
| 11 | print("Download progress: \(update.progress())") |
| 12 | } |
| 13 | |
| 14 | let messages = [ |
| 15 | ChatMessage.system().withText(text: "You are a helpful assistant"), |
| 16 | ChatMessage.user().withText(text: "Tell me a short, funny story about a robot") |
| 17 | ] |
| 18 | let session = try await engine.chat(model: model, config: .create()) |
| 19 | let stream = await session.replyWithStream(input: messages, config: .create()) |
| 20 | var message: ChatMessage? = nil |
| 21 | for try await update in stream.iterator() { |
| 22 | switch update { |
| 23 | case .replies(let replies): |
| 24 | message = replies.last?.message |
| 25 | case .error(let error): |
| 26 | print("Error: \(error)") |
| 27 | } |
| 28 | } |
| 29 | print("Text: \(message?.text() ?? "empty")") |
| 30 | } |