mlx-community/gemma-3-4b-it-4bit

A 4-bit quantized version of Google's Gemma 3 4B Instruct model, converted to Apple's MLX format for efficient on-device inference on Apple Silicon hardware. This community conversion was produced using mlx-vlm v0.1.18, preserving the multimodal capabilities of the original model in a significantly smaller footprint.

Origin & Architecture

The base model is google/gemma-3-4b-it, part of Google's Gemma 3 family — a lightweight yet capable series of instruction-tuned language models. Gemma 3 4B supports image-text-to-text generation, meaning it can process both visual and textual inputs to produce natural language responses. The 4-bit quantization dramatically reduces memory requirements, making it practical for local use on MacBooks and other Apple Silicon devices.

Key Highlights

• MLX-native format — purpose-built for Apple's MLX framework, enabling hardware-accelerated inference without external GPU dependencies.
• 4-bit quantization — substantially reduces model size and memory usage compared to the full-precision original, with minimal quality trade-off for most tasks.
• Multimodal capability — handles both image understanding and text generation tasks, from visual question answering to image description.
• Instruction-tuned — fine-tuned for conversational and instruction-following use cases, responding naturally to user prompts.

Use Cases

This model is well-suited for developers and researchers looking to run a capable multimodal model locally on Mac hardware. Common applications include image captioning, visual Q&A, conversational AI, and general text generation — all without cloud API dependencies.

Access

Usage requires acknowledgment of Google's Gemma license agreement via Hugging Face. Refer to the original model card for full details on capabilities and limitations.