meta-llama/Llama-3.2-3B-Instruct
Run locally on Apple devices with Mirai
- Type
- local
- From
- Meta
- Quantization
- No
- Parameters
- 3B
- Size
- 6.0 GB
- Source
Llama 3.2 3B Instruct is a compact, instruction-tuned large language model from Meta, designed for multilingual dialogue and text generation tasks. Part of the Llama 3.2 collection (available in 1B and 3B parameter sizes), this 3B variant strikes a balance between capability and efficiency — making it well suited for deployment scenarios where computational resources are limited, including on-device and edge applications.
Architecture & Training
Built on an optimized auto-regressive transformer architecture, the model incorporates Grouped-Query Attention (GQA) for improved inference scalability and throughput. It was trained on up to 9 trillion tokens from publicly available sources, with a knowledge cutoff of December 2023. The instruction-tuned variant was further refined through supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF) to align outputs with human preferences around helpfulness and safety.
Multilingual Support
The model officially supports eight languages: English, German, French, Italian, Portuguese, Hindi, Spanish, and Thai — making it a strong choice for multilingual conversational applications without the overhead of much larger models.
Key Strengths
- Agentic retrieval & summarization: Optimized for dialogue-driven workflows that involve information retrieval and content summarization.
- Competitive benchmarks: Outperforms many open-source and closed chat models of comparable size on standard industry evaluations.
- Flexible deployment: Quantized variants are available for resource-constrained environments, including mobile devices.
Use Cases
Llama 3.2 3B Instruct is ideal for chatbots, multilingual assistants, summarization pipelines, and lightweight agentic applications where a small footprint matters. Its instruction tuning ensures coherent, aligned responses out of the box, reducing the need for extensive prompt engineering.
spm https://github.com/trymirai/uzu.git
| 1 | import Uzu |
| 2 | |
| 3 | public func runChat() async throws { |
| 4 | let engineConfig = EngineConfig.create() |
| 5 | let engine = try await Engine.create(config: engineConfig) |
| 6 | |
| 7 | guard let model = try await engine.model(identifier: "meta-llama/Llama-3.2-3B-Instruct") else { |
| 8 | return |
| 9 | } |
| 10 | for try await update in try await engine.download(model: model).iterator() { |
| 11 | print("Download progress: \(update.progress())") |
| 12 | } |
| 13 | |
| 14 | let messages = [ |
| 15 | ChatMessage.system().withText(text: "You are a helpful assistant"), |
| 16 | ChatMessage.user().withText(text: "Tell me a short, funny story about a robot") |
| 17 | ] |
| 18 | let session = try await engine.chat(model: model, config: .create()) |
| 19 | let stream = await session.replyWithStream(input: messages, config: .create()) |
| 20 | var message: ChatMessage? = nil |
| 21 | for try await update in stream.iterator() { |
| 22 | switch update { |
| 23 | case .replies(let replies): |
| 24 | message = replies.last?.message |
| 25 | case .error(let error): |
| 26 | print("Error: \(error)") |
| 27 | } |
| 28 | } |
| 29 | print("Text: \(message?.text() ?? "empty")") |
| 30 | } |