This is Llama 3.2 1B Instruct converted to 4-bit quantized format optimized for use with the MLX framework. The model is based on Meta's Llama 3.2 1B parameter foundational large language model, fine-tuned with instruction-following capabilities to follow user directions and engage in multi-turn conversations. It supports multiple languages including English, German, French, Italian, Portuguese, Hindi, Spanish, and Thai. This quantized version reduces model size while maintaining performance, making it suitable for deployment on Apple Silicon and other resource-constrained environments through the MLX machine learning framework.
Meta
available local models on Mirai:
available local models on Mirai:
Name
Quantisation
Size
Llama-3.1-8B-Instruct
uint4
8B
Quant.
uint4
Size
8B
Llama-3.2-1B-Instruct
uint4
1B
Quant.
uint4
Size
1B
Llama-3.2-3B-Instruct
uint4
3B
Quant.
uint4
Size
3B
Llama-3.1-8B-Instruct-4bit
uint4
8B
Quant.
uint4
Size
8B
Llama-3.2-1B-Instruct-4bit
uint4
1B
Quant.
uint4
Size
1B
Llama-3.2-1B-Instruct-8bit
uint4
1B
Quant.
uint4
Size
1B
Llama-3.2-3B-Instruct-4bit
uint4
3B
Quant.
uint4
Size
3B
Llama-3.2-3B-Instruct-8bit
uint4
3B
Quant.
uint4
Size
3B
Llama-3.2-3B-Instruct-AWQ
uint4
3B
Quant.
uint4
Size
3B
This is Llama 3.2 1B Instruct converted to 4-bit quantized format optimized for use with the MLX framework. The model is based on Meta's Llama 3.2 1B parameter foundational large language model, fine-tuned with instruction-following capabilities to follow user directions and engage in multi-turn conversations. It supports multiple languages including English, German, French, Italian, Portuguese, Hindi, Spanish, and Thai. This quantized version reduces model size while maintaining performance, making it suitable for deployment on Apple Silicon and other resource-constrained environments through the MLX machine learning framework.
Meta
available local models on Mirai:
Name
Quantisation
Size
Llama-3.1-8B-Instruct
uint4
8B
Quant.
uint4
Size
8B
Llama-3.2-1B-Instruct
uint4
1B
Quant.
uint4
Size
1B
Llama-3.2-3B-Instruct
uint4
3B
Quant.
uint4
Size
3B
Llama-3.1-8B-Instruct-4bit
uint4
8B
Quant.
uint4
Size
8B
Llama-3.2-1B-Instruct-4bit
uint4
1B
Quant.
uint4
Size
1B
Llama-3.2-1B-Instruct-8bit
uint4
1B
Quant.
uint4
Size
1B
Llama-3.2-3B-Instruct-4bit
uint4
3B
Quant.
uint4
Size
3B
Llama-3.2-3B-Instruct-8bit
uint4
3B
Quant.
uint4
Size
3B
Llama-3.2-3B-Instruct-AWQ
uint4
3B
Quant.
uint4
Size
3B