This is Llama 3.2 3B Instruct converted to MLX format with 4-bit quantization. Llama 3.2 is a foundational large language model developed by Meta that supports multiple languages including English, German, French, Italian, Portuguese, Hindi, Spanish, and Thai. The model is designed for instruction-following tasks and can be used for text generation. This particular version has been optimized for use with MLX, a machine learning framework designed for Apple Silicon, making it suitable for efficient inference on compatible hardware.
Meta
available local models on Mirai:
available local models on Mirai:
Name
Quantisation
Size
Llama-3.1-8B-Instruct
uint4
8B
Quant.
uint4
Size
8B
Llama-3.2-1B-Instruct
uint4
1B
Quant.
uint4
Size
1B
Llama-3.2-3B-Instruct
uint4
3B
Quant.
uint4
Size
3B
Llama-3.1-8B-Instruct-4bit
uint4
8B
Quant.
uint4
Size
8B
Llama-3.2-1B-Instruct-4bit
uint4
1B
Quant.
uint4
Size
1B
Llama-3.2-1B-Instruct-8bit
uint4
1B
Quant.
uint4
Size
1B
Llama-3.2-3B-Instruct-4bit
uint4
3B
Quant.
uint4
Size
3B
Llama-3.2-3B-Instruct-8bit
uint4
3B
Quant.
uint4
Size
3B
Llama-3.2-3B-Instruct-AWQ
uint4
3B
Quant.
uint4
Size
3B
This is Llama 3.2 3B Instruct converted to MLX format with 4-bit quantization. Llama 3.2 is a foundational large language model developed by Meta that supports multiple languages including English, German, French, Italian, Portuguese, Hindi, Spanish, and Thai. The model is designed for instruction-following tasks and can be used for text generation. This particular version has been optimized for use with MLX, a machine learning framework designed for Apple Silicon, making it suitable for efficient inference on compatible hardware.
Meta
available local models on Mirai:
Name
Quantisation
Size
Llama-3.1-8B-Instruct
uint4
8B
Quant.
uint4
Size
8B
Llama-3.2-1B-Instruct
uint4
1B
Quant.
uint4
Size
1B
Llama-3.2-3B-Instruct
uint4
3B
Quant.
uint4
Size
3B
Llama-3.1-8B-Instruct-4bit
uint4
8B
Quant.
uint4
Size
8B
Llama-3.2-1B-Instruct-4bit
uint4
1B
Quant.
uint4
Size
1B
Llama-3.2-1B-Instruct-8bit
uint4
1B
Quant.
uint4
Size
1B
Llama-3.2-3B-Instruct-4bit
uint4
3B
Quant.
uint4
Size
3B
Llama-3.2-3B-Instruct-8bit
uint4
3B
Quant.
uint4
Size
3B
Llama-3.2-3B-Instruct-AWQ
uint4
3B
Quant.
uint4
Size
3B