Qwen3-4B-AWQ

Run locally Apple devices with Mirai

Type

Type

Local

From

From

Alibaba

Quantisation

Quantisation

uint4

Precision

Precision

No

Size

Size

4B

Source

Source

Hugging Face Logo

Qwen3-4B-AWQ is a 4 billion parameter quantized language model that represents the latest generation in the Qwen series. It is a quantized version of Qwen3-4B using 4-bit AWQ quantization, offering a compact yet capable model for efficient deployment. The model features a unique dual-mode capability that allows seamless switching between a thinking mode for complex logical reasoning, mathematics, and coding tasks, and a non-thinking mode for efficient general-purpose dialogue. This flexibility enables users to optimize performance across different types of queries within a single model. Qwen3-4B-AWQ supports 100+ languages and dialects with strong multilingual instruction-following capabilities. The model natively supports context lengths of 32,768 tokens and can be extended to 131,072 tokens using the YaRN scaling technique. It excels in reasoning, instruction following, agent capabilities with tool use, creative writing, role-playing, and multi-turn conversations.

1
Choose framework
2
Run the following command to install Mirai SDK
SPMhttps://github.com/trymirai/uzu-swift
3
Set Mirai API keyGet API Key
4
Apply code
Loading...

Qwen3-4B-AWQ

Run locally Apple devices with Mirai

Type

Local

From

Alibaba

Quantisation

uint4

Precision

float16

Size

4B

Source

Hugging Face Logo

Qwen3-4B-AWQ is a 4 billion parameter quantized language model that represents the latest generation in the Qwen series. It is a quantized version of Qwen3-4B using 4-bit AWQ quantization, offering a compact yet capable model for efficient deployment. The model features a unique dual-mode capability that allows seamless switching between a thinking mode for complex logical reasoning, mathematics, and coding tasks, and a non-thinking mode for efficient general-purpose dialogue. This flexibility enables users to optimize performance across different types of queries within a single model. Qwen3-4B-AWQ supports 100+ languages and dialects with strong multilingual instruction-following capabilities. The model natively supports context lengths of 32,768 tokens and can be extended to 131,072 tokens using the YaRN scaling technique. It excels in reasoning, instruction following, agent capabilities with tool use, creative writing, role-playing, and multi-turn conversations.

1
Choose framework
2
Run the following command to install Mirai SDK
SPMhttps://github.com/trymirai/uzu-swift
3
Set Mirai API keyGet API Key
4
Apply code
Loading...