# Mirai

> We are the frontier on-device AI lab. We build the models, inference runtime, and quantization stack. From the device constraint up. So AI can run at full capability on the hardware billions of people already own.

## What we believe?

The trinity: model, inference stack, hardware.

Companies that focus on a single component of this trinity lack sovereignty. They're constrained by the architectural choices made by others. Mirai Labs owns all three.

Most labs treat on-device models as scaled-down versions of their cloud-focused cousins. But LLM architectures that evolved for the cloud are not well-suited to on-device setups. Cloud models operate in the arithmetic-bound regime. They treat memory as an unlimited resource and optimize for throughput across a batch.

On a device, memory is the main bottleneck. The architecture itself has to be different.

## What we are building?

1. Models designed for memory-bound environments.

We focus on three core objectives: increasing arithmetic intensity at the decoding stage, reducing the size of the resident set, and maximally utilizing GPU neural accelerators. This leads to models that differ from standard transformers in meaningful structural ways.

2. A sovereign inference engine.

Full control of the inference stack is what gives us a unique advantage. We have the freedom to tailor the model to the hardware. And the hardware assumptions into the model. No generic runtime can do this.

3. Quantization co-designed with the architecture.

## On-device is not a smaller cloud. It’s a different system entirely.

Cloud

- Large batches
- Throughput-first
- Memory-rich
- Compute-saturating

On-device

- Batch size = 1
- Latency-first
- Memory-constrained
- Bandwidth-sensitive

## Performance emerges from co-design across the stack.

On a device, every request is processed alone. No batching, no parallelism. Mirai co-designs models and runtime specifically for this constraint, on Apple Silicon.

- Minimizing memory footprint.
- Maximizing arithmetic intensity.
- maximally utilizing GPU neural accelerators.
- Quantization-aware training (QAT).

## The full on-device stack to achieve realtime local intelligence.

1. Local models

Architectures designed for memory-bound execution and realtime decoding.

2. Inference engine

Hardware aware optimization of tensor multiplications & other operations.

3. Quantization

Compress models without collapsing interaction quality or latency.

4. Application layer

We start from your Apple device automation.

## Product

- [Platform / SDK](https://platform.trymirai.com/)
- [Inference Runtime](https://trymirai.com/inference-runtime): The fastest inference runtime for iPhone, iPad and Mac. Optimize and run your model on every Apple device. Up to 38% faster prompt processing vs MLX.
- [Models Conversion](https://trymirai.com/conversion-optimization-toolkit): Conversion and optimization toolkit. Convert and optimize your model for iPhone, iPad and Mac. One command to get your model running on 2 billion Apple devices.
- [Models Library](https://trymirai.com/local-models): AI models library optimized for on device. Search our library of available models to run on device and deploy in seconds with Mirai.
- [MacOS App](https://trymirai.com/chat-for-mac): Download Mirai for Mac.
- [Docs](https://docs.trymirai.com/)

## Models already running on Mirai.

- [Qwen3.5-0.8B-M Mirai](https://trymirai.com/local-models/alibaba-qwen3-5-0-8b-mirai-mirai-m-4)
- [Gemma-3 Google](https://trymirai.com/local-models/google-gemma-3-1b-it)
- [LFM2.5 LiquidAI](https://trymirai.com/local-models/liquidai-lfm2-5-350m)
- [Llama-3.2 Meta](https://trymirai.com/local-models/meta-llama-3-2-1b-instruct)
- [GPT-OSS OpenAI](https://trymirai.com/local-models/openai-gpt-oss-20b)

## Docs

- [About](https://docs.trymirai.com/index.md)
- [Chat](https://docs.trymirai.com/quick-start/chat.md)
- [Cloud](https://docs.trymirai.com/quick-start/chat-cloud.md)
- [Speculative classification](https://docs.trymirai.com/quick-start/chat-speculation-classification.md)
- [Speculative summarization](https://docs.trymirai.com/quick-start/chat-speculation-summarization.md)
- [Structured output](https://docs.trymirai.com/quick-start/chat-structured-output.md)
- [Classification](https://docs.trymirai.com/quick-start/classification.md)
- [Text to Speech](https://docs.trymirai.com/quick-start/text-to-speech.md)
- [openapi](https://docs.trymirai.com/api-reference/openapi.json)

## Blog

- [Sparse Buffers for KV Cache](https://trymirai.com/blog/sparse-buffers-for-kv-cache)
- [Introducing Mirai Quantization: Redefining the speed-quality frontier for local LLMs on Apple silicon.](https://trymirai.com/blog/quantization)

## Company

- [Main](https://trymirai.com/): Models, runtime & infrastructure to make on-device AI interactive, ambient & continuous.
- [About us](https://trymirai.com/about-us): We’re a small, senior team building the full on-device stack to achieve realtime local intelligence.
- [Careers](https://trymirai.com/careers): Join a small, senior team building the full on-device stack to achieve realtime local intelligence.
- [Contact Us](https://trymirai.com/i-am-interested): Talk to us.

## Open Positions

- [Machine Learning Engineer](https://trymirai.com/careers/machine-learning-engineer): Models & Research. Join a small, senior team building the fastest on-device AI inference engine. Powering real products, not demos.
- [Machine Learning Engineer, Model Optimization](https://trymirai.com/careers/machine-learning-engineer-model-optimization): Models Optimization. Join a small, senior team building the fastest on-device AI inference engine. Powering real products, not demos.
- [Inference Engineer](https://trymirai.com/careers/inference-engineer): Join a small, senior team building the fastest on-device AI inference engine. Powering real products, not demos.

## Links

- [Github](https://github.com/trymirai)
- [inference engine](https://github.com/trymirai/uzu)
- [model conversion toolkit](https://github.com/trymirai/lalamo)
- [X (Twitter)](https://x.com/trymirai)
- [LinkedIn](https://www.linkedin.com/company/trymirai)
- [Discord](https://discord.com/invite/gUhyn6Rb7x)
- [Privacy Policy](https://trymirai.com/privacy-policy)
- [Terms of Use](https://trymirai.com/terms-of-use)