Product

Models library

MacOS app

Docs

Company

...

Talk to us

Try Mirai

Try SDK

Try Mirai

Talk to us

Talk to us

Talk to us

Run AI on User Devices

Run AI on User Devices.

The fastest inference engine for  Apple Silicon. Ship text, voice, and code features that work locally on Mac, iPhone, and iPad

The fastest inference engine for  Apple Silicon. Ship text, voice, and code features that work locally on Mac, iPhone, and iPad

The fastest inference engine for  Apple Silicon. Ship text, voice, and code features that work locally on Mac, iPhone, and iPad.

Talk to us

Explore docs

Talk to us

Explore docs

Talk to us

Explore docs

Local chatOffline • Powered by Mirai
Q1-26-priorities...pdf
Summarize this document
Summarised locally on iPhone
300 ms
What is the key action item?
Transcribed locally on iPhone
150 ms
New Chat
Chats
Local Models
Agents
Routers
Document summarisa...
Settings
Local chatOffline • Powered by Mirai
Q1-26-priorities...pdf
Summarise this document
Summarised locally on Mac
300 ms • Offline • Local
What is the key action item?
Transcribed locally on Mac
150 ms • Offline • Local
Llama-3.2-1B-Instruct
Eject model
Llama-3.2-1B-Instruct-QLoRA
0.1.18
Local chatOffline • Powered by Mirai
Q1-26-priorities...pdf
Summarize this document
Summarised locally on iPhone
300 ms
What is the key action item?
Transcribed locally on iPhone
150 ms
New Chat
Chats
Local Models
Agents
Routers
Document summarisa...
Settings
Local chatOffline • Powered by Mirai
Q1-26-priorities...pdf
Summarise this document
Summarised locally on Mac
300 ms • Offline • Local
What is the key action item?
Transcribed locally on Mac
150 ms • Offline • Local
Llama-3.2-1B-Instruct
Eject model
Llama-3.2-1B-Instruct-QLoRA
0.1.18
New Chat
Chats
Local Models
Agents
Routers
Document summarisa...
Settings
Document summarisation local chat
Q1-26-priorities...pdf
Summarise this document

Summary:

  • Three priorities for Q1
  • Increased cloud inference costs
  • On-device execution reduces latency and spend
Summarised on device
300 ms
Create action items from this
Transcribed on device
150 ms

Action items:

  1. Finalize SDK API by Friday.
  2. Add on-device benchmarks.
  3. Prepare enterprise security review.
Add message...
Llama-3.2-1B-Instruct
Eject model
Llama-3.2-1B-Instruct-QLoRA
0.1.18

Backed by leading AI investors and builders

Backed by leading AI investors and builders

Backed by leading AI investors and builders

Backed by leading AI investors and builders

On-device inference is the next step for your models

No spinners, no loading states, no long "thinking..."

Make latency disappear.

autocomplete, live translation.

Real-time features:

no geographic variance or queue delays.

Consistent UX:

no loading states, retry logic, timeouts.

Simpler code:

Offline by default.

Near-zero marginal inference cost.

Data stays on device.

No spinners, no loading states, no long "thinking..."

Make latency disappear.

autocomplete, live translation.

Real-time features:

no geographic variance or queue delays.

Consistent UX:

no loading states, retry logic, timeouts.

Simpler code:

Offline by default.

Near-zero marginal inference cost.

Data stays on device.

No spinners, no loading states, no long "thinking..."

Make latency disappear.

autocomplete, live translation.

Real-time features:

no geographic variance or queue delays.

Consistent UX:

no loading states, retry logic, timeouts.

Simpler code:

Offline by default.

Near-zero marginal inference cost.

Data stays on device.

No spinners, no loading states, no long "thinking..."

Make latency disappear.

autocomplete, live translation.

Real-time features:

no geographic variance or queue delays.

Consistent UX:

no loading states, retry logic, timeouts.

Simpler code:

Offline by default.

Near-zero marginal inference cost.

Data stays on device.

Apple devices got
powerful enough to run AI

 Apple Silicon turned everyday devices into machines that can run real AI models

0 TOPS

Neural Engine on M4

enough to run a 7B model at conversation speed

0 GB/S

Unified memory bandwidth

Models load and execute without bottlenecks

0 T/S

Qwen 3B on MacBook Air M2

Faster than most cloud APIs respond

Stanford's Intelligence Per Watt study found that 88.7% of real-world AI queries can be accurately served by local models on consumer hardware. Research →

Performance

Mirai is the fastest on-device inference on Apple devices

On-device layer for AI model makers and products.

Outperforming

Apple MLX

Apple MLX

Apple MLX

Apple MLX

llama.cpp

llama.cpp

llama.cpp

llama.cpp

All inference engine measurements were done with real hardware. No synthetic benchmarks. Full benchmarks →

Text. Voice. Vision soon

On-device layer for AI model makers and products.

Three modalities. All running locally on Apple devices

Text LLMs

Summarization & extraction
documents, emails, tickets

Autocomplete
code, forms, replies

Classification
routing, content tagging, triage

Search

local knowledge, embeddings, RAG

Extraction

structured data, unstructured input

Audio

Speech-to-text
transcription, meeting notes, voice

Text-to-speech
narration, accessibility, voice UI

Speech-to-speech
real-time conversation, translation

Voice commands

hands-free control, dictation

Vision soon

Object detection
camera input, scene understanding

Document parsing
receipts, IDs, forms

Visual search
product lookup, image matching

OCR

text extraction from images

Built natively for Apple Silicon

On-device layer for AI model makers and products.

Mirai is a native inference engine designed specifically for Apple hardware and built from scratch. That's why it's faster.

Model optimization

Takes your model. Converts and optimizes it for Apple silicon.

Learn more

Inference engine

Rust-based runtime. Built from scratch. Executes on-device.

Learn more

CLI to test locally

Benchmark and serve models from your terminal.

Learn more

Platform

Configure on-device inference

Learn more

Speculative decoding on device

A small draft model predicts the next tokens; your target model verifies them in one pass. 1.5-2x faster generation with no quality loss.

Learn more

All major model architectures supported

On-device layer for AI model makers and products.

Bring your own model or pick from our optimized library

  • Llamba

    Cartesia

    1B

    3B

    8B

  • Gemma-3

    Google

    1B

    4B

  • Llama 3.1 3.2

    Meta

    1B

    3B

    8B

  • Qwen2.5

    Alibaba

    0.5B

    0.6B

    1.5B

    …

  • Llamba

    Cartesia

    1B

    3B

    8B

  • Gemma-3

    Google

    1B

    4B

  • Llama 3.1 3.2

    Meta

    1B

    3B

    8B

  • Qwen2.5

    Alibaba

    0.5B

    0.6B

    1.5B

    …

  • Llamba

    Cartesia

    1B

    3B

    8B

  • Gemma-3

    Google

    1B

    4B

  • Llama 3.1 3.2

    Meta

    1B

    3B

    8B

  • Qwen2.5

    Alibaba

    0.5B

    0.6B

    1.5B

    …

  • Llamba

    Cartesia

    1B

    3B

    8B

  • Gemma-3

    Google

    1B

    4B

  • Llama 3.1 3.2

    Meta

    1B

    3B

    8B

  • Qwen2.5

    Alibaba

    0.5B

    0.6B

    1.5B

    …

Explore supported models

Common Questions

What is inference?

Inference is the process of running a trained AI model to produce an output. For example, generating text, classifying a message, or transcribing audio. In simple terms: training = teaching the model, inference = using the model. Every AI response your users see is the result of inference.

What is local (on-device) inference?

Framer is fully visual with no code needed, but you can still add custom code and components for more control if you're a designer or developer.

Which models can run locally and which can’t?

This is a free, responsive FAQ section for Framer. Drop it into any project, customize styles and text, and use it to save time on support or info pages.

What can you actually build with local models?

After duplicating, copy and paste the component into your Framer project. Then edit the questions, answers, styles, and animations as needed.

What hardware does Mirai run on?

Yes, absolutely. The component is built using native Framer tools, so you can tweak fonts, colors, spacing, animations, and layout however you like.

Why do I need this? What’s the real business value?

Yes, the FAQ component is fully responsive and adapts seamlessly to desktop, tablet, and mobile screen sizes.

What is inference?

Inference is the process of running a trained AI model to produce an output. For example, generating text, classifying a message, or transcribing audio. In simple terms: training = teaching the model, inference = using the model. Every AI response your users see is the result of inference.

What is local (on-device) inference?

Framer is fully visual with no code needed, but you can still add custom code and components for more control if you're a designer or developer.

Which models can run locally and which can’t?

This is a free, responsive FAQ section for Framer. Drop it into any project, customize styles and text, and use it to save time on support or info pages.

What can you actually build with local models?

After duplicating, copy and paste the component into your Framer project. Then edit the questions, answers, styles, and animations as needed.

What hardware does Mirai run on?

Yes, absolutely. The component is built using native Framer tools, so you can tweak fonts, colors, spacing, animations, and layout however you like.

Why do I need this? What’s the real business value?

Yes, the FAQ component is fully responsive and adapts seamlessly to desktop, tablet, and mobile screen sizes.

What is inference?

Inference is the process of running a trained AI model to produce an output. For example, generating text, classifying a message, or transcribing audio. In simple terms: training = teaching the model, inference = using the model. Every AI response your users see is the result of inference.

What is local (on-device) inference?

Framer is fully visual with no code needed, but you can still add custom code and components for more control if you're a designer or developer.

Which models can run locally and which can’t?

This is a free, responsive FAQ section for Framer. Drop it into any project, customize styles and text, and use it to save time on support or info pages.

What can you actually build with local models?

After duplicating, copy and paste the component into your Framer project. Then edit the questions, answers, styles, and animations as needed.

What hardware does Mirai run on?

Yes, absolutely. The component is built using native Framer tools, so you can tweak fonts, colors, spacing, animations, and layout however you like.

Why do I need this? What’s the real business value?

Yes, the FAQ component is fully responsive and adapts seamlessly to desktop, tablet, and mobile screen sizes.

What is inference?

Inference is the process of running a trained AI model to produce an output. For example, generating text, classifying a message, or transcribing audio. In simple terms: training = teaching the model, inference = using the model. Every AI response your users see is the result of inference.

What is local (on-device) inference?

Framer is fully visual with no code needed, but you can still add custom code and components for more control if you're a designer or developer.

Which models can run locally and which can’t?

This is a free, responsive FAQ section for Framer. Drop it into any project, customize styles and text, and use it to save time on support or info pages.

What can you actually build with local models?

After duplicating, copy and paste the component into your Framer project. Then edit the questions, answers, styles, and animations as needed.

What hardware does Mirai run on?

Yes, absolutely. The component is built using native Framer tools, so you can tweak fonts, colors, spacing, animations, and layout however you like.

Why do I need this? What’s the real business value?

Yes, the FAQ component is fully responsive and adapts seamlessly to desktop, tablet, and mobile screen sizes.

Free your cloud

Free your cloud

The fastest inference engine for  Apple Silicon. Ship text, voice, and code features that work locally on Mac, iPhone, and iPad.

Run your models on Apple devices with the fastest inference engine

On-device layer for AI model makers and products.

Talk to us

Explore docs

Talk to us

Explore docs

Talk to us

Explore docs

Main

Main

Main

Apple Inference SDK

Apple Inference SDK

Apple Inference SDK

Apple Inference SDK

Smart Routing

Smart Routing

Smart Routing

Smart Routing

Models Library

Models Library

Models Library

Models Library

MacOS App

MacOS App

MacOS App

MacOS App

Blog

Blog

Blog

Blog

Docs

Docs

Docs

Docs

Company

About us

About us

About us

About us

Careers

Careers

Careers

Careers

Contact Us

Contact Us

Contact Us

Contact Us

Privacy Policy

Privacy Policy

Privacy Policy

Privacy Policy

Terms of Use

Terms of Use

Terms of Use

Terms of Use

Links

X (Twitter)

X (Twitter)

X (Twitter)

X (Twitter)

Github

Github

Github

Github

LinkedIn

LinkedIn

LinkedIn

LinkedIn

Discord

Discord

Discord

Discord