New: 2x faster than Ollama
Run open models as fast as your hardware allows
Magnitude is an open source inference engine that tunes its kernels on your hardware to find the fastest way to run each model.
Why Magnitude?
- 2x faster than Ollama: on average across popular local models, on Mac and NVIDIA
- Tuned on your hardware: kernels are tuned for each model when you download it
- Best drafter built in: each model ships with the fastest speculative decoding drafter
- Memory that flexes: quantized KV halves memory per agent, freed when agents stop
- Fast concurrent sessions: sessions share prefix caches to prevent slowdown
- Works with your agent: one click to connect Pi, OpenCode, Hermes, Codex, and more
- Free, private, open source: no token costs, nothing leaves your machine, Apache 2.0
2x faster than Ollama
Mac M4 Pro 48 GB
34%
faster prefill
vs Ollama (MLX)
100%
faster decode
vs Ollama (MLX)
NVIDIA DGX Spark
27%
faster prefill
vs Ollama (llama.cpp)
100%
faster decode
vs Ollama (llama.cpp)
Average of 11 popular local models, 4-bit, 64k context, with speculative decoding.
Ollama Magnitude
FAQ
What is Magnitude?
An open source inference engine that optimizes each model for your exact hardware. It ships as a desktop app that runs open models and connects them to the agent you already use.
How is it faster than llama.cpp, Ollama, or LM Studio?
They ship one-size-fits-all kernels, because tuning each model for each chip by hand would take forever. Magnitude tunes them automatically. When you download a model, it spends up to a minute on your actual chip trying different ways to split up the math (how many GPU threads team up, how big a chunk each one takes, how far ahead to load data) and keeps the fastest combination that doesn't change the model's output. See the benchmarks against Ollama.
What hardware do I need?
Any Apple Silicon, NVIDIA, or AMD GPU, or nothing but a CPU. There is no fixed minimum. Smaller machines run smaller models, and more memory lets you run larger ones.
What operating systems does it support?
macOS, Linux, and Windows.
Which models does it support?
See the full list at magnitude.dev/models. We write optimized kernels for the most popular open-weight families, which is how we beat generalist engines.
Which agents work with it?
One click connects Pi, OpenCode, Hermes, OpenClaw, Codex, Claude Code, Oh My Pi, and Cline. Anything else works through the OpenAI-compatible API.
Is it private?
Yes. Prompts, files, and models stay on your machine. No internet needed once a model is downloaded.
