Bonsai 1-Bit LLM: 8 Billion Parameters in 1 GB, Running on Your Phone (2026)

Bonsai 1-Bit LLM: 8 Billion Parameters in 1 GB, Running on Your Phone

PrismML emerged from stealth on March 31, 2026, with a model that breaks a longstanding assumption in AI: that smaller models mean dumber models. Their 1-bit Bonsai 8B packs 8.2 billion parameters into 1.15 GB of memory. A standard 16-bit model of the same size needs 16 GB. That 14x reduction changes where and how you can run serious AI.

The Bonsai 8B scores an average of 70.5 across IFEval, GSM8K, HumanEval+, BFCL, MuSR, and MMLU-Redux benchmarks (as of April 2026). Compare that to Llama 3.1 8B at 67.1 and LFM2 8B at 69.6, both of which consume the full 16 GB. Qwen 3 8B tops the class at 79.3, but at 14x the memory cost.

This matters because most people do not own H100 clusters. They own phones, laptops, and maybe a gaming GPU.

What "1-Bit" Means in Practice

Standard LLMs store each weight as a 16-bit or 32-bit floating-point number. Bonsai replaces every weight with a single bit: each value becomes -1, 0, or +1. One FP16 scale factor per 128 weights preserves enough precision for the math to work. The effective storage is 1.125 bits per weight in the GGUF format.

PrismML applies this compression across the entire model: embeddings, attention layers, MLP layers, and the language model head. No component gets a higher-precision escape hatch. Previous attempts at 1-bit quantization broke down in at least one of these layers. PrismML claims their proprietary training approach prevents that degradation.

This differs from post-training quantization tools like GPTQ or AWQ, which compress an already-trained 16-bit model after the fact. Bonsai trains at native 1-bit precision on Google TPU v4 pods. The distinction matters: post-training quantization always loses some quality. Native 1-bit training can, in theory, learn to compensate for the reduced precision during training itself.

Speed Numbers That Change the Conversation

Raw throughput tells the story of what Bonsai enables:

• M4 Pro Mac: 131 tokens per second
• RTX 4090: 368 tokens per second
• iPhone 17 Pro Max: 44 tokens per second

For context, a standard 16-bit 8B model cannot fit on any current iPhone at all. Bonsai runs on one at speeds fast enough for real-time conversation.

On desktop hardware, 131 tokens per second on an M4 Pro means you get responses faster than most cloud API calls, with zero network latency and zero per-token cost. The 368 tokens per second on an RTX 4090 makes batch processing or agent workflows on consumer GPUs a realistic option.

PrismML also released two smaller variants: Bonsai 4B at 0.57 GB and Bonsai 1.7B at 0.24 GB. The 1.7B model fits on hardware as old as a GTX 1080, though Hacker News users reported it "hallucinates like crazy" on knowledge-heavy questions. The 8B model handles knowledge tasks with more reliability.

The Intelligence Density Metric

PrismML introduces a metric they call "intelligence density": the negative log of the model's average error rate divided by model size in GB. Bonsai 8B scores 1.06 per GB. The closest competitor, Qwen 3 8B, scores 0.10 per GB. That 10x gap represents the core PrismML thesis: you can deliver the same intelligence in a fraction of the space.

Skeptics on Hacker News and r/LocalLLaMA raised valid concerns. The benchmark suite PrismML chose (IFEval, GSM8K, HumanEval+, BFCL, MuSR, MMLU-Redux) is reasonable but not exhaustive. Nobody has tested Bonsai on extended reasoning chains, multi-turn conversations, or domain-specific tasks where full-precision models tend to pull ahead. PrismML's whitepaper does not include results for these scenarios.

The company also compares against Llama 3.1 8B as a primary baseline. That model released in mid-2024. Comparing a March 2026 model against a mid-2024 baseline makes the results look stronger than a comparison against current state-of-the-art would. The more honest comparison is against Qwen 3 8B, where Bonsai trails by 8.8 points on average benchmarks while using 14x less memory. Whether that tradeoff works for you depends on your use case.

Who Built This and Why It Matters

PrismML spun out of Caltech, co-founded by Babak Hassibi, Sahin Lale, Omead Pooladzandi, and Reza Sadri. The company raised $16.25 million from Khosla Ventures and Cerberus Ventures, with compute grants from Google and Caltech.

The Caltech connection matters for credibility. Hassibi is a professor of electrical engineering and computer science at Caltech with decades of work in information theory and signal processing. This is the kind of mathematical foundation that 1-bit model design demands. Compressing a model this hard without destroying its capabilities requires deep understanding of how information flows through neural networks.

All three Bonsai models ship under the Apache 2.0 license. You can download them from Hugging Face (under the prism-ml namespace), run them through llama.cpp forks, or use the MLX framework on Apple Silicon. The open-source release means the community can verify PrismML's claims independently, and early testing from r/LocalLLaMA users confirms the models work as advertised for basic coding, math, and general Q&A.

How Bonsai Fits Into the Local AI Ecosystem

If you run models on your own hardware, you know the tradeoffs: bigger models give better results but need expensive GPUs. Tools like Ollama make running local models straightforward, but you still need hardware that fits the model. A 70B parameter model requires 35-40 GB of VRAM even in 4-bit quantization.

Bonsai opens a different path. At 1.15 GB for 8B parameters, the model fits alongside other applications on any modern laptop. You could run it on a Raspberry Pi 5 with 8 GB of RAM and still have headroom. For developers building with LlamaIndex or Llama Stack, Bonsai offers a local inference backend that costs nothing to run and responds faster than most API endpoints.

The practical comparison for most users: ChatGPT costs $20/month for Plus, and Claude charges $20/month for Pro. Running Bonsai 8B on your own hardware costs $0/month after the one-time hardware investment you already made. The quality gap between Bonsai 8B and GPT-4o or Claude Sonnet remains significant for complex tasks. But for code completion, quick Q&A, data extraction, and structured output, Bonsai performs well enough to handle the job without a cloud round-trip.

For coding workflows, Cursor and GitHub Copilot connect to cloud models by default. Bonsai's speed on local hardware makes it a candidate for offline coding assistance, though its benchmark scores on HumanEval+ suggest it handles straightforward coding tasks better than complex multi-file refactors.

What This Means for Edge AI and Agents

The bigger picture goes beyond chatbots. A 1 GB model running at 44 tokens per second on a phone enables:

On-device AI agents that process your data without sending it to a server. Privacy-sensitive industries like healthcare and legal have blocked AI adoption because cloud inference means data leaves the building. Bonsai removes that objection. Offline intelligence for environments without reliable internet: field work, military applications, remote research stations, airplanes. A model that fits on a phone and runs without connectivity opens use cases that cloud AI cannot touch. Embedded AI in robotics and IoT where power budgets and memory constraints eliminate larger models. PrismML reports 4-5x energy savings over 16-bit inference. For battery-powered devices, that multiplier determines whether AI is feasible at all. Cost elimination for high-volume inference. If you run thousands of API calls per day through Perplexity or OpenAI, and your queries fall within Bonsai's capability range, switching to local inference zeroes out that line item.

The Honest Limitations

Bonsai 8B is not a replacement for frontier models. GPT-4o, Claude Opus, and Gemini Ultra handle complex reasoning, long-context synthesis, and nuanced instruction-following at levels Bonsai cannot match. The 70.5 benchmark average sits well below current frontier models scoring in the 85-90+ range on similar benchmarks.

The model's 65,536-token context window looks good on paper, but nobody has published results showing how well 1-bit weights preserve information across long contexts. Full-precision models already struggle with context degradation at extreme lengths. Extreme quantization could make this worse.

PrismML's benchmarking approach also raises questions. They selected six benchmarks that play to instruction-following and code generation strengths. Benchmarks testing common-sense reasoning, factual knowledge depth, and multi-hop inference would provide a more complete picture. The community needs time to run independent evaluations.

The "1-bit" label itself invites misunderstanding. Each weight stores a ternary value (-1, 0, +1) with a shared FP16 scale factor per 128 weights. The effective bit rate is 1.125 bits per weight, not a pure single bit. PrismML acknowledges this in their technical documentation, but the marketing leans into "1-bit" as a headline.

How to Try Bonsai Today

1. Mac users: Install the PrismML MLX fork and download from Hugging Face (prism-ml/Bonsai-8B-mlx-1bit). Run through mlx-lm with temperature: 0.5, top-k: 20.
2. GPU users (NVIDIA): Download the GGUF format (prism-ml/Bonsai-8B-gguf) and run through PrismML's llama.cpp fork. Standard llama.cpp does not support the Q10g128 format yet.
3. iPhone users: PrismML partnered with Locally AI for iOS support.
4. Browser: A Google Colab notebook is available for testing without local setup.

All models use Apache 2.0 licensing. No restrictions on commercial use.

The Bottom Line

Bonsai does not dethrone frontier models. It does something more interesting: it proves that a 1 GB model can perform useful work that previously required 16 GB. PrismML backed that claim with $16.25 million in funding, a Caltech research pedigree, and open-source weights anyone can test.

The AI industry has spent three years in an arms race toward bigger models, bigger clusters, and bigger costs. Bonsai points the opposite direction. If future iterations close the gap with full-precision models further, the implications for device manufacturers, app developers, and anyone paying per-token API fees are massive.

For now, download the model, run it on your hardware, and see what 1.15 GB of intelligence can do. The answer might surprise you.

Benchmark scores and pricing referenced as of April 2026. Visit PrismML for the latest model releases and documentation.