Outlines

Outlines is a Python library for structured text generation with LLMs, using constrained decoding to guarantee that model outputs conform to specified formats. Unlike post-hoc parsing approaches where you hope the LLM generates valid JSON and retry if it doesn't, Outlines constrains the token generation process itself so that invalid tokens are never sampled. The output is guaranteed valid — not 99% reliable, but mathematically guaranteed.

The library works by building finite state machines from output schemas (JSON Schema, regular expressions, Pydantic models, or context-free grammars) and using them to mask invalid tokens at each generation step. Only tokens leading to valid completions are considered during sampling.

Outlines supports multiple model backends: Hugging Face Transformers, vLLM (high-throughput serving), llama.cpp (local inference), ExLlamaV2 (quantized models), and MLX (Apple Silicon). It works with any model these backends support — Llama, Mistral, Phi, Gemma, Qwen, and more.

Generation modes include: JSON from Pydantic models or JSON Schema, regex-guided generation, choice selection from a list, grammar-guided generation (context-free grammars for SQL, code, etc.), and type-based generation. The @outlines.prompt decorator turns functions into prompt templates.

Honest assessment: Outlines is the right tool when you need guaranteed structured output from local models. It's the gold standard for constrained generation. However, it only works with local models where you have access to logits — it doesn't work with API-based models. For API-based structured output, use Instructor instead. Outlines is also more computationally expensive than unconstrained generation due to FSM construction and token masking.