Guidance

Guidance is a prompting language from Microsoft Research that gives you precise control over LLM generation by interleaving template text with generation commands. Instead of writing a prompt and hoping the model formats its response correctly, you write a Guidance program that controls exactly where the model generates, what constraints apply, and how the output is structured.

The key insight is treating LLM interaction as a programming language, not a text completion. A Guidance program looks like a template with embedded commands: {{gen 'name' maxtokens=50}} for generation, {{select 'category' opticategories}} for constrained selection, {{#geneach 'items' numiterati3}} for repeated generation. The model generates only in specified slots, with template text serving as guaranteed context.

Guidance supports both API-based models (OpenAI, Anthropic, Azure) and local models (Transformers, llama.cpp). With local models, it uses token healing and constrained generation similar to Outlines. With API models, it uses prompt templating and output parsing.

Generation commands include: gen (open-ended with optional regex/stop constraints), select (choose from a list), geneach (generate multiple items), block/if/else (conditional generation), and function calls. These compose naturally for complex generation programs.

Guidance's acceleration features are notable: token healing corrects tokenization artifacts at generation boundaries, prefix caching reuses computation across calls, and the program structure enables efficient batching.

Honest assessment: Guidance is elegant and powerful for fine-grained output control. It's particularly good for complex extraction, form filling, and structured reasoning. However, it has a steeper learning curve, its community is smaller than major frameworks, and Microsoft's investment has been inconsistent. For teams that need precise output control and are willing to learn a new paradigm, Guidance offers unique capabilities.