DSPy

DSPy (Declarative Self-improving Python) is a framework from Stanford NLP that flips the standard approach to working with language models. Instead of writing and tweaking prompts by hand, you write structured Python programs using declarative modules, and DSPy's optimizers automatically compile those programs into effective prompts or fine-tuned weights for your target LLM. Think of it as the jump from assembly to a high-level language, but for AI programming.

In DSPy, you define what you want — input/output signatures like question -> answer or context, question -> reasoning, answer — and compose modules that implement this logic. A module might chain a retriever with a language model, add a self-consistency check, or implement multi-hop reasoning. The key insight: you describe the structure of your AI program, not the exact text of your prompts. DSPy handles prompt engineering automatically.

This matters because hand-crafted prompts are brittle. Change your model from GPT-4 to Claude, and prompts that worked perfectly may degrade. Swap in a smaller model, and few-shot examples that fit GPT-4's context window need complete rework. DSPy programs are model-portable — the optimizer generates model-specific prompts from your program structure.

DSPy's optimizers are what make it genuinely different from other frameworks. Given a program, a metric function, and a small set of examples (often just 10-50), optimizers like BootstrapFewShot, COPRO, and MIPROv2 automatically find the best prompts, few-shot demonstrations, or fine-tuning data for your program. A typical optimization run costs about $2 and takes 20 minutes with a cloud LLM. The result: DSPy-optimized programs on small models (Llama2-13b) routinely outperform hand-prompted GPT-3.5 on the same tasks.

DSPy handles the patterns that matter in production AI: RAG pipelines where retrieval and generation need to work together effectively, multi-hop reasoning chains that break complex questions into retrievable sub-questions, classification with structured outputs and confidence scores, agent loops where the LM decides which tools to use and how to combine results, and complex QA systems that need to reason over multiple documents.

The framework integrates with every major LLM provider through LiteLLM — OpenAI, Anthropic, Google Gemini, Databricks, Ollama for local models, and any OpenAI-compatible endpoint.

DSPy has 25,000+ GitHub stars, an active Discord community, and backing from Stanford HAI. The research paper was published at ICLR 2024 with significant follow-up work. Production deployments span enterprise RAG systems, research pipelines, and commercial AI products. The framework is fully open-source under MIT license with no paid tier.