DSPy

DSPy (Declarative Self-improving Python) is a framework from Stanford NLP that fundamentally reimagines how developers build applications with large language models by replacing fragile hand-written prompts with composable, optimizable Python modules.

Instead of manually crafting prompt strings and iterating through trial-and-error, DSPy lets you define what your program should do using typed Signatures (like context, question -> reasoning, answer) and compose behavior from built-in modules such as ChainOfThought, ReAct, and ProgramOfThought. The framework then uses automatic optimizers — including MIPROv2, GEPA, BootstrapFewShot, and COPRO — to compile your program into highly effective prompts or fine-tuned weights, given just a metric function and a small set of labeled examples.

This approach delivers several transformative benefits. First, it makes LLM programs model-portable: switching from GPT-4 to Claude to Llama requires re-optimization rather than prompt rewriting, because the optimizer discovers provider-specific strategies automatically. Second, it enables systematic quality improvement — teams routinely achieve 10-30% accuracy gains over hand-prompted baselines by letting optimizers search the space of possible instructions and demonstrations. Third, it dramatically reduces inference costs by optimizing smaller models (Llama 3, Mistral, Phi) to match or exceed the accuracy of larger models at a fraction of the per-token price.

DSPy is published at ICLR 2024 and has accumulated over 25,000 GitHub stars. It is backed by Stanford HAI and maintained by an active research team that continues to release new optimizers (GEPA for reflective prompt evolution, SIMBA for scalable optimization) and capabilities including MCP tool support, streaming, async execution, and structured output generation. The framework integrates with all major LLM providers via LiteLLM, supports vector databases like Pinecone, Weaviate, Qdrant, and Chroma for RAG pipelines, and works with observability tools including LangSmith, Langfuse, and MLflow.

DSPy is best suited for teams building production AI systems who want measurable, reproducible quality improvements rather than subjective prompt tweaking. It excels in RAG pipelines, multi-hop reasoning, classification, information extraction, and agent workflows where output quality can be quantified with a metric.