Llama Deploy

LlamaDeploy (formerly llama-agents) is LlamaIndex's production deployment framework for running multi-agent and RAG systems at scale. It transforms LlamaIndex applications from single-process scripts into distributed, production-grade microservices with built-in message queuing, service discovery, and orchestration.

The framework structures agent systems as a collection of services communicating through a central control plane. Each agent, tool, or pipeline becomes an independent service that can be deployed, scaled, and monitored separately. The control plane handles request routing, service registration, load balancing, and orchestration logic.

LlamaDeploy provides multiple message queue backends — RabbitMQ, Redis, Kafka, and a simple in-memory queue for development. This decouples services and enables reliable asynchronous communication between agents, which is critical for production systems where agents may have different processing speeds and resource requirements.

The deployment model supports both synchronous request-response patterns (user asks a question, gets an answer) and asynchronous workflows (kick off a multi-step research task that completes in the background). The framework manages workflow state, handles retries, and provides status endpoints for long-running tasks.

Integration with LlamaIndex is seamless — any LlamaIndex query engine, agent, or pipeline can be wrapped as a LlamaDeploy service with minimal code changes. For teams already using LlamaIndex, this provides the shortest path from prototype to production deployment.

The framework includes a Python SDK for programmatic deployment, Docker Compose configurations for local development, and Kubernetes manifests for cloud deployment. Monitoring endpoints expose service health, queue depths, and processing metrics.

LlamaDeploy fills a critical gap in the agent infrastructure stack. While frameworks like LangChain and LlamaIndex excel at building agent logic, deploying those agents as reliable, scalable services requires infrastructure that most teams build ad-hoc. LlamaDeploy provides this infrastructure as a ready-made solution, handling the distributed systems complexity so developers can focus on agent behavior.