TruLens

TruLens provides a library of pre-built feedback functions that automatically score LLM outputs on metrics like groundedness, context relevance, and coherence. These functions can use LLM-based evaluation or custom logic, and are extensible so teams can add domain-specific metrics. This replaces manual review with scalable, repeatable quality measurement.

TruLens supports OpenTelemetry for distributed tracing of AI agent and LLM application execution flows. Traces capture tool calls, retrieval steps, planning decisions, and model interactions, and can be exported to any OTel-compatible backend. This enables deep debugging of complex agentic workflows and integration with existing observability infrastructure.

The built-in leaderboard allows developers to compare different LLM application configurations across multiple evaluation metrics simultaneously. Teams can evaluate variations in prompts, models, hyperparameters, and retrieval strategies to objectively select the best-performing configuration based on data rather than subjective assessment.

TruLens is specifically designed to evaluate and trace AI agents, capturing the full execution flow including planning steps, tool calls, and intermediate reasoning. This provides visibility into where agents succeed or fail, enabling targeted improvements to agent behavior and reliability before production deployment.

Beyond built-in metrics, TruLens offers an extensible framework for defining custom evaluation criteria tailored to specific use cases. The platform surfaces weaknesses in application performance to inform iteration on prompts, hyperparameters, and architecture, creating a tight feedback loop between evaluation and improvement.