LangSmith

LangSmith is the observability and evaluation platform built by LangChain Inc., designed specifically for developing, testing, and monitoring LLM applications. While Langfuse and other open-source alternatives exist, LangSmith's deep integration with the LangChain ecosystem — the most widely used LLM application framework — gives it a significant distribution advantage and first-party support for LangChain and LangGraph constructs.

The platform's tracing system captures every step of an LLM application's execution: model calls, retrieval operations, tool invocations, chain compositions, and custom spans. Traces are displayed as hierarchical trees with latency, token counts, costs, input/output payloads, and metadata at every node. For LangChain/LangGraph applications, tracing is nearly zero-configuration — adding a few environment variables enables automatic capture of all framework operations. Non-LangChain applications can use the LangSmith SDK directly or the OpenTelemetry integration.

LangSmith's evaluation system is its most differentiated feature. You create datasets of input-output examples, define evaluator functions (which can be LLM-based, heuristic, or human), and run your application against the dataset to get scored results. The platform tracks evaluation results over time, lets you compare runs across different prompts or model configurations, and provides statistical analysis of quality changes. This evaluation-driven development workflow — change something, evaluate, compare, iterate — is critical for production LLM applications where prompt changes can have unexpected effects.

The prompt management hub allows teams to version, test, and deploy prompts collaboratively. Prompts stored in LangSmith can be pulled dynamically at runtime, enabling prompt changes without code deployments. Combined with the evaluation system, teams can test prompt variations against evaluation datasets before deploying them to production.

For production monitoring, LangSmith provides dashboards for tracking latency, error rates, token usage, and costs across all LLM operations. The filtering and search capabilities allow you to find specific traces by metadata, user feedback, or content patterns. Rules-based alerts can notify teams of quality degradations or error spikes.

Pricing follows a tiered model: a free Developer tier with limited traces (5,000/month), a Plus tier for small teams with higher limits, and Enterprise tier with unlimited traces, SSO, RBAC, and dedicated support. The primary limitation is that LangSmith is a closed-source, hosted-only platform — there's no self-hosted option, which is a dealbreaker for some enterprises. The tight coupling with the LangChain ecosystem is both a strength and weakness: it's excellent if you use LangChain, but less compelling if you don't.