LangSmith

LangSmith is the commercial control plane LangChain Inc. sells alongside its open-source frameworks. It is observability, evaluation and prompt management in one product, tightly integrated with LangChain, LangGraph and OpenAI's Agents SDK but usable from any stack via SDK or OpenTelemetry. Every LLM call, tool invocation and retrieval becomes a trace with token-by-token cost breakdown, full input/output payloads, latency, and any custom metadata you attach. You can filter traces by latency, error, user, tag, model, or prompt version, then send any interesting trace straight into a dataset for regression testing.

The evaluations layer is the reason most teams pay for LangSmith rather than rolling tracing themselves. It ships LLM-as-judge templates (factuality, harmfulness, helpfulness, custom rubrics), code-based checks for deterministic assertions, pairwise comparisons for shoot-outs, and human review queues so subject-matter experts can grade samples at scale. Eval runs produce summary scores and per-example diffs you can attach to a pull request, which means you can actually gate releases on quality rather than vibes. The Prompts feature versions prompts independently of code, supports A/B traffic splits in production, and lets non-engineers iterate on prompts from the web UI without redeploying.

Pricing: Developer is $0 with a generous monthly trace allowance for individuals. Plus is $39/user/month with team features and larger trace volume. Enterprise is custom and includes self-hosting, SSO, SOC 2 documentation, audit logs, and the LangGraph Platform tier that adds managed agent deployment, persistence, scheduling and human-in-the-loop UIs. Overages on Plus are usage-based per trace, so heavy production workloads should price out a year of traces before committing.

LangSmith's real moat is integration with the LangChain ecosystem: if you already use LangGraph for agents, instrumentation is one environment variable and you get nested run trees for free. For non-LangChain stacks it still works — OpenTelemetry, the Python and TypeScript SDKs, and a REST API cover most cases — but you do trade a little ergonomic polish.

If you are comparing options, look at Langfuse as the open-source self-hosted alternative, Arize Phoenix for an OSS observability path with ML lineage, Braintrust for an eval-first competitor, Helicone for a proxy-style observability layer that is cheaper at scale, and Opik from Comet for a similar feature set. My recommendation: start on Developer to instrument one agent, move to Plus once you have eval suites that block deploys, and only buy Enterprise when SSO, self-hosting or LangGraph Platform are firm requirements. For production rollouts, instrument before you ship, define a 'golden set' of 30–100 representative inputs early, and run that dataset on every prompt change so you catch regressions before users do.