Arize Phoenix

Arize Phoenix is an open-source observability and evaluation platform for LLM applications and agents. Developers use it to capture traces, inspect retrieval and tool calls, organize datasets, run evaluations, and compare behavior over time. Phoenix is the open-source project; Arize AX is the hosted commercial platform shown on the vendor pricing page. Teams can self-host Phoenix, use AX SaaS, or negotiate Enterprise deployment.

Features and pricing

Tracing exposes model calls, retrieval, tools, latency, tokens, errors, and intermediate agent steps. Evaluations score properties such as relevance or correctness across datasets. Experiments enable repeatable comparisons when models, prompts, or retrieval settings change. Phoenix uses OpenTelemetry-oriented instrumentation and offers MCP tooling relevant to agent workflows.

The September 2026 Arize page lists AX Free at $0: 10 Signal issues, 25,000 traced spans, 1 GB ingestion monthly, 15-day retention, SaaS deployment, unlimited users, and unlimited evaluations. AX Pro is $50 monthly with 25 Signal issues, 50,000 spans, 10 GB ingestion, 30-day retention, unlimited users, and unlimited evaluations. Enterprise is custom-priced with unlimited issues, custom spans, ingestion and retention, and SaaS or self-hosted deployment. Open source avoids a license fee but still incurs storage and operations.

Benefits include self-hosting, detailed trace timelines, and repeatable regression tests. Limits include instrumentation effort, telemetry growth, and evaluator calibration. Fifty thousand spans can disappear quickly if one run creates dozens. Thirty-day Pro retention may be short for audits. LLM judges can be biased and need human labels. Traces can contain prompts, retrieved documents, personal data, or secrets unless redacted before collection.

Practical evaluation checklist

Run a two-week pilot on one bounded production-like workflow. Use at least 50 representative tasks and preserve a manual baseline. Record completion rate, factual or technical correctness, p50 and p95 latency, total usage cost, setup hours, correction time, and failure categories. Test permissions with an account that should not see the target data. Confirm export, deletion, outage handling, rate limits, support response, retention, subprocessors, regional processing, and whether customer content trains models. For every write-capable workflow, begin in a sandbox, require human approval, and document rollback before granting production access.

Do not choose Arize Phoenix from a feature checklist alone. Calculate annual cost at realistic volume with a 25% usage buffer and include implementation, monitoring, reviewer labor, and external model charges. Compare the same test set with langfuse, langsmith, helicone, braintrust, ai agent observability how to monitor debug and trace agents in production. A good purchase produces measurable net time savings after review and governance work; an impressive demo that creates more corrections does not.

Best fit

This tool is most useful when its specific integrations and workflow match an existing bottleneck. It is a weaker fit when the team cannot define an owner, representative evaluation set, permission boundary, or fallback process. Recheck vendor pricing and limits at purchase because cloud plans can change after this research date.