Comprehensive analysis of AI Gateway's strengths and weaknesses based on real user feedback and expert evaluation.
Native integration with Unity Catalog means permissions, audit logs, and lineage work identically to the rest of your Databricks data assets without extra IAM plumbing
OpenAI-compatible client interface allows existing application code to point at AI Gateway endpoints with minimal refactoring
Governs three distinct asset types (LLM endpoints, MCP servers, coding agents) in a single pane of glass â rare across the 870+ tools in our directory
No charges during Beta (confirmed on docs as of April 15, 2026), letting teams pilot full governance workflows before committing to enterprise pricing
Supports major coding agents including Cursor, Claude Code, Gemini CLI, and Codex CLI, covering the dominant agent tools developers use in 2026
Inference tables land as Delta tables in Unity Catalog, making audit and monitoring queries trivially accessible via SQL or notebooks
6 major strengths make AI Gateway stand out in the developer category.
Only available inside the Databricks platform â teams not already on Databricks cannot adopt AI Gateway as a standalone product
Currently in Beta, meaning feature set, APIs, and limits may shift before GA and enterprise SLAs may not apply
Two parallel versions exist (new AI Gateway in left nav vs. previous AI Gateway for serving endpoints), which creates documentation and migration ambiguity
Custom MCP server hosting requires packaging as a Databricks App, adding a layer of platform-specific deployment knowledge
Pricing is opaque enterprise-contract based with no public tier breakdown, making TCO comparisons against standalone gateways difficult
5 areas for improvement that potential users should consider.
AI Gateway has potential but comes with notable limitations. Consider trying the free tier or trial before committing, and compare closely with alternatives in the developer space.
If AI Gateway's limitations concern you, consider these alternatives in the developer category.
LiteLLM: Y Combinator-backed open-source AI gateway and unified API proxy for 100+ LLM providers with load balancing, automatic failovers, spend tracking, budget controls, and OpenAI-compatible interface for production applications.
Observe and control AI applications with caching, rate limiting, and analytics for any LLM provider.
Open-source LLM observability platform and API gateway that provides cost analytics, request logging, caching, and rate limiting through a simple proxy-based integration requiring only a base URL change.
The new AI Gateway, launched in Beta and visible in the left nav of the Databricks UI, is a broader central governance layer that covers LLM endpoints, MCP servers, and coding agents together. The previous AI Gateway was scoped only to model serving endpoints â external model endpoints, Foundation Model API endpoints, and custom model endpoints â and focused on usage tracking, payload logging, rate limits, and guardrails at the endpoint level. Both versions coexist in the documentation as of April 15, 2026, and Databricks recommends account admins enable the new version from the account console Previews page. Existing serving-endpoint governance continues to function while teams migrate.
According to the official documentation, AI Gateway features do not incur charges during the Beta period. Standard Databricks consumption charges for model serving, DBU usage, and underlying compute still apply, and once the product moves to GA, enterprise pricing will be set through standard Databricks contracts. Because pricing is not published publicly, prospective customers should request a quote through their Databricks account team. This makes the Beta window a good opportunity to pilot full governance before any commercial commitment.
The documentation explicitly calls out support for Cursor, Gemini CLI, Codex CLI, and Claude Code, which covers most of the dominant AI coding agents developers use in 2026. Integration routes each agent's model calls through the AI Gateway, so prompt/response payloads, token usage, and cost attribution are captured in Unity Catalog inference tables. This lets platform teams apply the same rate limits and guardrails to developer coding traffic that they apply to production LLM workloads. Other OpenAI-compatible agents can also point at AI Gateway endpoints using the OpenAI client.
AI Gateway supports three MCP deployment patterns: Databricks-managed MCP servers that expose native platform features, external MCP servers connected through managed connections, and custom MCP servers hosted as Databricks Apps. For each, AI Gateway enforces access control through Unity Catalog permissions and logs every MCP interaction for audit. Non-Databricks MCP clients can also connect to Databricks-hosted MCP servers through documented client connection flows. This unified governance is differentiated from pure LLM gateways â based on our analysis of 870+ AI tools, AI Gateway is the only offering that natively governs MCP servers alongside LLM endpoints.
AI Gateway emits two complementary telemetry streams into Unity Catalog. System tables capture endpoint-level usage and cost aggregates for budgeting and chargeback, while inference tables capture full request and response payloads as Delta tables for granular audit, replay, and quality monitoring. Both are queryable through standard SQL, notebooks, or BI tools, and inherit Unity Catalog row- and column-level access controls. Rate limits can be configured per endpoint to cap capacity and prevent runaway cost, and guardrails can be applied to block unsafe content across providers consistently.
Consider AI Gateway carefully or explore alternatives. The free tier is a good place to start.
Pros and cons analysis updated March 2026