Cloudflare Workers AI

Deploy AI models across 300+ edge locations worldwide, leveraging Cloudflare's anycast network to route requests to the nearest available GPU for optimized performance. Latency varies by model size and GPU availability at each location — smaller models like Mistral 7B and Gemma 2B typically achieve median latencies well under 100ms from nearby locations, while larger models may route to a more limited set of GPU-equipped data centers. The system automatically balances proximity, capacity, and load to deliver the best available response time for each request.

Access 50+ curated open-source models including Meta's Llama 3.3 and Llama 4 Scout, Mistral 7B for efficient text generation, Google's Gemma for lightweight inference, and Stable Diffusion XL for image generation. The catalog spans text generation, embeddings (BGE), speech-to-text (Whisper), translation, and image models — all optimized for edge deployment and accessible through a unified API.

Pay-per-use pricing at $0.011 per 1,000 neurons with 10,000 neurons free daily. Neurons represent normalized compute units across different model types, providing predictable billing without idle costs or minimum commitments. Each model in the catalog publishes its neuron cost per request, enabling developers to estimate expenses before deploying. For example, a typical Llama 3.1 8B text generation request costs approximately 50 neurons (~$0.00055), while image generation models consume more neurons per request due to higher compute requirements.

Zero infrastructure management with automatic scaling, batching optimization, and resource allocation. Models warm automatically based on usage patterns to minimize cold start latency. The platform handles GPU provisioning, model loading, request queuing, and scaling entirely behind the scenes, allowing developers to treat AI inference as a simple API call without any operational burden.

Native integration with AI Gateway for observability and control, Vectorize for vector storage, Workers for edge computing, and R2/D1 for data storage, creating complete AI application stacks. AI Gateway provides unified caching, rate limiting, retry logic, fallback routing, and real-time analytics across both Workers AI and external providers. Vectorize enables semantic search and RAG pipelines, while D1 and R2 handle structured and object storage respectively.

Support for function calling, structured JSON outputs, reasoning tasks, vision processing, and multi-turn conversations with extended context windows for document processing applications. LoRA adapter loading enables fine-tuned model variants without redeploying base models, and batch processing handles high-volume offline workloads efficiently. The Agents SDK and Workflows product enable stateful, multi-step agent pipelines combining inference with durable execution.