Free account
Read this concept with a free account
Read this concept at no cost by signing in with Google.
no card needed
Core
Model Gateway and Routing: One Front Door for Every Model Call
A model gateway is a single internal service that every model call passes through, handling routing, fallback, budgets, logging, caching and policy. Routing sends each request to the most suitable model by task, difficulty, region or cost. This guide covers gateway functions, routing strategies and their evaluation, and the risks.
CONCEPT / AI ARCHITECTURE PATTERNS
RELATED CONCEPTS
PRACTISE THIS CONCEPT
AI System DesignWhy would a company put a gateway between its applications and the model providers it uses?→AI System DesignDesign the AI system for a food-delivery company's customer support, handling a million conversations a day.→LLM & RAG FundamentalsNew models accept enormous context windows. Should we drop RAG and just put all our documents in the prompt?→AI System DesignYour main model provider goes down for two hours during peak traffic. What should the product do, and how do you design for it in advance?→LLM & RAG FundamentalsWe need to extract data from 2 million receipt photos a month. Send images straight to a multimodal model, or run OCR first and send text?→LLM & RAG FundamentalsYou're launching a new AI product for customers in the US, EU and India. How do you choose the model stack?→