Adaptive model routing

Send every request to the model it actually needs

Adaptive model routing in Fabric Gateway reads every prompt and matches it to an appropriate model in real time. Simple prompts stop paying for frontier models, your apps keep calling one endpoint, and output quality stays high.

Start buildingRead the docs
Requests routed to three model tiersMany incoming request threads of different weights pass through the Fabric gateway and fan out into three bundles: a large teal bundle to compact models, a medium blue bundle to balanced models, and a small amber bundle to frontier models.

Prompts vary in size, difficulty and reasoning. A classifier sorts each one, and Fabric sends it to a fast, balanced, advanced or frontier model, so heavy models only handle heavy work.

  • Compact
  • Balanced
  • Frontier

See what the classifier decides

Pick a prompt. The classifier labels it, Fabric routes it to the matching model, and you can see what that does to tokens and cost.

Classifiers supported: Jev and GPT-5.4 nano.

Incoming prompt
PromptTag a support ticket, about 300 tokens in
ClassifierLabels each prompt
  • Compact modelSimple tasks
    Selected
  • Balanced modelModerate reasoning
  • Frontier modelComplex reasoning and code
Routed to
Compact model
Labeled simple
Tokens used
340
460 on frontier
Cost vs all-frontier
95% lower
than always using frontier
promptTag a support ticket, about 300 tokens in
classsimple, task: classification
routeCompact model, inside your allowlist
tokens300 in, 40 out (frontier would write about 160)
cost95% lower than sending this prompt to the frontier model

Token counts and prices are made up for illustration. In Fabric, candidate models come from the allowlist you set on each route.

Spend less on every prompt

Pay for the model a prompt needs, and only the tokens it needs

Most AI traffic is simple. A classifier spots those prompts before any model is called and sends them to efficient models, keeping your most capable ones for work that needs them.

Lower price per token
Routine prompts are billed at compact-model rates, not frontier rates.
Fewer tokens generated
Right-sized models tend to answer simple prompts more concisely.
Less waste
No more defaulting to the biggest model "just in case," or burning tokens on retries and escalations.
Cost, every prompt on the frontier model330
Cost - with Adaptive Model Routing99
Output tokens, all frontier10,260
Output tokens, routed6,840
Illustration only. 22 prompts: 11 simple, 7 moderate, 4 hard. Cost in relative units; output tokens assume larger models answer simple prompts more verbosely. Your results depend on your traffic, models, and prompts.

One endpoint. No routing logic in your code.

Point your OpenAI- or Anthropic-compatible SDK at Fabric and call a routed model by name. The gateway decides which model answers, so swapping models, adding a provider, or changing goals is a configuration change, not a release.

app.py
import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.fabricgateway.ai/v1",
    api_key=os.environ["FABRIC_GATEWAY_KEY"],
)

reply = client.chat.completions.create(
    # a routed model you define in Fabric
    model="support-assistant",    messages=[
        {"role": "user", "content": "Summarize this ticket."},
    ],
)

Routing that stays inside your rules

The classifier runs in the same control plane that governs the rest of your AI traffic, so every routing choice happens under your policies.

Model allowlists
Only the models you approve are ever candidates for a route.
Budgets and token limits
Per-team, token-aware limits apply to routed traffic, so savings from routing are not given back by runaway usage.
Credentials stay in the gateway
Apps use scoped Fabric tokens. Provider keys are never exposed to them.
Policy on every request
Authentication, rate limits, and data controls run on each request alongside routing.

Supports OpenAI and Anthropic models today. More providers coming soon.

Questions, answered

  • What is adaptive model routing?

    It is a routing strategy that chooses a model for each prompt as it arrives. Instead of hardcoding one model, you call a routed model. The classifier reads the prompt, labels the task and its difficulty, and Fabric sends it to the best matching model from the ones you have approved.

  • Which classifiers are supported?

    Fabric supports Jev and GPT-5.4 nano as classifiers. The classifier reads each prompt and labels the task and its difficulty, and Fabric uses that label to pick the model.

  • How does it reduce cost and token consumption?

    Simple prompts are billed at compact-model rates instead of frontier rates, which lowers the price of every token. Right-sized models also tend to produce shorter answers and fewer reasoning tokens on simple work, and routing up front means fewer retries and manual escalations. Traces show tokens and cost per request, so you can measure the effect on your own traffic.

  • How is it different from fallback or load balancing?

    Fallback reacts after something fails. Load balancing splits traffic using fixed weights. Adaptive routing looks at the prompt itself and decides per request. Retries, circuit breakers, and fallbacks still apply as a safety net when a provider has trouble.

  • Will routing to smaller models hurt quality?

    Routing chooses models based on the capability matrix of the models you allow, so the quality floor is not compromised. The defaults should work for most teams, and every model selection decision is debuggable through traces, so you can refine your allowed list if needed.

  • Do I need to change my application code?

    Only the base URL and the model name. Fabric exposes OpenAI-compatible and Anthropic-compatible endpoints, so supported SDKs keep working. Changing models, providers, or routing later happens in configuration.

  • Where does routing run, and who sees my provider keys?

    Routing runs in the data plane, which you can self-host in your VPC, have Postman host, or run in a hybrid setup. Provider credentials are held by the gateway and injected at request time. With a self-hosted data plane they stay in your environment, and your applications, agents and Postman's control plane never see them.

Bring every AI interaction under control

Connect your first model, MCP server or internal API and apply consistent governance from one gateway.

Sign upRead the docs