You are deciding whether every AI request in your organisation should go to the most capable model you license, or whether some of that traffic deserves a cheaper one. That decision used to be a rounding error. In October 2026 it became one of the largest levers on your AI budget.
On 7 October, Anthropic released Claude Haiku 5.5 at US$0.10 per million input tokens and US$0.50 per million output tokens for prompts up to 100,000 tokens, roughly 90% below the per-token price of Haiku 4.5, according to MarkTechPost’s launch analysis. OpenAI’s GPT-6 Luna lists the same short-context rates. When small models become this cheap and this capable, the question is no longer which model to buy. It is which request should go where. That discipline has a name: model routing.
What is LLM model routing?
LLM model routing is the practice of sending each AI request to the most suitable model for that task, based on rules or a trained classifier, instead of sending everything to one model. Simple requests go to small, cheap, fast models. Complex or high-risk requests go to frontier models. The goal is frontier-level quality at a fraction of the cost.
Think of it as triage. A hospital does not send every patient to the most senior consultant. A nurse assesses each case and directs it to the right level of care. A model router does the same for prompts, usually inside an AI gateway that sits between your applications and the model providers.
Routing is not the same as switching vendors. A single router can direct traffic across models from one provider, such as a small and a large model in the same family, or across several providers at once.
Why does model routing matter for enterprises in 2026?
Model routing matters now because the price gap between small and frontier models has widened sharply while small-model quality has risen. Haiku 5.5 and GPT-6 Luna both list US$0.10 per million input tokens. Enterprises that still send all traffic to their most expensive model are paying frontier prices for work a small model can now do.
The quality side of the equation moved too. Anthropic reports that Haiku 5.5 scores 72.4% on the OSWorld 2.1 offline subset, against 15.7% for Haiku 4.5, as summarised by MarkTechPost. The larger models in the same family still lead every benchmark row, which is exactly why routing, not wholesale replacement, is the right response.
Volume is the other driver. Pilots consumed thousands of requests a month. Production agents and copilots consume millions. AlphaSense tested Haiku 5.5 on a feature that handles about 8 million calls a week. At that scale, the price per million tokens stops being a technical detail and becomes a line the CFO reads.
If your organisation has already started an AI FinOps practice, routing is the most direct way to act on what that practice measures.
How does model routing work?
Model routing works through four main methods: fixed rules that map task types to models, classifiers that score each prompt's difficulty, cascades that try a small model first and escalate on low confidence, and semantic routers that match a request's meaning to predefined routes. Most enterprises begin with rules and add a classifier once they have usage data.
The four routing methods
--- Rule-based routing: the application declares the task, such as summarise, classify or draft contract clause, and each task type has a fixed model. Simple, predictable and easy to audit.
--- Classifier routing: a lightweight model predicts whether a prompt needs a strong model. LMSYS’s open-source RouteLLM framework trains such routers on human preference data.
--- Cascade routing: the small model answers first. If its confidence or a validation check fails, the request escalates to a larger model.
--- Semantic routing: the request is embedded and matched to the closest predefined intent, each mapped to a model and a policy.
These methods usually run inside an AI gateway. The category is maturing quickly: MarkTechPost ranked nine AI gateways on 14 October 2026, and open projects such as vLLM Semantic Router released new decision models the same week. The routing logic itself is becoming a commodity. The judgement about which tasks need which model is not.
How much can model routing save?
Published research shows routing can cut model costs by 35% to 85% while keeping about 95% of frontier-model quality, depending on the workload. LMSYS's RouteLLM study reported cost reductions above 85% on MT Bench, 45% on MMLU and 35% on GSM8K compared with sending every request to GPT-4.
Those figures come from the LMSYS RouteLLM study, which routed between one strong and one weak model. Treat the 85% figure as a best case. The 35% to 45% results on harder benchmarks are a more realistic planning range for mixed enterprise traffic.
A simple way to model your own saving
Assume 70% of your requests can move to a model that costs one-tenth as much per task, and 30% must stay on the frontier model. Your new cost is 0.30 plus 0.70 multiplied by 0.10, or 37% of the original. That is a 63% saving before any engineering cost.
Two adjustments keep the estimate honest. First, compare cost per completed task, not per token. Haiku 5.5’s new tokenizer counts the same text as roughly 30% more tokens than Haiku 4.5, which is why Anthropic estimates its average saving at about 75% rather than 90%. Second, include the cost of escalations, because every cascade that fails pays for two calls.
Which enterprise workloads should be routed to smaller models?
Route high-volume, well-defined, low-ambiguity tasks to smaller models: classification, extraction, summarisation of internal documents, first-line customer replies and agent sub-tasks. Keep open-ended reasoning, regulated advice, contract drafting, complex code and anything with irreversible external consequences on frontier models, with human review where the stakes require it.
--- Strong candidates for small models: ticket and email classification, invoice and form field extraction, meeting and call summaries, FAQ-style replies, translation of routine notices, and subagent steps such as pulling one figure from a filing.
--- Keep on frontier models: multi-step analysis, credit or investment narratives, legal drafting, board papers, complex software changes and decisions that touch money or personal data.
--- Route by risk, not only by difficulty: a short request can still be high stakes, such as a reply to a regulator.
Rogo, a financial research platform, illustrates the pattern: a Haiku 5.5 subagent pulls a revenue line from a 10-K filing while a larger model builds the final deck. The expensive model does the judgement. The cheap model does the fetching.
How should an enterprise introduce model routing?
Introduce model routing in four steps: inventory AI workloads by volume and risk, build an evaluation set that defines acceptable quality for each workload, start with rule-based routing on the two or three highest-volume tasks, then monitor cost per task and quality weekly and re-tune thresholds as models change.
The four-step routing framework
--- Step 1, inventory: list every AI use case with monthly request volume, average tokens, the model used and a risk rating. In most organisations, three or four workloads account for most of the spend.
--- Step 2, define quality: assemble 100 to 300 real, anonymised examples per workload with agreed correct answers. This becomes the test every candidate model must pass.
--- Step 3, route the obvious first: move the highest-volume, lowest-risk workload to a small model behind a rule, and keep a one-click fallback to the original model.
--- Step 4, monitor and re-tune: track cost per completed task, escalation rate and quality score every week. New model releases, like the one this month, should trigger a re-test, not a rebuild.
Routing across more than one provider has a secondary benefit: it reduces model concentration risk, because the same gateway that routes for cost can fail over when one provider has an outage or changes its terms.
How does model routing apply in Hong Kong industries?
In Hong Kong, routing fits wherever AI volume is high and tasks are repetitive: banks classifying customer messages, logistics firms extracting shipping document fields, and professional services firms summarising matter files. In each case a small model handles the bulk, while frontier models are reserved for advice, drafting and anything regulators or clients will read.
Financial services. A retail bank receives tens of thousands of customer messages a week in English and Chinese. A small model classifies each by intent and urgency. Complaints that may become regulatory matters, and any request involving investment suitability, route to a frontier model with a human reviewer, consistent with HKMA expectations on consumer protection in generative AI.
Logistics. A freight forwarder processes bills of lading, invoices and packing lists. Field extraction is routed to a small model with a validation check against the booking record. Only documents that fail validation escalate to a larger model, then to a human.
Professional services. An accounting firm summarises engagement files for partner review. Summaries run on a small model. The partner-facing risk memo, which carries professional liability, stays on the frontier model.
Most Hong Kong enterprises reach these models through cloud platforms such as Amazon Bedrock, Google Cloud or Microsoft Foundry, all of which list Haiku 5.5. A routing policy should also encode where personal data may go, so that requests covered by the Personal Data (Privacy) Ordinance only reach approved endpoints.
What are the common mistakes with model routing?
The most common mistakes are routing on token price instead of cost per task, skipping the evaluation set, letting quality drift go unmeasured, ignoring data-handling rules in the routing policy, and treating the router as a one-time project. Each mistake either erodes the saving or quietly lowers quality until a client or regulator notices.
--- Token-price thinking: a cheaper model that needs two attempts, or a tokenizer that counts 30% more tokens, can cost more per task.
--- No evaluation set: without agreed examples, quality debates become opinion, and the team reverts to the expensive model to be safe.
--- Silent drift: providers update models. Sample routed outputs every week and compare against the baseline.
--- Missing data rules: a router that optimises only for cost may send personal data to an endpoint your privacy team never approved.
--- One-off projects: the best route changes with every model release. Assign an owner.
When you present routing to the CFO, frame it as unit economics: cost per resolved ticket, per processed document, per summary. Pair it with the usage-based billing models your vendors now use, and the business case becomes a forecast rather than a promise.
Key facts at a glance
--- Definition: model routing sends each AI request to the most suitable model for that task.
--- Trigger in 2026: Claude Haiku 5.5, released 7 October 2026, lists US$0.10 input and US$0.50 output per million tokens up to 100,000-token prompts.
--- Research benchmark: LMSYS RouteLLM reported 35% to 85% cost reduction while keeping 95% of GPT-4 quality.
--- Four methods: rules, classifiers, cascades, semantic routing.
--- Four steps: inventory, define quality, route the obvious first, monitor and re-tune.
Conclusion: pay frontier prices only for frontier work
Model routing turns a pricing headline into a management discipline. The organisations that benefit from this month’s price cuts will not be the ones that switch everything to the cheapest model. They will be the ones that know, task by task, what quality they need and what it should cost, and that revisit the answer every time the market moves.
We understand AI. We understand you. With UD by your side, AI never feels cold.
Reviewed by the UD enterprise AI team. Prices and benchmark figures were checked on 15 October 2026 against the sources linked above. Model prices change often; confirm current rates with your provider before budgeting.
Put the right model on the right task
Now that you have the framework, the next step is identifying which of your workflows can run on lighter, cheaper AI without losing quality. We’ll walk you through every step, from workload inventory and quality testing to deployment and cost tracking, backed by 28 years of enterprise IT experience in Hong Kong.
