Choosing AI Models: Open Source LLMs, Frontier Models, Thinking and Cost

If you are comparing an open source LLM with a closed frontier model, the choice has more parts than a leaderboard shows. A model has a price, a license, a context size, a thinking setting, a sampling setting and a place in your system. This guide covers each part with dated numbers, so you can pick a model for a job and know when to change it.

It has two readers. Leaders can read the framework, the cost sections and the checklist. Engineers can add the code, the settings tables and the test method. Prices and names change every few weeks, so each figure carries its check date and the review cycle is 90 days.

The Five Dials framework

Most model advice names a winner. That ages in a month. When we compare models we work through five settings instead, which we call the Five Dials. The grouping is our own engineering judgment, not a standard. Each dial has its own section below. Move a dial up only as far as your tests demand.

1. Model class. Frontier, mid-tier or small, and open weights or closed. This sets the price range and the ceiling on quality.
2. Thinking. Whether the model reasons before it answers, and how much. This is the biggest swing in cost per task.
3. Sampling. Temperature and related settings. On newer hosted models some are fixed for you.
4. Context and cache. How many tokens you send, and how much of that prefix is reused from the cache.
5. Route and check. Which tier gets each request, and what verifies the answer before it ships.

What OpenRouter shows

OpenRouter is a gateway that lets one API key reach many models. Its model list held 463 entries on 2026-10-27. Its rankings page publishes how many tokens each model processed. It is the best public view we found of what developers send traffic to. We also see four biases in it, listed below.

The snapshot, opened 2026-10-27

The page says "Usage data through Oct 26, 2026". The table below is the top ten models by weekly tokens as the page listed them. The change column is the page's own, and we read it as week over week.

The same page has other views. We read each of them, and each says something different.

• Latest full day. DeepSeek V4.1 Flash 1017 led with 5.42T tokens, ahead of GLM 5.3 Flash at 2.27T and DeepSeek V4.1 Flash at 2.21T.
• Trailing 30 days. The older DeepSeek V4.1 Flash led with 123T, GLM 5.3 Flash had 50.7T, and a stealth model named Space Bunny Alpha had 46.9T. Claude Opus 5.5 was tenth with 23.2T.
• Share of text requests by maker, latest full week. DeepSeek 26.5 percent, Google 20.1, OpenAI 18.5, Z.ai 10.7, Anthropic 5.5 and Qwen 4.7.
• Requests by model, 10K context window, week of Oct 19. The page's context-length summary ranked an "Others" group first. Two versions of Jev, a model TypeSafe describes as non-generative, took second and fifth place with 603 million and 259 million requests. What we make of it: request counts and token counts rank very different things.
• New and Trending tab. Kimi K3.1 Pro topped it as a new entry with 819B tokens.

We took one dated snapshot, so the change columns are our only trend signal. Our read of the shape: over the 30 days, usage spread across many cheap models from China and others, while two closed models, GPT-6 Luna and Claude Opus 5.5, held places in the top ten. Re-open the page before you quote any of these numbers.

Why these models rank

OpenRouter does not publish reasons. What follows is how we read the public facts. It is our hypothesis, labeled as such.

Price per token. On OpenRouter's model list, DeepSeek V4.1 Flash costs $0.14 per million input tokens and $0.42 per million output tokens. Claude Opus 5.5 costs $4 and $20 on Anthropic's pricing page. We divided the two and got about 29 times more for input and 48 times more for output. Agent loops and coding tools send a lot of tokens, so in our judgment price shows up fast.

A design built for cheap input. On DeepSeek's model card we read that the model uses 8 billion active parameters per token while it reads the prompt and 16 billion while it writes, and that this improves cost efficiency for input-heavy agent work. It reports a context of up to one million tokens and an MIT license. We treat that as the vendor's description of its own design.

Open weights. Seven of the ten models on the weekly list have a repository with weights on Hugging Face under the same name. The eighth, MiMo-V2.6-Flash, has Xiaomi-published Flash-RL and Flash-MOPD variants there, but we found no repository with that exact name when we searched, so we count it as open with a caveat. Anyone can host open weights. DeepSeek V4.1 Flash, GLM 5.3 Flash and Qwen3.8 27B all list temperature and top_p as supported settings on OpenRouter.

Free variants. Two entries on the weekly list carry the label "free": MiniMax M3.1 Flash and Nemotron 3 Ultra. A free variant costs the caller nothing, so it draws traffic that a paid model may not. NVIDIA's model is a 550 billion parameter mixture of experts with 55 billion active, under the OpenMDW 1.1 license. NVIDIA lists a minimum of eight B200 or H200 GPUs, or sixteen H100, to serve it yourself. Our read is that it ranks because it is free to try, not because many teams run it at home.

New snapshots. The first two entries are two snapshots of one model. OpenRouter's dated model IDs show the older version from 2026-09-10 and the newer one from 2026-10-17. The older snapshot fell 41 percent while the newer rose 382 percent, which we read as traffic moving from one to the other. Counting both, DeepSeek V4.1 Flash had 52.8T tokens, about 43 percent of the ten-model total when we summed the rows.

Qwen. Qwen is a different story. Its share of requests was 4.7 percent, and its 27B dense model has an Apache-2.0 license and, by our estimate, a size that one data-center GPU can serve (the 0.15 GB per-request figure elsewhere is our own arithmetic, not the card's). Its draw is the self-hosting route, which OpenRouter's count cannot see.

What the ranking does not tell you

We see four limits when we use this page.

1. Tokens are not quality. A model that writes long answers or loops in an agent racks up tokens without being better.
2. The users are a particular group. OpenRouter's audience is developers and app builders who want many models behind one key. The page's app data lists coding and agent tools first. The time window of that list was not clear to us, so we do not lean on it.
3. Free and promotional traffic inflates counts. A free variant and a launch week both skew a snapshot.
4. It cannot see private deployments. Companies that run models inside their own network never appear.

Frontier models or open weights

People say open source LLM when they mean a model whose weights they can download. The licenses differ a lot, so we say open weights here. The models in this guide carry MIT, Apache-2.0, OpenMDW 1.1 and custom licenses. We read each license before recommending a model, and we advise the same. For a worked case, see the Llama 4 license summary in our self-hosted LLM guide.

What the index says

Artificial Analysis publishes an Intelligence Index, a single score per model. On 2026-10-27 its main leaderboard put Claude Opus 5.5 at max effort first, at 58, Claude Sonnet 5.5 second at 56, and Kimi K3.1 Pro at 55. Its open weights page named MiMo-V2.6-Pro (46), DeepSeek V4.1 Flash 1017 at max effort (45) and GLM-5.3 at max effort (45) as the leading open models. Kimi K3.1 Pro did not appear in that open list, and we found no official Kimi K3.1 Pro repository on Hugging Face, so we do not treat its weights as public.

When we subtracted, the best open weights score was about 12 points below the best closed one (58 against 46). The site does not publish that gap itself, and we took the two numbers from two pages on the same day, so we treat 12 as approximate. What we see in it: the quality gap is real but narrower than the price gap.

Task by task

A single index hides where each side leads. DeepSeek's model card puts its open model next to Claude Opus 5.0 and other frontier models, at max reasoning effort, across many tests. Here is a slice of that table. These are the vendor's own numbers, and we note that the comparison model is Opus 5.0, not the newer Opus 5.5.

We compared the rows and the pattern is useful. The open model matches or beats the frontier model on Terminal-Bench 2.1 (90.6 against 89.1) and DeepSWE v1.1 (74.2 against 74.0). It is close on GPQA Diamond (90.9 against 93.4). It trails on the harder Terminal-Bench 3.0 (30.0 against 43.3) and 4.0 (31.2 against 51.8), and on Humanity's Last Exam (36.8 against 56.3). Our read is that the open model is strong on well-defined coding tasks and weaker on very long, open-ended ones. We would not rely on one vendor's table for that, and we recommend testing it on your own work.

Where each side leads, by task

This table is our engineering judgment, drawn from the sources above. It is where we would start testing, not a result.

The models behind these figures

The table below lists the text models named in this guide. Prices are per million tokens as listed on 2026-10-27 and will change. The rows come from reference data we keep in one place, so a fact changes once and every guide that uses it follows.

Small models

A small language model is a model that fits on an ordinary device and answers fast enough for one user. In the NVIDIA position paper, which we read in full, the authors define it that way and say that, at the time of writing, they would count most models under 10 billion parameters as small. They argue that small models are strong enough, better suited and cheaper for many calls inside agent systems, and that systems using several models are the natural choice when you also need general conversation.

We checked their estimates for three open source agents in the full text. The authors judged that about 60 percent of MetaGPT's model queries, 40 percent of Open Operator's and 70 percent of Cradle's could be handled by small models fitted to the task. Those are the authors' own estimates on three agents. They do not predict your numbers, and we treat them as a reason to run a test, not as a result.

For a use case with no network, read our post on offline AI for field teams. For sizing and hardware, read the self-hosted LLM guide.

Thinking and reasoning effort

Many models can reason before they answer. The reasoning text is made of tokens, and you pay for them. So thinking is a cost control as well as a quality control.

What thinking costs

In Anthropic's documentation we read that thinking tokens are billed as output tokens, and that the billed count does not match the visible count, because you pay for the full thinking and not the summary you see. You can read the billed thinking in usage.output_tokens_details.thinking_tokens. OpenAI's guide says reasoning tokens are not visible through the API, still take space in the context window, and are billed as output tokens. It reports them under output_tokens_details.reasoning_tokens.

Output tokens cost several times more than input tokens. On Claude Opus 5.5 the ratio is five to one. So a request that thinks for 12,000 tokens can cost more than the whole prompt. The chart shows one request on Opus 5.5, from the router sample later in this guide, as thinking grows. The thinking counts are examples we chose, and the arithmetic is from our router sample.

Artificial Analysis shows the same effect in its own runs. On 2026-10-27 we read its cost per task for each effort level. Claude Opus 5.5 (listed with a fallback model) scored 51 at medium effort for $1.34 per task, 54 at high for $1.82, 56 at xhigh for $3.46 and 58 at max for $5.98. Going from medium to max bought 7 index points and cost about 4.5 times as much when we divided $5.98 by $1.34. Whether that is worth it depends on what a wrong answer costs you.

The same page is why we do not compare on per-token price. Claude Sonnet 5.5 at max effort scored 56 at $5.46 per task, while Opus 5.5 at xhigh scored 56 at $3.46. A cheaper model that thinks longer can cost more per finished task.

Thinking controls by vendor

Every vendor now exposes its own control, and the names differ. We compiled this table from the vendor pages on 2026-10-27. We recheck it against the vendor page at build time, because these controls change with each release.

Two details we would flag. First, GLM's default is the highest level, so a default call is the most expensive one. Second, Anthropic's fixed budget is gone on current models. Extended thinking with budget_tokens is deprecated on the 4.6 models, and Claude 4.7 and later models reject it with a 400 error. We still see tutorials that teach budget_tokens. On the newest models you set thinking: {"type": "adaptive"} and choose an effort level.

When to turn thinking on

Turn thinking on, or raise effort, when the task has these traits:

• It needs several dependent steps, such as debugging, planning a migration or working through a calculation.
• It checks a document against a list of rules, where one missed rule matters.
• It is the planning step of an agent, where an early wrong turn costs many later calls.
• The input is ambiguous and the model must weigh options.

Turn it off, or lower effort, when the task looks like this:

• It labels, routes or extracts fields from clean text.
• It rewrites or formats something you already wrote.
• It is a short chat turn where speed matters. OpenAI's guide lists voice, fast information retrieval and classification as cases for none.
• It processes tool results inside a loop. Anthropic notes that follow-up requests that only process tool results can skip thinking at any level.

We reach for lowering effort first, since it is a calibrated control. Anthropic advises lowering effort before you try to talk the model into thinking less, and says prompt wording is less reliable.

Cheap is not always cheaper

Lower effort can backfire in agents. Qwen's model card warns that in multi-turn agent tasks, lower reasoning effort does not always reduce total time. It can lead to insufficient analysis, more failures and repeated retries, which raise total latency and token use. Anthropic says something similar for its smallest model. At low effort, in long agent prompts, Claude Haiku 5.5 is more likely to skip a search, stop early or skip a check.

We recommend testing effort on the whole task, not on one call. Run the same 30 cases at two levels, and count finished tasks, total tokens and total cost. Choose the lower level only if the finished count holds.

Effort and the cache

One more cost rule we noted. Changing the thinking configuration or the top-level effort between requests invalidates the prompt cache for the message blocks. Anthropic's example shows it: with the cache in place, a third request at a different effort re-created the whole 3,546-token cache instead of reading it.

We would pick one effort level per conversation and keep it. On models that support it, a per-message effort change keeps the cache, but it needs a beta header at the time of writing.

Temperature and sampling

A model scores every possible next token. Sampling turns those scores into odds, and picks one. Temperature divides the scores before they become odds. A low value sharpens the odds toward the top choice. A high value flattens them. The top_p setting cuts off the unlikely tail, keeping the smallest set of tokens whose odds add up to the chosen value.

A small demonstration

The program uses invented scores for one imaginary next word, so there is no model involved. It prints the odds at four temperatures and shows what top_p keeps.

"""What temperature does to next-token odds. Pure maths, no model. A model scores every candidate token (a logit). Sampling turns scores into odds withsoftmax(logit / temperature). Low temperature sharpens the odds, high temperature flattens them.The scores below are made up for one imaginary next word."""import math TOKENS = ["refund", "credit", "exchange", "apology", "escalation"]LOGITS = [3.0, 2.2, 1.4, 0.6, -0.2]  def softmax(logits, temperature):    scaled = [x / temperature for x in logits]    peak = max(scaled)    exps = [math.exp(x - peak) for x in scaled]    total = sum(exps)    return [e / total for e in exps]  def top_p_keep(probs, p):    """Smallest set of tokens whose odds add up to at least p (nucleus sampling)."""    order = sorted(range(len(probs)), key=lambda i: -probs[i])    kept, running = [], 0.0    for i in order:        kept.append(i)        running += probs[i]        if running >= p:            break    return sorted(kept)  print(f"{'temp':>5} " + " ".join(f"{t:>10}" for t in TOKENS))for temp in (0.2, 0.7, 1.0, 1.5):    probs = softmax(LOGITS, temp)    print(f"{temp:>5} " + " ".join(f"{p:>10.1%}" for p in probs)) probs = softmax(LOGITS, 1.0)kept = top_p_keep(probs, 0.95)print("top_p 0.95 at temperature 1.0 keeps:", ", ".join(TOKENS[i] for i in kept))

We ran it with Python 3.13.3 on 2026-10-27. This is the real output:

 temp     refund     credit   exchange    apology escalation  0.2      98.2%       1.8%       0.0%       0.0%       0.0%  0.7      68.3%      21.8%       6.9%       2.2%       0.7%  1.0      56.1%      25.2%      11.3%       5.1%       2.3%  1.5      44.4%      26.1%      15.3%       9.0%       5.3%top_p 0.95 at temperature 1.0 keeps: refund, credit, exchange, apology

At 0.2 the model picks the top word 98 percent of the time. At 1.5 the top word falls to 44 percent and rare words get a real chance. That is all temperature does. It adds variety. It does not add knowledge.

What vendors allow now

The old advice was to set temperature to 0 for factual work and 0.7 for creative work. When we read the current vendor rules, they had moved on.

On hosted Claude models, Anthropic's documentation says that on Opus 5.5, Sonnet 5.5, Haiku 5.5 and several earlier models, non-default temperature, top_p or top_k values return a 400 error on every request. It adds that from API version 2026-10-21, a request that includes any of the three at any value returns a 400 error on every model. In our reading, the vendor has taken the dial away.

The OpenRouter model list points the same way for OpenAI. In the list we read, the listing for GPT-6 Luna does not include temperature among supported parameters, and neither does its listing for Claude Opus 5.5. The listings for DeepSeek V4.1 Flash, GLM 5.3 Flash, Qwen3.8 27B and Nemotron 3 Ultra do. We recommend confirming on the vendor's page before you rely on it.

Open model cards go further and recommend a higher value than the old habit of 0. We collected the settings they publish in this table.

What we see in it: for reasoning models the vendor setting is usually 1.0, and a low temperature may hurt more than help. For non-reasoning extraction we prefer a low value such as 0 to 0.3. That is our recommendation and not a vendor rule, so we test it on each task.

Self-hosting adds a trap. A server can apply a default sampling file from the model repository without you asking. Our self-hosted guide covers the vLLM flag for that.

A settings checker

Because rules differ by model and change often, we keep them in code. We wrote this checker to read a request as a plain dictionary and report what the vendor pages say would fail or hurt. It makes no network call. The rules are the ones cited above, as we read them on 2026-10-27.

"""Check request settings against what each vendor documents. Offline: it only reads dicts. The rules are copied from the vendor pages listed in the guide (read 2026-10-27). They changewith every model release, so treat this file as a pattern and re-check the pages.""" ANTHROPIC_NEW = {"claude-opus-5-5", "claude-sonnet-5-5", "claude-haiku-5-5"}ANTHROPIC_EFFORT = {"low", "medium", "high", "xhigh", "max"}  def check(model: str, params: dict) -> list[str]:    problems: list[str] = []    if model in ANTHROPIC_NEW:        for key in ("temperature", "top_p", "top_k"):            if key in params:                problems.append(f"{key}: non-default sampling values return a 400 error on {model}")        thinking = params.get("thinking", {})        if thinking.get("type") == "enabled" or "budget_tokens" in thinking:            problems.append("thinking: a fixed budget_tokens returns a 400 error; use type adaptive and output_config.effort")        if model == "claude-opus-5-5" and thinking.get("type") == "disabled":            problems.append("thinking: disabled returns a 400 error on claude-opus-5-5; lower effort instead")        effort = params.get("output_config", {}).get("effort")        if effort is not None and effort not in ANTHROPIC_EFFORT:            problems.append(f"output_config.effort: {effort!r} is not one of {sorted(ANTHROPIC_EFFORT)}")        if params.get("max_tokens", 0) < 8000 and effort in {"high", "xhigh", "max"}:            problems.append("max_tokens: thinking counts toward it, so leave room at high effort")    elif model.startswith("qwen3.8"):        thinking = params.get("enable_thinking", True)  # on by default        want = (1.0, 0.95) if thinking else (0.7, 0.8)        got = (params.get("temperature", want[0]), params.get("top_p", want[1]))        if got != want:            problems.append(f"sampling: Qwen recommends temperature {want[0]} and top_p {want[1]} with thinking {'on' if thinking else 'off'}")    elif model.startswith("deepseek-v4.1"):        if params.get("temperature", 1.0) != 1.0:            problems.append("temperature: DeepSeek recommends 1.0")        effort = params.get("reasoning_effort")        if effort is not None and not 1 <= effort <= 100:            problems.append("reasoning_effort: DeepSeek V4.1 takes a number from 1 to 100")    else:        problems.append(f"{model}: no rules written; read the model card before you set anything")    return problems  CASES = [    ("claude-sonnet-5-5", {"temperature": 0.2, "max_tokens": 4000}),    ("claude-opus-5-5", {"thinking": {"type": "enabled", "budget_tokens": 8000}, "max_tokens": 16000}),    ("claude-opus-5-5", {"thinking": {"type": "adaptive"}, "output_config": {"effort": "medium"}, "max_tokens": 16000}),    ("qwen3.8-27b", {"enable_thinking": True, "temperature": 0.2}),    ("deepseek-v4.1-flash", {"temperature": 1.0, "reasoning_effort": 150}),] if __name__ == "__main__":    for model, params in CASES:        issues = check(model, params)        print(f"{model}: {'ok' if not issues else 'problems'}")        for issue in issues:            print(f"   - {issue}")

We ran it with Python 3.13.3 on 2026-10-27. This is the real output:

claude-sonnet-5-5: problems   - temperature: non-default sampling values return a 400 error on claude-sonnet-5-5claude-opus-5-5: problems   - thinking: a fixed budget_tokens returns a 400 error; use type adaptive and output_config.effortclaude-opus-5-5: okqwen3.8-27b: problems   - sampling: Qwen recommends temperature 1.0 and top_p 0.95 with thinking ondeepseek-v4.1-flash: problems   - reasoning_effort: DeepSeek V4.1 takes a number from 1 to 100

Routing and right-sizing

Routing means sending each request to the smallest model that handles it. Right-sizing means picking the thinking level and context size to match. Together they are, in our judgment, the largest saving most teams can make, and they need no new model.

The survey data shows how common the idea is becoming. The Stack Overflow 2026 Developer Survey article lists using different models for different work at 27 percent among cost controls. The same survey says developers care more about quality (75 percent) than cost (38 percent) when picking models. Both are true, and our view is that routing is how you serve both.

Tiers and rules

We recommend three tiers: a small or open model, a mid-tier model and a frontier model. The router picks a tier with plain rules, tries it, checks the answer, and goes up one tier if the check fails.

We prefer to start with rules, not with a model that decides. Rules are cheap, fast and readable, and you can test them. Add a small classifier model later if rules stop being enough. OpenRouter's Auto Router shows the other approach. It classifies each prompt into one of about 30 task types, ranks models by their trailing seven-day share of spend for that task, applies a cost band you choose, and routes to the top candidates. We read it as a good option when you want the market's choice. It does not know your quality bar, so we would still run our own tests.

A router and cost estimator you can run

We wrote the sample below to do two jobs. It picks a tier for each task by rule, and it estimates monthly cost from token counts, prefix caching, thinking tokens and the batch discount. The prices are in a JSON file so you can change them. The workload is invented: four task types with token counts we chose.

{  "asOf": "2026-10-27",  "note": "USD per 1M tokens. Anthropic rows from platform.claude.com/docs/en/about-claude/pricing. DeepSeek row from the OpenRouter model listing (deepseek/deepseek-v4.1-flash). Cache and batch multipliers from the same pages. Check them again before you rely on them.",  "tiers": {    "small": "deepseek-v4-1-flash",    "mid": "claude-sonnet-5-5",    "frontier": "claude-opus-5-5"  },  "models": {    "deepseek-v4-1-flash": {"in": 0.14, "out": 0.42, "cache_read": 0.0042, "cache_write_mult": 1.0, "batch_mult": null, "open_weights": true},    "claude-haiku-5-5": {"in": 0.10, "out": 0.50, "cache_read": 0.01, "cache_write_mult": 1.25, "batch_mult": 0.5, "open_weights": false},    "claude-sonnet-5-5": {"in": 2.00, "out": 10.00, "cache_read": 0.10, "cache_write_mult": 1.25, "batch_mult": 0.5, "open_weights": false},    "claude-opus-5-5": {"in": 4.00, "out": 20.00, "cache_read": 0.20, "cache_write_mult": 1.25, "batch_mult": 0.5, "open_weights": false}  }}

"""Offline model router and monthly cost estimator. No network and no API key. It does two things:  1. route(task): pick a tier (small, mid, frontier) with a plain rule and a reason.  2. cost(task, model): estimate monthly spend from token counts, prefix caching,     thinking tokens and the batch discount. The workload below is made up to show the method. Replace it with counts from your logs."""import jsonfrom dataclasses import dataclassfrom pathlib import Path CFG = json.loads((Path(__file__).parent / "prices.json").read_text())PRICES, TIERS = CFG["models"], CFG["tiers"]CACHE_HIT_RATE = 0.9  # share of requests that find the shared prefix already cached  @dataclassclass Task:    name: str    requests: int            # per month    input_tokens: int        # per request, including the shared prefix    prefix_tokens: int       # stable start of the prompt (instructions, examples, documents)    output_tokens: int       # visible answer    thinking_tokens: int     # extra reasoning tokens, billed as output (0 when thinking is off)    needs_reasoning: bool    # multi-step logic, planning or checking against rules    high_risk: bool          # a wrong answer is costly or hard to catch    batch_ok: bool           # can wait hours for the result  def route(t: Task) -> tuple[str, str]:    """Plain rules, easiest to read first. Returns (tier, reason)."""    if t.needs_reasoning and t.high_risk:        return "frontier", "reasoning and high risk"    if t.needs_reasoning:        return "mid", "reasoning, errors are caught later"    return "small", "no reasoning needed"  def cost(t: Task, model: str) -> float:    """Monthly USD for task t on a model."""    p = PRICES[model]    fresh = t.input_tokens - t.prefix_tokens    hit = CACHE_HIT_RATE    prefix_per_m = hit * p["cache_read"] + (1 - hit) * p["in"] * p["cache_write_mult"]    per_request = (        fresh * p["in"] + t.prefix_tokens * prefix_per_m + (t.output_tokens + t.thinking_tokens) * p["out"]    ) / 1e6    total = per_request * t.requests    if t.batch_ok and p["batch_mult"]:        total *= p["batch_mult"]    return total  def report(tasks: list[Task]) -> None:    frontier = TIERS["frontier"]    print(f"{'task':<20}{'tier':<10}{'model':<22}{'routed $/mo':>12}{'all-frontier':>14}{'saving':>9}")    routed_sum = base_sum = 0.0    for t in tasks:        tier, why = route(t)        model = TIERS[tier]        # Baseline: the same task on the frontier model with no extra thinking, which favours the baseline.        base_task = Task(**{**t.__dict__, "thinking_tokens": t.thinking_tokens if t.needs_reasoning else 0})        r, b = cost(t, model), cost(base_task, frontier)        routed_sum += r        base_sum += b        print(f"{t.name:<20}{tier:<10}{model:<22}{r:>12,.2f}{b:>14,.2f}{1 - r / b:>9.0%}   ({why})")    print(f"{'TOTAL':<52}{routed_sum:>12,.2f}{base_sum:>14,.2f}{1 - routed_sum / base_sum:>9.0%}")  WORKLOAD = [    Task("ticket-triage", 150_000, 900, 700, 30, 0, False, False, False),    Task("doc-summary", 30_000, 7_000, 3_000, 500, 0, False, False, True),    Task("contract-draft", 8_000, 14_000, 6_000, 1_200, 4_000, True, False, False),    Task("exception-review", 1_500, 20_000, 8_000, 2_000, 12_000, True, True, False),] if __name__ == "__main__":    print(f"prices as of {CFG['asOf']}, cache hit rate {CACHE_HIT_RATE:.0%}")    report(WORKLOAD)

We ran it with Python 3.13.3 on 2026-10-27. This is the real output:

prices as of 2026-10-27, cache hit rate 90%task                tier      model                  routed $/mo  all-frontier   savingticket-triage       small     deepseek-v4-1-flash           7.96        281.40      97%   (no reasoning needed)doc-summary         small     deepseek-v4-1-flash          24.70        420.60      94%   (no reasoning needed)contract-draft      mid       claude-sonnet-5-5           560.32      1,120.64      50%   (reasoning, errors are caught later)exception-review    frontier  claude-opus-5-5             500.16        500.16       0%   (reasoning and high risk)TOTAL                                                   1,093.14      2,322.80      53%

Three things we read in that output.

• The saving comes from volume. Ticket triage is 150,000 requests a month, so moving it to a cheap model saves $273 of the $281 it would cost on Opus 5.5.
• The hard cases stay on the frontier model. The exception review is only 1,500 requests, and it is also the one step where the router refuses to downgrade. Its cost is the same in both columns.
• Thinking is most of the mid and top lines. The contract draft carries 4,000 thinking tokens and the exception review 12,000. Both are billed as output.

The all-frontier column adds no thinking to tasks that need none, which favors the baseline, so we regard 53 percent as a cautious figure for this workload. It says nothing about yours, because every input is ours.

We also wrote test_router.py, which checks the code against numbers we worked by hand. It asserts, for example, that ticket triage costs $7.96 on DeepSeek and $281.40 on Opus 5.5, and that the batch discount halves an Anthropic price. It prints router checks passed.

Check and escalate

In our judgment a router without a check only moves risk around. The check can be a schema test, a rule such as "every quoted amount appears in the source", a confidence score, or a second model that reviews the first. When the check fails, retry once on the next tier. When the top tier fails, send the case to a person.

Log the tier, the check result and the cost for every call. After a month the log shows which tasks escalate often and which rules send hard cases down by mistake.

Caching and cost maths

Token prices have three levers beyond the base rate: caching, batching and context length. Each is documented, and we think each is easy to get wrong.

What the vendors publish

Anthropic's rules, as we read them on 2026-10-27, are these. A cache write with the default five-minute lifetime costs 1.25 times the base input price. A one-hour write costs 2 times. A cache read costs 0.1 times, and 0.05 times on Opus 5.5 and Sonnet 5.5. The cache refreshes at no cost each time the content is used. The minimum cacheable prompt is 512 tokens on Opus 5.5, Sonnet 5.5 and Haiku 5.5. Changing a tool definition invalidates the whole cache.

OpenAI's pricing page lists a cached-input price beside each input price. For GPT-6 Luna it is $0.01 against $0.10, for GPT-6 Sol $0.20 against $2.00, and for GPT-6 Astra $1.00 against $10.00. Above 272,000 input tokens it lists higher prices. The OpenRouter list shows a cache read price for DeepSeek V4.1 Flash of $0.0042 per million tokens, against $0.14 for input.

Both Anthropic and OpenAI give batch pricing at half the standard rate for work that can wait. Claude Haiku 5.5 charges more for prompts over 100,000 tokens, at $0.50 input and $2.50 output instead of $0.10 and $0.50.

Break-even for the cache

A cache write costs more than a plain call, so a prefix used once loses money. The second call is where it turns. We wrote the program below to compute the cost of N calls that share a 20,000-token prefix on Opus 5.5.

"""When does prompt caching pay? Cost of N calls that share one prompt prefix. Anthropic multipliers: 5-minute cache write 1.25x the input price, cache read 0.05x on Opus 5.5.Calls must arrive within the cache lifetime for the reads to hit."""INPUT_PER_M = 4.00      # Claude Opus 5.5, USD per 1M input tokensWRITE_MULT, READ_MULT = 1.25, 0.05PREFIX = 20_000         # tokens in the shared prefix  def uncached(n: int) -> float:    return n * PREFIX * INPUT_PER_M / 1e6  def cached(n: int) -> float:    return PREFIX * INPUT_PER_M / 1e6 * (WRITE_MULT + READ_MULT * (n - 1))  print(f"prefix {PREFIX:,} tokens on Opus 5.5 at ${INPUT_PER_M:.2f} per 1M input tokens")print(f"{'calls':>5}{'no cache $':>12}{'cached $':>10}{'saving':>8}")for n in (1, 2, 3, 5, 10, 50, 100):    u, c = uncached(n), cached(n)    print(f"{n:>5}{u:>12.4f}{c:>10.4f}{1 - c / u:>8.0%}")

We ran it with Python 3.13.3 on 2026-10-27. This is the real output:

prefix 20,000 tokens on Opus 5.5 at $4.00 per 1M input tokenscalls  no cache $  cached $  saving    1      0.0800    0.1000    -25%    2      0.1600    0.1040     35%    3      0.2400    0.1080     55%    5      0.4000    0.1160     71%   10      0.8000    0.1360     83%   50      4.0000    0.2960     93%  100      8.0000    0.4960     94%

The first call is 25 percent worse. In our run it wins from the second call. The condition is that the calls land inside the cache lifetime, which is five minutes by default and refreshes each time the cache is read. A nightly job that runs once will never see a hit. A support bot with steady traffic will.

What to put in the prefix

The cache works on the start of the prompt, so we put the stable parts first and the changing parts last. Stable parts are the instructions, the tool definitions, the examples and any reference document. Changing parts are the user's question and the retrieved passages for this request.

For a wider list of cost techniques, see our post on reducing LLM costs. For when owning hardware beats paying per token, use the break-even calculator in the self-hosted LLM guide.

Settings for accuracy and fact-checking

In our judgment no model setting makes a model truthful. Accuracy comes from what you give the model and what you check afterward. Settings only change how much room the model has to wander.

What a hallucination benchmark does and does not show

We read Vectara's leaderboard, last updated 2026-10-22. It tests grounded summarization. Each model summarizes a document using only the facts in it, and a judge model scores whether the summary stays consistent with the source. The test set is private, with over 7,700 articles, and the rate is 100 minus the factual consistency rate. Vectara says it set temperature to 0 where possible.

The results surprised us. The lowest rates were 1.8 percent for a 32 billion parameter model from Ant Group and 3.1 percent for OpenAI's GPT-5.4 nano. Among the flagship OpenAI and Anthropic entries we looked up, GPT-6 Sol scored 6.5 percent, GPT-6 Astra 8.7, Claude Opus 5.5 9.9 and GPT-5.6 Sol 12.4. The bottom of the list reached 24.2 percent.

Our read: it tests faithful summarizing of a supplied document, not open-domain facts and not your documents. It does show two things we find useful. Small models can be very faithful when the source is in front of them. And a bigger, newer model does not guarantee faithfulness: the 3.1 percent small model beat every flagship entry we looked up.

A settings checklist for accuracy

We think the last row matters most. OpenRouter lists DeepSeek V4.1 Flash as separate dated versions, and traffic moved between them within weeks. A model alias that quietly points to a new version can change answers. We pin the version where the vendor lets us, and keep a fixed test set. If the answers must come from your own documents, our post comparing RAG, fine-tuning and prompt engineering shows when retrieval beats a model change.

Image and video models

Image and video models are a smaller part of most business projects, and the facts age even faster. In this section we report dated numbers from one source, read on 2026-10-27, and label what the vendors claim.

How the leaderboards work

Artificial Analysis ranks image and video models by Elo from pairwise human votes. Voters compare two outputs from the same prompt without knowing which model made which. The image page says votes come from validated private evaluators. The video page fixes Kling 3.0 1080p (Pro) at 1000 as the reference. Elo measures what voters preferred. It does not tell you whether a model suits your brand, follows your layout or fits your budget.

Image models, 2026-10-27

On the text-to-image page we read, the top ten were led by OpenAI's GPT Image 2.5 Sunburst (max) at Elo 1222, followed by a second GPT Image 2.5 variant at 1207 and GPT Image 2 (high) at 1186. Google's Nano Banana 2.1 came fourth at 1176, then Microsoft's MAI-Image-2.6 and Grok Imagine Image 2.0 at 1166 each.

Price changes the picture, as we found when we divided. The page lists API cost per 1,000 images at 1024 by 1024 with default settings. GPT Image 2.5 Sunburst was $210.7 and Nano Banana 2.1 was $33.6. The leader costs about six times as much per image for 46 more Elo points by our division. No model in the top 20 had open weights. The best open one, FLUX 3 Image Klein, ranked 21st at 1043.

Video models, 2026-10-27

On the text-to-video page we read, Alibaba's Wan 3.0 led at 1174, ahead of Utopai X at 1168 and MiniMax H3 (768p) at 1158. HeyGen Video 1 (1157), Dreamina Seedance 2.5 (1156) and FLUX 3 Max (1155) were bunched within two points of each other. The page lists API prices per minute of 1080p video for most of them. Wan 3.0 was $12.00, MiniMax H3 at 768p was $4.80, HeyGen Video 1 was $2.40 and Seedance 2.5 was $34.12. Utopai X had no API.

MiniMax H3 was the only open weights model in the top 20, and its Hugging Face card names the MiniMax H3 Community License, so read it. The top six video scores sit within 19 points of each other, so we would weigh price, clip length, audio support and license above rank.

Choosing an image or video model for work

We would run four checks.

1. Cost per usable asset. Generate 20 assets on your real prompts, count how many you would publish, and divide the spend by that count.
2. Fit with your brand. Check text, logos, faces and product shapes on your own examples.
3. License and rights. Read the terms on output use, and ask a lawyer about training data and likeness. This is not legal advice.
4. Where it runs. If assets contain confidential designs, you may need an open weights model on your own hardware, which narrows the list to a few lower-ranked models.

A selection framework you can run

The decision tree below is how we turn the Five Dials into a first pass. It does not replace testing. It tells us which test to run first.

1. Write the task and the bar

   Describe the task in one paragraph, with the input, the output and what counts as correct. Set a pass mark, such as 95 percent of labeled cases correct, and say what a wrong answer costs.

2. Build a test set

   Collect 30 to 50 real cases for a first pass, and 200 or more for a decision you will pay for. Include the hard ones and the odd ones. Keep the set fixed.

3. Pick three candidates

   Take one model from each tier, with at least one open weights model if data or volume points that way. Use OpenRouter and the Artificial Analysis pages to shortlist, not to decide.

4. Run each at two effort levels

   Run every candidate at a low and a high effort level, with thinking off where it can be turned off. Keep sampling at vendor settings. Record the finished-task count, the tokens, the thinking tokens and the cost.

5. Choose the cheapest passing setup

   Pick the cheapest model and effort that clears your bar on cost per finished task. If none passes, move up a tier or improve the prompt and the retrieved context first.

6. Add routing, a check and a log

   Send easy cases to the cheap tier, add a check and escalation, and log cost per call. Re-run the test set every 90 days and whenever a model version changes.

A worked example, clearly invented

This is a worked example on our invented workload. Take a support operations team with the four task types in the router sample. The team starts by sending everything to Claude Opus 5.5, a $2,322.80 monthly bill in the router output above.

Triage and summaries are narrow, so the team tests DeepSeek V4.1 Flash against its own 200 labeled tickets and passes it. Those two tasks fall from $702 to about $33 a month. The contract drafts need reasoning, but a person reviews them, so the team tests Sonnet 5.5 at medium effort and passes it, which halves that line. The exception review stays on Opus 5.5 with a check, because a miss is costly. The new total is $1,093.14, or 53 percent lower.

No client is behind this example. The savings assume each cheaper model passes the test, and the number is not a forecast. The method is the point.

Production checklist

Use this list before a model choice goes live.

• [ ] The task, the pass mark and the cost of a wrong answer are written down.
• [ ] A fixed test set of real cases exists and the result is stored with the date.
• [ ] Each candidate was tested at two effort levels, with cost per finished task recorded.
• [ ] The model version is pinned where the vendor allows it.
• [ ] Sampling settings are checked against the vendor page, in code, before a request goes out.
• [ ] max_tokens leaves room for thinking and the answer.
• [ ] The stable part of the prompt comes first, and the cache hit rate is measured.
• [ ] Thinking tokens are logged apart from the visible output.
• [ ] A check runs on every answer, and a failed check escalates once, then goes to a person.
• [ ] Open weights came from the official repository at a pinned revision.
• [ ] The data classes allowed to reach each provider are listed and enforced in code.
• [ ] The license for any open weights model was read, and the answer is on file.
• [ ] Prices in your estimate carry a date, and the review is on the calendar for 90 days.
• [ ] A fallback model is chosen in case the first one is deprecated or slow.

A log template

One row per test run keeps the choice repeatable.

Common mistakes

• Choosing from one leaderboard. An index mixes many tests. Your task is one test, and we run it.
• Comparing price per token. Compare cost per finished task. The Artificial Analysis rows we read show a pricier model can cost less per task.
• Copying a temperature of 0 onto a reasoning model. The model cards we read recommend 1.0. On the newest Claude models you cannot set it at all.
• Teaching a fixed thinking budget. On Claude 4.7 and later, budget_tokens returns a 400 error. Use adaptive thinking and effort.
• Leaving effort at the default. GLM 5.3 Flash defaults to max. Opus 5.5 defaults to medium. A default is a choice someone else made.
• Changing effort mid-conversation. It breaks the cache on most models.
• Treating OpenRouter rank as quality. It counts tokens. Free variants and agent loops inflate them.
• Calling every downloadable model open source. Licenses differ. Read each one.
• Skipping the check. Routing without a check sends hard cases to a model that cannot handle them.
• Pricing once and forgetting. Every figure here is dated for a reason. Prices and rankings move in weeks.

When not to switch models

Switching has its own cost, in tests, prompts and risk. Stay where you are in these cases.

• The saving is small. If the bill is a few hundred dollars a month, an engineer's week costs more than a year of savings.
• The current model passes and the task rarely changes. A stable, tested setup is worth money.
• You cannot test. Without a test set, a cheaper model is a guess. Build the set first.
• Data rules block the new route. A model that is cheaper on a gateway may be barred for your data.
• Nobody owns the result. A routing layer needs an owner who watches the log.
• You have not proven the use case yet. Prove value on a thin slice first, as our post on proof of concept versus production explains.
• The documents are private. Read how to build a private AI chatbot for company documents before you pick where the model runs.

Glossary

Work with EdgeFirm

EdgeFirm helps teams pick models, set thinking and cost controls, and test the choice before they spend the build budget. Our AI consulting service covers model selection, build or buy advice and cost modeling. If you need a private deployment, start with the self-hosted LLM guide.