Amazon hands ticket labelling to a small model that only chooses
Two Amazon engineers replaced Claude Sonnet 4.6 with a small decision model that, by their own figures, is 44 times cheaper per ticket. Two days later, Google launched a similar service.
What happened
Two Amazon engineers have taken ticket labelling away from Claude Sonnet 4.6 and handed it to a small model that does not write text but only chooses. By their own figures, it costs about 44 times less per ticket and is almost five times as fast.
Carr Onstott and Mehul Jaiswal work on one of Amazon’s largest internal ticketing systems. Every ticket, from a customer escalation to a security finding, is given one of 20 categories. Sonnet did that job, with a long set of instructions sent along with every ticket. On 9 November, on the blog of Strands, an Amazon open-source project, they described how they replaced it with Strands Decider E4B: a decision model built on Gemma 4, Google’s open model. It receives the ticket with the categories as options and returns a probability for each option, plus a score for how confident it is.
The average call went from 4,215 to 897 milliseconds, and the cost from $0.0222 to about $0.0005 per ticket. The switch ran through a three-way setting: Sonnet only; both at once on live traffic, with Sonnet still deciding and the team reviewing every disagreement; and the small model only. Switching back takes 45 seconds. According to the team, the small model did better than Sonnet on a separate set of tickets. The post gives no accuracy figures, ticket volumes or total savings; it speaks of millions of dollars a year.
On 11 November Google launched the Gemini Decisions API, with a model called Gemini Decision 1. It lets you ask up to 1,000 questions about a single input in one go: yes or no, a choice between options, or a score on a scale. Google says an answer takes 150 to 250 milliseconds, at $0.10 per million input tokens and nothing for output. Google Search uses it as a safety layer for MCP tools, the connections through which AI agents use other software. On the tech forum Hacker News, Amazon’s story scored 727 points; sceptics called it a plain classifier or an advert for AWS.
“The part we’d copy regardless of model is the three-way switch.”
Carr Onstott and Mehul Jaiswal of Amazon in their post of 9 November 2026
The background
Choosing is not the same as writing
A language model like Sonnet writes its answer token by token, a token being a fragment of a few characters, and the whole model runs again for each one. To give a single label, Sonnet wrote around 170 tokens of JSON and reasoning. A decision model skips all that. For its first version, Amazon took the open model Qwen3.5-2B and replaced the part that produces text with a small component of just over a million parameters. That component scores all the options in a single pass. The model can literally do nothing but choose.
Classic classifiers, such as the BERT models long used to sort text, are trained for each task on labelled examples. With a decision model you simply describe the options in a sentence or two.
A wave of decision models
The idea gathered pace this autumn. In September TypeSafe, a company founded by former OpenAI employee Diogo Almeida, launched Jev, a model that only answers questions with fixed answers. Imitators soon followed, The Batch noted. At Amazon, engineer Marc Brooker built his own version, which came out on 1 October as Strands Decider 2B.
On 15 October four versions followed on Gemma 4, which Google released as an open model in April. They were trained by distillation: a larger model, Qwen3.5-4B, supplied the example answers, and the small model only learned from them where they matched the correct label.
How the bill adds up
With an AI service you pay per token. Sonnet 4.6 costs $3 per million input tokens and $15 per million output tokens. Each ticket meant 6,554 tokens in and 170 out: 2.22 cents in all. The small model gets 2,009 tokens and returns one.
But it runs on a server of its own, billed by the hour whether tickets come in or not: $1.408 an hour for a machine with one 24 GB Nvidia A10G graphics card, of which the model uses about 19 GiB. At full load it handles some 5,600 tickets an hour, roughly $250 per million tickets. Allowing room for peaks, the team works with $500.
Our analysis
For a fixed choice, a model that writes is an expensive tool
Many organisations use a large language model for small, closed choices: labelling an email, routing a request, checking an invoice against rules. That makes sense, because it works straight away and needs no training data. But at volume you pay for writing nobody reads, and above all for the long instructions you send again every time. Our expectation: the work will split. Large models will write and reason, while small models take the many small decisions, more and more often on hardware the organisation controls.
Do not take the factor of 44 as a yardstick. It sets Sonnet’s list price against a well-used server of your own. Amazon probably pays less internally, and the post does not say whether the fixed instructions came from a cache; at Anthropic, part of a prompt read from the cache costs a tenth of the normal price. Accuracy figures are missing. And the team itself says where the approach does not fit: categories that change often, or tasks that need an explanation.
The most useful part is the switch itself: the old and the new model side by side on real work, every difference reviewed and a way back. Use the confidence score to pass doubtful cases to a colleague or a large model.
What stands out
- The order of the keys mattered The same content with the JSON keys in a different order shifted the probabilities by up to 0.19. The team now sorts the keys in every request.
- The money was in the question Of Sonnet’s 2.22 cents per ticket, almost 2 cents went on input, by our calculation from the post’s figures.
- Millions means tens of millions of tickets Each million tickets saves just over $21,000. Saving one million dollars a year therefore takes around 46 million tickets, by our calculation.
- Google uses it as a gatekeeper Google Search has its decision model check traffic to MCP tools for misuse.
What it means for you
If a large AI model makes a fixed choice for you, you are paying for text nobody reads.
- Find the fixed choices Labelling an email, sending a request to the right team, checking an invoice against fixed rules: any step with a fixed list of answers is a candidate.
- Test side by side on real work Run the old and the new model together for a while, review every difference and make sure you can switch back within a minute.
- A small model can run in-house A model for one task often fits on a single graphics card. Your data stays inside and you do not pay per question.
Where we come in · Development
We develop a model of your own for one task
With AI Model Engineering we choose the best open-weight base for each task and refine it, with distillation among other methods, into a model that does one thing well. We test it on data it has never seen, and it runs on your own hardware, with no cost per prompt. We start with a feasibility check of the task.
Which fixed choice do you still leave to a large model?
Tell us which step it is and how often it comes up. We’ll think it through with you and always reply within one working day.