Do you need the most expensive AI model for Fishbowl questions?
When a company enables team members to query Fishbowl via AI, someone has to pick a model, and the price lists run from a few cents to tens of dollars per million tokens. In our AI model shoot-out, we tracked what every answer cost. For a direct question, the cheapest models that got it right cost less than a cent.
What we measured
Every model was connected to the demo Fishbowl database through ILC MCP and asked which inventory expires in the next 30 days, three times with each of two wordings. We grouped the models by their list price per million input tokens on OpenRouter, the service we used to reach most of them: premium at $4 or more, mid from $1 to $4, and budget under $1.
| Tier | Direct question | “How to find” question | Avg cost per direct answer | Avg time |
|---|---|---|---|---|
| Premium | 15 of 15 | 10 of 15 | $0.38 | 48 s |
| Mid | 16 of 18 | 5 of 21 | $0.07 | 38 s |
| Budget | 21 of 27 | 6 of 27 | $0.02 | 80 s |
Asked directly, price didn’t matter much
Seven budget models found the right stock in all three runs: GPT-6 Luna, DeepSeek V4 Pro, DeepSeek V4.1 Flash, GLM 5.3, Kimi K3, Qwen 3.8 27B and Gemini 3.8 Flash. The only budget models that failed were the two smallest.
GPT-6 Luna was the cheapest right answer in the test, at about $0.002 and 26 seconds. When I graded it by hand, its answer was refreshingly short, and it found the expiration tracking in one try. Claude Fable 5.1, the most expensive model we ran, got the same answer for about $0.38 through OpenRouter.
Where the premium models earned it
The premium models pulled ahead when the question was vague. Asked “how to find less than 30 day expired inventory”, they found the stock in 10 of 15 runs, against 6 of 27 for the budget models. Most models answered that wording with steps for a report. The premium ones more often went on to query the data after finding the report, and Fable did it every time.
So the result depends on three things together: how well the model handles a vague request, how much help its tools give it, and how clear the question is. A premium model makes up for some vagueness. Better tooling helps every model, and ILC MCP gives the AI notes on how Fishbowl stores its data so it spends less effort guessing. And when the question asks for the data directly, a budget model does the job.
Price per token isn’t price per answer
The list price is per token, and the number of tokens an answer uses varies a lot. Every query result the AI reads gets sent back to the model on the next step, so a model that runs more queries pays for more reading.
- Gemini 3.8 Flash is a budget model by list price, but it made about 17 tool calls a run and read about 350,000 input tokens. It cost about $0.10 an answer, nearly 50 times what GPT-6 Luna cost.
- The app you run the model in matters too, and it’s hard to pin down. Fable cost about $0.81 an answer in Claude Code and $0.38 through OpenRouter. Claude Code’s instructions and tool list are built into the app and sent with every step, so its count includes more tokens. On a subscription you don’t pay per token at all, so both figures are estimates from token counts at list prices, not a bill.
- A cheap wrong answer isn’t cheap. Granite 4.2 8B cost under a cent and failed every run. Someone still has to find the right answer. Running a model on your own hardware brings the cost per answer down further, with its own trade-offs.
How to choose
- Try a few budget and mid models on your own questions before you pay for premium. For straightforward lookups, several did as well as the most expensive ones.
- Compare the cost per answer, not the price per token. Run the same five or ten questions through each model and look at the bill.
- Keep a premium model available for harder or vaguer questions, and for long sessions that build on earlier answers. Most AI apps let people switch models in the same chat.
What we didn’t test
Every run in our test was one question in a fresh chat. Real use often looks different. Someone asks which stock expires soon, then which customers bought those lots, then how that compares with last quarter, and the chat keeps growing. Each new answer carries everything before it, including every query result the AI has already read.
That’s where we suspect the larger models pull ahead. They can hold more of a long conversation, and in our experience they keep better track of earlier answers. Smaller and cheaper models may start dropping details or contradicting themselves a few questions in. We haven’t measured that yet, so read the budget results above as results for single questions. Long sessions are next on our list to test.
Results can also change over time. People regularly report a model getting worse some weeks after launch, and blame changes the provider makes or demand for GPU capacity. Some are now measuring it: LiveNerf runs a fixed set of questions through Claude Opus 5.5 every day to check for a drop, and when we wrote this it hadn’t reached a conclusion. Our numbers come from one day.
This was also one question on one database, with prices from OpenRouter on the day of the test, and prices change often. The same connection worked with all 19 models we tried, so you can change models later without changing how the AI reaches Fishbowl.
Related: