Why does AI make up Fishbowl table names?

The same question asked with and without access to the Fishbowl database. The first query is an illustration built from table names the models invented in our test.

If you’ve asked ChatGPT or Claude for a Fishbowl query, you may have gotten SQL that failed with “table doesn’t exist”. We asked 19 AI models the same Fishbowl question 306 times to see how often that happens. Answering from memory, 3% of their answers worked.

The test

The question was one a warehouse manager might ask: which inventory expires in the next 30 days? Every model got it three times with no tools at all, answering from what it already knew. It then got it three more times connected to the demo database through ILC MCP, a read-only connection with guardrails. We scored every answer against the real stock in the database. The full method and results are in the deep-dive.

Without access

Without database access: 99 answers gave general steps, 36 wrote SQL that wouldn't run, 7 ended in errors, 6 returned the wrong rows, 5 were correct.

Of 153 answers from memory, 5 worked. Most of the rest described Fishbowl’s screens and reports without any SQL. The 36 that wrote SQL that wouldn’t run used tables and columns Fishbowl doesn’t have: partlot, lot, tagtracking, inventory, ti.tagId, pt.trackingTypeId.

The invented queries looked reasonable. They had sensible joins and date math, and the names were what you’d guess a lot table would be called. You only find out they’re wrong when you run them.

Some answers didn’t get that far. DeepSeek V4 Pro, with no tools available, answered “Let me look up the relevant information for you.” That was the whole answer.

All five that worked came from Claude models, Fable 5.1 and Opus 5.5, and even those didn’t repeat reliably. Fable got it three times out of three in Claude Code and zero out of three through OpenRouter. When we reviewed its best query by hand, it was close, but it found the expiration tracking type by its name. That works on our demo database and wouldn’t hold up on a real one.

Why it happens

Fishbowl keeps expiration dates in a table called trackingdate, one row per inventory tag, linked to the part tracking set up as Expiration Date. Nothing about that name says “expiration” or “lot”. The schema isn’t published in any one place, so a model trained on public text has very little to go on. It writes the query that a generic inventory schema would need, with a lot table and an expiration column.

Fishbowl could make this easier by documenting its schema. Even then, a model only knows what’s in its training data, and new material takes months to get there. Until it does, the model would have to search the web, find a reliable description of the schema, and trust it.

It also has to understand what you mean. When you ask about an “Expiration Date”, you mean the inventory expiration tracking on each tag. A database can also have a custom field someone named “Expiration Date”, and a model that searches for the words can land on that instead.

With access

Connected through the MCP, the models could look at the actual schema before writing anything. Asked “What inventory in Fishbowl expires in the next 30 days?”, 79% of runs came back with the right answer: part FG2200, three tags of 480, expiring in 17 days. They found trackingdate, ran a query, and reported the stock, along with the query they used.

Being able to check made the difference. A connected model that guesses partlot gets an error, looks at the table list, and tries again. It can also read ILC’s notes on how Fishbowl stores its data and search Fishbowl’s current help before it writes the query.

The MCP also keeps a shared agent memory for your company. A power user can save the right way to find this information once, for example where expiration dates live and which tracking type to use. Every later question starts from that note, so the AI needs fewer tool calls and turns to reach an accurate answer.

The wording of the question mattered too. Asked “how to find” instead of “what expires”, many connected models answered with steps for a report instead of the stock. We cover that in the next post.

What to do

If you’re using a general AI chat for Fishbowl SQL, run the query on a test database before you trust it, and expect to fix table names. If your team wants answers from Fishbowl regularly, give the AI read access to the database so it can check the schema and the data before it answers.

ILC MCP for Fishbowl is that connection: read-only, signed in with each person’s own Fishbowl login, with every query logged.