The usual setup: connect your data to a chatbot

The common setup is quick. Point an AI assistant at the database, a shared drive or a set of exports, and let people ask questions in plain English. Demos go well, because demo questions are simple and demo data is clean.

Real questions are not simple. "Which customers are behind on payments?" depends on what counts as a customer, which invoices count as open and which system holds the payments. The assistant has to settle all of that before it can answer, and nobody has told it how.

Why it fails: the model fills the gaps by guessing

Three things go wrong in most setups.

  • No shared meaning. Table and column names are abbreviations chosen years ago. The model has to guess which field holds what, and which tables belong together.
  • Sources disagree. The same customer appears in three systems with three spellings. A price list in a spreadsheet is newer than the one in the order system. The model cannot tell which one is the record.
  • Required steps live in instructions. "Leave out cancelled orders" and "use the current price list" are written into the prompt. The model follows them most of the time.

When the model meets a gap, it does not stop and ask. It writes the answer that sounds most likely. A guess and a fact come back in the same confident tone, so the person asking cannot tell them apart.

Suppose, as an example, a flooring distributor asks for sales by customer. In the order system a customer is a ship-to account. In accounting it is a billing account. In the CRM it is a contact. Each choice produces a real-looking total, and only one of them is what the owner meant.

Illustration titled Missing meaning becomes a guess. A panel labeled Straight to AI shows amber answer cards each labeled Guess. A panel labeled With a data model shows one green card labeled Checked answer. A bar at the bottom reads: The AI answers from what the system defines.
Figure 1. In this example, the same question gives different answers depending on which system the AI reads. A shared data model gives it one meaning to answer from.

The model is not the system

The usual response to wrong answers is to buy a bigger model. A stronger model guesses better, but it still guesses, because the missing meaning is not in any model. It is in your business: how you define a customer, which system holds the price, which orders count.

The fix is to put structure around the model:

  • A data model. Customers, products, orders and prices are defined once, with how they relate, so each term has one meaning. What an ontology is, explained for business owners explains this model in plain terms.
  • One source of record for each fact. Where two systems disagree, the model already says which one wins.
  • Required steps in software. Lookups and checks that must always happen run as code every time, instead of as instructions the model may skip. Put business rules in your data model covers where these rules belong.

The model still does the language work: reading the question and writing the answer. The system supplies the meaning and runs the steps. Ask your business data questions in plain English shows what that looks like day to day.

The evidence: the same model, with and without structure

Buffaly published a test of this idea in Make a Smaller Model Behave Like a Frontier Model: Put Buffaly Around It (Matt Furnari, Buffaly blog, August 1, 2026). Seven models answered the same 150 questions from MedQA, a public benchmark built on medical licensing exams. Each model answered every question twice: once directly, and once with Buffaly supplying structured context first.

The structure was one required step. Before the model answered, Buffaly looked up all four answer choices in SNOMED CT, a standard clinical vocabulary, and the model could ask for up to four terms from the question to be defined. The model could not skip the lookup.

The published results:

  • GLM 5.2, an open model, on its own: 101 of 150 correct.
  • The same GLM 5.2 with Buffaly around it: 144 of 150. Same weights, no fine-tuning, no larger model.
  • GPT-5.5, a frontier model, on its own: 146 of 150.

According to the article, GLM corrected 46 answers it had missed and got 3 wrong that it had answered correctly before. The gap to the frontier model went from 45 answers to 2. Not every model gained: in the published table, Gemma3 27B scored 117 both times.

The test used medical exam questions, not business data. What it measured is the effect of one enforced lookup against one defined vocabulary. As the article puts it, the benchmark is medical; the layer is not. For a business, the defined vocabulary is your own data model: your customers, products, prices and orders.

What to fix first: the data model, then the AI

  1. List the questions. Write down the questions your team would ask an AI first, in their own words.
  2. Define the terms. For each question, note the terms it depends on, such as customer, open order or margin, and the definition your team actually uses.
  3. Name the record. For each term, name the system that holds the true value. Where two systems disagree, decide which one wins.
  4. Find the required steps. Note anything that must happen every time: leaving out cancelled orders, using the current price list, converting units. Those belong in software.
  5. Then choose the model. With meaning and steps in place, test a smaller model before paying for the largest one.
Illustration titled Fix the structure first, then the AI. Connected blocks read Define each term, Name the record source and Make key steps required, followed by a block reading Choose the model. A bar at the bottom reads: The model is one part of the system.
Figure 2. Define the meaning, name the source of record and make key steps required before choosing a model.

Questions to ask a vendor

  • When the AI answers a question about our data, where does it get the meaning of each field?
  • If two of our systems disagree, how does the AI know which one is right?
  • Which steps run every time in software, and which depend on the model following its instructions?
  • Can we see how an answer was produced: which records and which definitions it used?
  • When an answer is wrong, what do we fix: the prompt, the model or the data model?

Send us a question your AI got wrong

Send us a question an AI tool answered wrong about your business, or one you would not trust it with yet. We will show you where the meaning was missing and what your data model would need to define for the answer to come back right. Tell us about your current system, or read how full service works.