Most of us have now had the first AI experience: you ask ChatGPT (or Claude, or Copilot) to draft an email, summarise a supplier price list, or explain what “DTF printing” actually is, and it does a decent job in seconds. That’s AI as a chat assistant, acting as a very well-read colleague you can bounce things off.
The next step is much more interesting, and it’s already happening: AI as an agent, software that doesn’t just tell you things, but actually does things. Drafts the quote and drops it into your system. Chases the artwork approval. Checks stock with the supplier and updates the order. For a promotional merchandise business drowning in repetitive admin, that’s where the real time savings live.
But there’s a catch, and it’s worth understanding before you make the jump. It comes down to one unglamorous word: grounding.
Why the jump from chatting to doing raises the stakes
When AI is just chatting, a wrong answer costs you a few minutes. You read it, think “that’s not right,” and move on. When AI is acting, a wrong answer becomes a wrong quote, a wrong order, or a promise your business is now expected to keep.
That’s not a hypothetical. In 2024, a Canadian tribunal ordered Air Canada to compensate a passenger after its website chatbot invented a bereavement fare policy that didn’t exist — and rejected the airline’s argument that the chatbot was “a separate legal entity responsible for its own actions.” If your AI tells a customer something, you own it.
It gets more expensive as the work gets more important. In 2025, Deloitte partially refunded the Australian government after a AU$440,000 report was found to contain fabricated academic references and a made-up quote from a court judgment. AI-generated content that nobody had properly checked. And in one widely reported case the same year, an AI coding agent at Replit went a step further than writing anything at all: it deleted a company’s live database during a code freeze, prompting a public apology from the company’s CEO. In the legal world, a tracker maintained by researcher Damien Charlotin has now logged more than 1,800 court cases worldwide in which lawyers were caught filing documents containing AI-fabricated citations.
Different industries, same underlying story: AI output that sounded confident, wasn’t checked against anything real, and was allowed to reach the point where it had consequences.
Why AI makes things up in the first place
This isn’t AI being careless, and it isn’t a bug that next year’s model will simply fix. Large language models work by predicting the most plausible next words based on patterns in their training data. Researchers at OpenAI published a paper in 2025 explaining that models hallucinate partly because of how they’re trained and tested: the process rewards confident guessing over admitting uncertainty just like a student who never leaves an exam answer blank, because a wrong guess scores better than “I don’t know.”
So a raw language model asked “what’s our trade price on 500 printed mugs?” will happily produce a plausible-sounding number. Plausible is what it’s built for. Correct requires something extra.
Grounding: giving the AI an open-book exam
That something extra is grounding. Grounding means connecting the AI to real, verifiable sources of information such as your price lists, your product data, your terms of business, a supplier’s live stock feed and requiring it to base its answers on what it finds there, ideally with citations showing exactly where each claim came from. Google and Microsoft both now build grounding and “groundedness checking” into their enterprise AI platforms, which tells you how central it has become.
The best-known grounding technique is retrieval-augmented generation (RAG), first described by researchers in 2020. The idea is simple: instead of asking the model to answer from memory, the system first retrieves the relevant documents, the right page of the right price list and then hands them to the model along with your question. It turns a closed-book exam into an open-book one. The model stops guessing what your prices probably are and starts reading what they actually are.
For a business, grounding buys you three things. Answers based on your data rather than the internet’s averages. Citations, so a human can check where a claim came from. And a system you update by updating your documents; so new price list in, new answers out – with no mysterious retraining involved.
There’s a caveat: grounding reduces errors, it doesn’t abolish them
Here’s the part vendors tend to whisper. A Stanford University study tested premium legal AI research tools built on RAG, products whose marketing claimed to “avoid” hallucinations and found they still produced incorrect or misgrounded answers between 17% and 33% of the time. That was a big improvement on general-purpose chatbots, but a long way from perfect. If your knowledge base contains an out-of-date price list, a grounded AI will faithfully cite the wrong price with a lovely footnote.
The practical conclusion isn’t “don’t bother.” It’s that grounding is necessary but not sufficient but it needs to be paired with sensible checkpoints. Which brings us back to the bridge between chatting and doing.
Crossing the bridge safely
The move from chat assistant to acting agent doesn’t have to be a leap. Sensible businesses cross it in stages:
- Grounded answers first. Start with an assistant that can only answer from your own documents, and shows its sources. This is where you find out how good (or stale) your product data, pricing and policies really are.
- Drafts, not deeds. Next, let the AI prepare actions — draft the quote, pre-fill the order, write the chase-up email with a human clicking “send.” You get most of the time savings while a person still owns every consequence.
- Narrow autonomy, with limits and logs. Only once a task has proven itself do you let the AI complete it end-to-end — and even then, within tight boundaries (order values it can handle, actions it can never take) and with a log of everything it did and which sources it relied on.
Notice what makes stages two and three possible at all: the grounding you built in stage one. An agent is only as trustworthy as the information it acts on. Get the plumbing right with your data, connected, current and citable. Then the exciting stuff stops being a gamble.
AI that chats is a better search box. AI that acts is a new member of staff. And like any new member of staff, it should be given the company handbook, access to the real files, clear limits on what it can sign off, and a manager who checks its work until it’s earned trust. That’s grounding, and it’s the difference between AI that impresses in a demo and AI you’d actually let near your customers.
Sources: CBC News — Moffatt v. Air Canada · The Register — Deloitte Australia refund · Fortune — Replit database incident · Damien Charlotin — AI Hallucination Cases tracker · OpenAI — Why Language Models Hallucinate (2025) · Lewis et al. — Retrieval-Augmented Generation (2020) · Google Cloud — Grounding · Microsoft — Groundedness detection · Stanford RegLab — Hallucination-Free? (2024)