AI Sounds So Sure of Itself. How Do I Know When It's Wrong?

Almost every business owner I talk to tells me the same story. They tried ChatGPT, and the answer was tidy, confident and better written than they would have managed themselves. Then comes the question: "Fine, but how do I know when I can trust it?"

The short answer is an uncomfortable one. You can't tell from the tone. AI sounds exactly as sure when it knows as when it is guessing.

The new hire who never says "I don't know"

Picture a new employee. Fast, polite, a great writer, answers any question in seconds. There is just one habit: they never say "I don't know." When they don't know, they answer the same way, with the same confidence, and fill the gap with something that sounds plausible.

That employee is useful. But no sensible manager would send their work to a client without reading it first.

This is not just my impression. OpenAI explains that models are trained and scored in a way that rewards an answer over an honest "I don't know." It works like a multiple-choice exam: guess and you might get lucky, leave it blank and you get zero. So the model guesses.

What we see in everyday work

We work with AI assistants every day and catch them in mistakes like these regularly. These are not special studies, just ordinary working days. Here are only some of the cases from the last two weeks:

• The links. We asked for job postings and got a neat table of 27 links, complete with descriptions and rates. 16 of them were dead or invented. Of the 10 links to Upwork (the freelance job marketplace), 9 led to "Job not found." On top of that, it reported open postings as closed.
• The law. Asked about the ® symbol, the assistant answered from memory that using it without a registered trademark is punishable. When we opened the actual law, the "punishable" part did not hold up.
• The upgrade advice. Twice the assistant told us to pick "the latest stable version" of a piece of software without checking the list. The top version on that list turned out to be a test release. The real target was a different one.
• The misleading name. Going by an automation's name, the assistant claimed a website form was sending the wrong report and a different client reference number. Neither was true. I had tested every form by hand, end to end.
• Yesterday's note. A working note written by AI in an earlier session pointed to a website page that did not exist. The same note said a workflow had been deleted when it was still there.
• The quote that wasn't a quote. While translating 105 of our notes, we found six with text in quotation marks that looked word for word but was a paraphrase of a paraphrase.
• Today's arithmetic. While preparing the topics for this very series, the assistant put two numbers from two different counts into one sentence. It sounded convincing and implied something we had not done.

None of these mistakes came with a warning. Every one of them sounded just as confident as the correct answers.

The lesson is not that AI is bad. The lesson is that even when you work with it every day, you still catch it making mistakes. The question is not whether it will be wrong, but whether anyone will notice.

Four things you always check

You don't need to check every word. You need to know where AI most often fills in the gaps.

There is also a simple trick that works with any AI assistant: tell it explicitly that "I don't know" is an acceptable answer, and ask for a source for every claim. Anthropic recommends exactly this to reduce made-up answers. It doesn't stop them entirely, but it makes them much easier to catch.

How not to make the same mistake twice

This is the difference that rarely gets discussed. When an employee gets something wrong, they learn and are more careful next time. When AI gets something wrong, it will make the same mistake tomorrow, because it does not remember the previous conversation.

So the memory has to belong to the company, not to the AI.

Here is how it works for us. Every AI mistake we catch is written down as a rule in our company memory, the place every AI assistant starts its work from. After the links incident, the rule reads: every link is opened before it is used. Since then every assistant follows it without anyone having to remind it. Every external link in this article was opened and checked under that rule.

An organised memory is not enough on its own. You also have to prove that the right note reaches the work in time. How we measure that, and where it still misses, is in "Our Company Brain Looked Good in Tests".

That is the difference between using AI and having a system around it. The first gives you speed. The second gives you speed that you don't pay for with mistakes in front of clients.

What to do tomorrow

1. One rule for the team: links, numbers, laws and quotes from AI don't leave the company unchecked.
2. One place for mistakes: a simple shared document where anyone notes where AI got it wrong and how it was caught. After a month you will see the mistakes that keep coming back.
3. One owner: someone responsible for making sure the rule is followed. Without an owner, a rule is just a wish.

Teaching teams to recognise when AI is wrong is the core of the AI Ready Standard (lecture in Bulgarian). For more examples of AI getting it wrong, see "When ChatGPT, Claude and Gemini All Got It Wrong" and "The 7 AI Landmines".

And if you want to see where AI is already working unchecked in your company, that is where my analysis starts. Start here.