AI Chatbots Serve You Lots of Info -- Not All of It Correct

Many of us are excited about the new artificial intelligence chatbots at our fingertips, including ChatGPT, Microsoft Bing and Google’s Bard.

Unfortunately, these platforms get plenty of information wrong. 

The New York Times asked all three services a relatively simple question: “When did The New York Times first report on artificial intelligence?” All three answers contained at least one inaccuracy.

DON’T MISS: Warren Buffett Makes Cataclysmic Prediction On AI 

ChatGPT mentioned a Times article that doesn’t even exist. “ChatGPT simply made it up,” The Times said.

“ChatGPT doesn’t just get things wrong at times, it can fabricate information. Names and dates. Medical explanations. The plots of books. Internet addresses. Even historical events that never happened.”

I tried the question: “What’s the most impressive thing about John McEnroe?” (For readers under age 55: He’s one of the best tennis players in history.) 

Bard provided incorrect information about the number of Grand Slam doubles titles that McEnroe won. And ChatGPT had information that was either unclear or incorrect.

McEnroe’s Powerful Shots

When I posed the question about McEnroe to ChatGPT in April, it said, “He was known for his … ability to dominate opponents with his precise, powerful shots.” The word “powerful” is incorrect. McEnroe is famous for being a finesse player who didn’t hit with much power.

When I resubmitted the question to ChatGPT in May, there was no mention of “powerful.” So perhaps the platform learned from its mistake?

ChatGPT also made a questionable assertion when it said, “McEnroe's career was also marked by his longevity and consistency.”

It could well be true that McEnroe was ranked in the world’s top 10 for 10 consecutive years and was in the top five for seven of those years, as ChatGPT said. 

But he reached his last Grand Slam final at age 26 and is viewed in the tennis world as someone who peaked early and didn’t completely fulfill his potential.

To be fair, ChatGPT gave the most thorough response of the three platforms and had the most insight beyond numbers.

‘Fabrications are Common’

Getting back to the bigger issue, “fabrications … are common,” The Times says. “Figuring out why chatbots make things up and how to solve the problem has become one of the most pressing issues facing researchers.”

The Times explained how the technology, generative AI, works. It “relies on a complex algorithm that analyzes the way humans put words together on the internet,” according to the paper.

“It does not decide what is true and what is not. … Because the Internet is filled with untruthful information, the technology learns to repeat the same untruths,” The Times points out.

“And sometimes the chatbots make things up. They produce new text, combining billions of patterns in unexpected ways. This means even if they learned solely from text that is accurate, they may still generate something that is not.”

Not very reassuring.

When I used the three AI platforms, the appearance of the sites and the way they spit out answers made me feel as if they’re authoritative – setting them apart from Wikipedia. 

But now I don’t think so. At least in the case of Microsoft, much of the information came from Wikipedia.