AI Insight
Research experiments indicate that people systematically overestimate the degree of confidence expressed by AI conversational systems such as ChatGPT and Gemini when they provide responses to user queries. Users tend to perceive AI-generated answers as more certain and authoritative than the systems' actual output warrants. This miscalibration between perceived and actual AI confidence may contribute to uncritical acceptance of information that could be inaccurate or unreliable.
Why it matters
This finding has significant implications for how AI tools are deployed in everyday decision-making contexts, as overestimated confidence may lead users to act on incorrect information without appropriate skepticism. It also raises urgent questions for developers and policymakers about how AI systems should communicate uncertainty more transparently.
Understand the Science
Artificial intelligence (AI) systems, particularly conversational agents such as ChatGPT or Gemini, are now used daily by a growing number of people worldwide. While many users trust the answers of AI agents to their queries, these are not always accurate and reliable.
Source: People overestimate how confident AI systems are in their responses, experiments reveal