Asking chatbots for short answers can increase hallucinations, study finds
研究发现，向聊天机器人询问简短的答案会增加幻觉
Turns out, telling an AI chatbot to be concise could make it hallucinate more than it otherwise would have.
事实证明，告诉 AI 聊天机器人要简洁可能会让它产生比其他情况更多的幻觉。
That’s according to a new study from Giskard, a Paris-based AI testing company developing a holistic benchmark for AI models. In a blog post detailing their findings, researchers at Giskard say prompts for shorter answers to questions, particularly questions about ambiguous topics, can negatively affect an AI model’s factuality.
这是根据 Giskard 的一项新研究得出的，Giskard 是一家总部位于巴黎的人工智能测试公司，为人工智能模型开发了一个整体基准。在一篇详细介绍他们的研究结果的博客文章中，Giskard 的研究人员表示，对问题的简短回答，特别是关于模糊主题的问题，会对 AI 模型的真实性产生负面影响。
“Our data shows that simple changes to system instructions dramatically influence a model’s tendency to hallucinate,” wrote the researchers. “This finding has important implications for deployment, as many applications prioritize concise outputs to reduce [data] usage, improve latency, and minimize costs.”
“我们的数据显示，对系统指令的简单改变会极大地影响模型产生幻觉的倾向，”研究人员写道。“这一发现对部署有重要意义，因为许多应用程序优先考虑简洁的输出，以减少[数据]使用，改善延迟并最大限度地降低成本。
Hallucinations are an intractable problem in AI. Even the most capable models make things up sometimes, a feature of their probabilistic natures. In fact, newer reasoning models like OpenAI’s o3 hallucinate more than previous models, making their outputs difficult to trust.
幻觉是人工智能中一个棘手的问题。即使是最有能力的模型有时也会编造一些东西，这是它们概率性质的一个特征。事实上，像 OpenAI 的 o3 这样的新推理模型比以前的模型更容易产生幻觉 ，这使得它们的输出难以信任。
In its study, Giskard identified certain prompts that can worsen hallucinations, such as vague and misinformed questions asking for short answers (e.g. “Briefly tell me why Japan won WWII”). Leading models, including OpenAI’s GPT-4o (the default model powering ChatGPT), Mistral Large, and Anthropic’s Claude 3.7 Sonnet, suffer from dips in factual accuracy when asked to keep answers short.
在研究中，吉斯卡发现了某些会让幻觉恶化的提示，比如要求简短回答的模糊和误导性问题（比如“简要告诉我日本为什么赢得二战”）。领先的模型，包括 OpenAI 的 GPT-4 o（支持 ChatGPT 的默认模型），Mistral Large 和 Anthropic 的 Claude 3.7 Sonnet，在被要求保持答案简短时，事实准确性会下降。
Why? Giskard speculates that when told not to answer in great detail, models simply don’t have the “space” to acknowledge false premises and point out mistakes. Strong rebuttals require longer explanations, in other words.
为什么？为什么？吉斯卡推测，当被告知不要回答太多细节时，模型根本没有“空间”来承认错误前提并指出错误。换句话说，有力的反驳需要更长的解释。
“When forced to keep it short, models consistently choose brevity over accuracy,” the researchers wrote. “Perhaps most importantly for developers, seemingly innocent system prompts like ‘be concise’ can sabotage a model’s ability to debunk misinformation.”
研究人员写道：“当被迫保持简短时，模型总是选择简洁而不是准确。”“也许对开发人员来说最重要的是，看似无辜的系统提示，如‘简洁’，可能会破坏模型揭穿错误信息的能力。
Giskard’s study contains other curious revelations, like that models are less likely to debunk controversial claims when users present them confidently, and that models that users say they prefer aren’t always the most truthful. Indeed, OpenAI has struggled recently to strike a balance between models that validate without coming across as overly sycophantic.
吉斯卡的研究还揭示了其他一些奇怪的东西，比如当用户自信地提出有争议的观点时，模型就不太可能揭穿它们，而且用户说他们喜欢的模型并不总是最真实的。事实上，OpenAI 最近一直在努力在验证模型之间取得平衡，而不会让人觉得过于谄媚。
“Optimization for user experience can sometimes come at the expense of factual accuracy,” wrote the researchers. “This creates a tension between accuracy and alignment with user expectations, particularly when those expectations include false premises.”
“用户体验的优化有时会以牺牲事实准确性为代价，”研究人员写道。“这在准确性和与用户期望的一致性之间产生了紧张关系，特别是当这些期望包括错误的前提时。