New Approach to Detecting Hallucinations in AI

TapTechNews Jun 24th news, In recent years, artificial intelligence has been booming, and applications such as chatbots have gradually become popular. People can obtain information from these chatbots (such as ChatGPT) through simple instructions. However, these chatbots are still prone to the "hallucination" problem, that is, providing wrong answers, and sometimes even dangerous information.

One of the reasons for "hallucination" is inaccurate training data, insufficient generalization ability, and side effects in the data collection process. However, researchers at the University of Oxford have taken a different approach and detailed in the latest issue of Nature magazine a newly developed method for detecting the "fabrication" (that is, arbitrarily generated incorrect information) problem of large language models (LLMs).

LLM generates answers by looking for specific patterns in the training data. But this method doesn't always work. Just as humans can see animals in the shape of clouds, AI robots may also find patterns that don't exist. However, humans know that the clouds are just shapes, and there are no floating giant elephants in the sky. LLM may take this as real and "fabricate" non-existent new technologies and other false information.

Researchers at the University of Oxford use the concept of semantic entropy to judge whether LLM has a "hallucination" through probability. Semantic entropy refers to the situation where the same word has multiple meanings. For example, "desert" can refer to a desert or abandon someone. When LLM uses such words, it may be confused about the meaning expressed. By detecting semantic entropy, researchers aim to judge whether there is a possibility of "hallucination" in the output of LLM.

The advantage of using semantic entropy is that it can quickly detect the "hallucination" problem of LLM without additional supervision or reinforcement learning. Since this method does not depend on data for a specific task, it can be applied even if LLM is faced with a new task that has never been encountered before. This will greatly enhance the trust of users in LLM, even if it is the first time for AI to encounter a certain problem or instruction.

The research team said: "Our method can help users understand when they need to be cautious about the output of LLM and opens up new horizons for LLM applications that were previously limited due to unreliability."

If semantic entropy is proven to be an effective "hallucination" detection method, then we can use such tools to double-check the output of artificial intelligence and make it a more reliable partner. However, TapTechNews needs to remind that just as humans are not impeccable, even with the most advanced error detection tools, LLM may still make mistakes. Therefore, it is still advisable to always carefully check the answers provided by chatbots such as ChatGPT.