ChatGPT's efficiency is now questioned by experts after discovering that the AI chatbot incorrectly answered 52% of software engineering questions.
ChatGPT 52% Inaccurate When Answering Software Engineering Questions
The AI chatbot quickly gained popularity when OpenAI released ChatGPT to the public.
But, researchers from Purdue University in the U.S. said that although the ChatGPT is popular, there's a lack of thorough investigation into its quality and usability when it comes to answering software engineering questions.
Because of this, they decided to conduct a new study to check the answers of ChatGPT to 517 questions from Stack Overflow (SO).
"Our examination revealed that 52% of ChatGPT's answers contain inaccuracies and 77% are verbose," involved researchers said via News 18.
But, what alarms them the most is that ChatGPT answered 54% of the questions incorrectly because it didn't understand the concept of the queries.
Purdue University researchers said that although there are some instances when ChatGPT understands the questions, the AI model failed to show proper solutions.
These issues in the AI chatbot led to many conceptual errors.
Read Also: Microsoft Called Out for 'Negligent' Cybersecurity Policies; US Senator Seeking Probe
Other Issues in ChatGPT
Researchers discovered other issues in ChatGPT. One of them is the limited ability of the AI chatbot to provide reasoning.
They explained that ChatGPT provided solutions, codes, and formulas in many cases. However, artificial intelligence didn't think about the outcome.
Aside from this, the new analysis showed that ChatGPT also suffers from other quality problems. These include inconsistency, verbosity, etc.
ZDNet reported that this could cause major issues among issues.
Purdue University experts explained that many people tend to overlook incorrect information provided by ChatGPT because these responses are well-articulated.
Researchers explained that since ChatGPT's answers are well-articulated, comprehensive, and provide humanoid insights, users cannot identify incorrect information in the AI's responses.
They claimed that this happens more than 39% of the time. These common efficiency issues in AI-powered chatbots are a significant problem because they can contribute to the spread of misinformation.
Because of this, experts warned that people should reconsider using OpenAI's generative artificial intelligence when it comes to complicated queries.
To learn more about the new study showing the efficiency issue in ChatGPT, you can click here.
