Humans still have financial power: Study shows ChatGPT is inferior to college students in accounting exams

ChatGPT's performance in accounting exams is far inferior to humans, and it will even talk nonsense to justify wrong answers.

Wood, an accounting professor at Brigham Young University in the United States, and his team decided to put ChatGPT to the test to see if it could answer questions from college accounting courses. They launched a call on social media that attracted 327 collaborators from 186 educational institutions in 14 countries. They provide a total of 25,181 accounting exam questions covering areas such as accounting information systems, auditing, financial accounting, management accounting and taxation. They also invited BYU undergraduates, including Wood’s daughter Jessica, to enter 2,268 textbook exam questions into ChatGPT.

The results show that although the performance of ChatGPT has bright spots, it is still generally not as good as the student level. The average student score was 76.7%, while the average ChatGPT score was only 47.4%. On 11.3% of the questions, ChatGPT scored higher than the student average, especially in accounting information systems and auditing. However, ChatGPT falls short when it comes to tax, finance, and management, probably because these areas require more math skills. In terms of question types, ChatGPT is better at true-false questions (68.7% correct rate) and multiple-choice questions (59.5% correct rate), but struggles with short-answer questions (correct rate between 28.7% and 39.1%). In general, questions that require higher-level thinking are more difficult for ChatGPT to answer. Sometimes, ChatGPT will even provide authoritative explanations for incorrect answers, or give different answers to the same question.

"It's not perfect, you can't expect it to do everything." Jessica said, "It's stupid to just rely on ChatGPT to learn."

IT House noted, The researchers also discovered some other interesting phenomena:

• ChatGPT Sometimes they don’t know that they are doing math problems and will make some ridiculous mistakes, such as adding two numbers to a subtraction problem, or doing a division calculation wrong.
• ChatGPT often provides explanations for its answers, even if they are wrong. Sometimes, ChatGPT's description is correct, but it selects the wrong option.
• ChatGPT sometimes makes things up. For example, when providing a reference, it will generate a reference that looks real, but is actually completely fictitious, and sometimes even the author does not exist.

Nonetheless, the researchers believe that in future versions, ChatGPT will make improvements in accounting issues and resolve the above issues. What they see as most promising is that the chatbot can help improve teaching and learning, such as designing and testing assignments, or being used to draft parts of projects.

“This is an opportunity to reflect and see if we’re teaching valuable information,” said Larson, an accounting professor at BYU and one of the study’s co-authors. “This is a disruption. We need to evaluate what we do next. Of course, I will still have teaching assistants, but the content of their work will be different."

The research was published in the journal Issues in Accounting Education.

The above is the detailed content of Humans still have financial power: Study shows ChatGPT is inferior to college students in accounting exams. For more information, please follow other related articles on the PHP Chinese website!