GPT-4 was given 4,550 novel questions representing...
# 06-technical-discussion
GPT-4 was given 4,550 novel questions representing the entire “MIT Mathematics and EECS undergraduate curriculum, including problem sets, midterms, and final exams” and with a well chosen prompting strategy achieved a correct answer rate of 100% The authors explore a range of prompting strategies in the paper. With their final prompting strategy, GPT-4 scored 100% across all homework assignments and tests in the MIT undergraduate computer science and mathematics curriculum. I recommend reading the paper yourself, but tl;dr the winning prompt strategy was: GPT-4 + Few Shot + Chain of Thought + Self Critique + Use the LLM to Identify a specific Named Expert to use as a Persona to reference as part of the Prompting (The Expert prompting component is a new addition to the prompting strategy literature added by this paper which bumped the success rate on questions from 97% to 100%)