Mathematics, like many other scientific endeavors, is increasingly using artificial intelligence. Of course, math is the backbone of AI, but mathematicians are also turning to these tools for tasks like literature searches and checking manuscripts for errors. But how well can AI perform when it comes to solving genuine, high-level research problems?
To date, there is still no widely accepted realistic methodology for assessing AI’s capabilities to solve math at this level. So a group of mathematicians decided to put the machines to the test as they detail in a study available on the arXiv preprint server.
Previous attempts at testing AI have used math contest problems and questions already found in textbooks. What makes this study different is that the questions the programs faced were drawn from mathematicians’ own research. They had never been posted or published online, which means AI couldn’t memorize answers from its training data.








