Public urged to collaborate on creating the ‘ultimate test’ to detect AI reaching peak intelligence | Latest Science & Technology Updates
Scientists are developing a test known as “humanity’s last exam” to assess the intelligence level of AI and determine when it reaches expert-level intelligence.
The Center for AI Safety (CAIS) and Scale AI are asking individuals to submit questions to create “the world’s most challenging artificial intelligence test.”
The creators of the quiz stated, “Existing tests have become too simplistic to accurately monitor AI progress or determine how close it is to expert-level.”
A few years ago, AI was providing almost random responses to questions on exams, but that is no longer the case.
Last week, Dan Hendrycks, the executive director of CAIS, mentioned that OpenAI’s latest model, OpenAI o1, “outperformed the most popular reasoning benchmarks.”
Despite this progress, AI still struggles with answering complex research and intellectual questions.
According to Stanford University’s AI Index Report from April, AI also performs poorly on tests involving planning and visual pattern-recognition puzzles.
Therefore, “humanity’s last exam” will focus on abstract reasoning to gauge AI’s intelligence level.
The questions submitted should not be typical quiz questions.
The quiz creators mentioned, “Questions written by undergraduates tend to be too simple for the models.”
More updates from Sky News:
Russian state media banned from Facebook and Instagram
Three mpox scenarios the UK is preparing for
Teen Instagram users to get strict privacy settings in update
They recommend that question writers have at least five years of experience in a technical industry job like SpaceX or are a PhD student or higher.
The questions should challenge non-experts and not be easily searchable online, avoiding trick questions.
The quiz creators explained, “If a randomly chosen undergraduate can understand the question, it is likely too straightforward for the advanced LLMs of today and tomorrow.”
Stay updated with the latest news from the UK and worldwide by following Sky News
Individuals who submit successful questions will be listed as co-authors on the paper and have the opportunity to win from a prize pool of $500,000 (£378,400), with the best question writers earning $5,000 (£3,780) each.
Questions must be submitted by November 1st.
