AI Analysis Ability Test: The pass rate of Claude Opus is only 54%
2026-08-12 15:01:28
According to CoinMeta, an artificial intelligence analysis capability test aa-analystagent was recently launched. The test consists of 80 questions, and each question requires the model to attempt it independently 5 times. Only by answering all questions correctly 5 times in a row can one pass. The results show that Claude Opus ranked first with a pass rate of 54%, followed by GPT-5.5 with a pass rate of 50%, and Claude Fable with 49%. Among the models with open weights, Kimi K3 performed the best, achieving a pass rate of 39%. If we consider only the average accuracy of each attempt, GPT-5.5 had the highest average accuracy at 66%. However, when the requirement was to answer the same question correctly 5 times in a row, its pass rate dropped to 50%. The test analysis also pointed out that a common issue with the AI model was a misunderstanding of the questions, which led to deviations in subsequent answers.
Bullish 0
Bearish 0
Source:Internet
This content is for market information only and does not constitute investment advice.