AI financial advice failed 57% of Saturn’s test, underscoring the value of expert verification
Mainstream chatbots faltered on complex money questions in a fintech-commissioned benchmark spanning more than 10,000 responses.
A company-commissioned Saturn test put 18 widely used AI models through 121 financial-advice questions five times apiece and marked 57% of more than 10,000 answers incorrect. Reported errors rose to 88% on harder questions and included fabricated rules and tax guidance that the study said could have triggered a £17,500 charge. The benchmark used Saturn’s own methodology and scoring, and the fintech has a commercial interest in tighter oversight of AI advice. For specialist publishers, the findings strengthen the value of attributable, expert-checked information as conversational interfaces compete for users’ trust.