If You’re so Smart, Why Aren’t You Rich?

This well-known aphorism highlights the discrepancy between intellect and the accumulation of wealth. Not all smart people are rich, and vice versa.

It is, however, a good yardstick to filter out some AI hucksters: People who know the #1 secret to building 7-figure businesses with AI (in a single weekend) would just do that, instead of hawking their video courses online. That much should be obvious. The technology underlying the unfounded promises of massive wealth has changed (it used to be cryptocurrency), but the playbook is the same.

A more subtle display of this discrepancy is the focus of some AI companies on demonstrating their models' proficiency at obscure, elite-level math and coding problems. See here for an example: Inside the Secret Meeting Where Mathematicians Struggled to Outsmart AI | Scientific American

To me, this is a distraction. What's most likely happening here (the article is light on details) is that the AI recognizes specific repeating patterns in the math problems it has already been trained on and reproduces them.

My retort would be: If this AI (OpenAI's o4-mini model) is so smart, why do we read about the great AI disillusionment? Where are the novel breakthroughs in math and science, made purely by AI with no human input? Where are the actual 10x economic benefits?

These models are undoubtedly powerful, but only if they're carefully integrated, with lots of plumbing, engineering, and human ingenuity, into an overall system that produces results.