AI megathread

[Laird]

Shifting the discussion from the 3 Substances in 3 Environments thread into this more appropriate one:

I recall learning a lot of the tricks in my Computational Linguistics class. Once you get a sense of how the magic trick works it just becomes less impressive?

Or rather I appreciate the effort that goes into the trick, while not believing the magic?

If you don't understand the basics of how LLMs are programmed, then it will appear unfathomable.

You need to think of LLMs as effectively a very fancy next-word predictor, because that is effectively what they boil down to, despite the apologism from those lost in the hype. Yes, the algorithms can be very fancy, but it's still nothing more than that ~ a fancy, dumb algorithm operating on inputs and going through a database.

Don't let the metaphors confuse you.

I don't expect that I as a layman have any particularly more impoverished understanding of LLMs than you guys as laymen too, but I still think that their performance on tasks that for humans would require deep understanding is very unexpected.

Consider in particular that, as I posted above, OpenAI's new, yet-to-be-released o3 model achieved 25% on the FrontierMath problem set, which is a set of problems apparently very hard even for Fields Medalists to solve, with some requiring hours and even days.

Think about that. The best mathematicians in the world have to deeply understand and think hard about how to solve these problems. This is not the sort of "trick" where an AI is paraphrasing and regurgitating insights that it's hoovered up from the web. Nor is it like Mathematica or MatLab where the algorithmic solutions to problems are preprogrammed by humans. This is an AI that has learnt without task-specific training to solve novel, unseen problems in what seems to me to be a paradigmatic case that we would expect to require deep understanding and real thought from the best and brightest humans - yet here is a machine, incapable of true understanding and thought, coming up with correct solutions anyhow.

Still don't find that hard to fathom? If so, I still find that hard to fathom. The best I can make of it is that AI of this capacity challenges some aspects of the mainstream views we hold on this forum, and it's easier to summarily dismiss it than to reflect on the nature of the challenge and on how to respond more thoughtfully.