AI megathread

@Valmar, firstly, what is your personal experience with LLMs? What, if anything, have you used them for? When did you last use them?

I don't have to use one myself to look at how other people respond to them on both sides of the spectrum. I am very wary of hype and blind faith in anything.

Secondly, did you read the article I shared? If not, read it in full.

Yes, I've read it, and it just reads like any other piece by a true believer. It doesn't inspire me. It just tells that there is blind faith in something not understood. They think they understand, but their perspective feels so far removed from the reality of what LLMs actually are.

Anybody who has meaningful personal experience with LLMs, or who listens to those who do, and therefore has an at least basic understanding of how powerful this tech already is and how rapidly it's improving, will be either astonished, terrified, or a mixture of both.

LLMs have hit a massive wall, and there is nothing but marketing keeping it afloat. "Danger" this, "amazing" that, it feels very overdone, overly emotional.

The fact that you (and @Sci) are instead poo-pooing and minimising it is proof that you are simply badly uninformed (and, judging by the video that you shared, misinformed).

Or rather, I have a strong preference of not letting emotion get the best of me. I am extremely cautious of powerful emotions ~ I know all too well how dangerous powerful emotions can be, as I have been through a cyclone of emotional purging lately. Not just that, but in the past, I have been very prone to mania and anxiety as the other polar extreme of depression and nihilistic emptiness. Not "bipolar" but rather my mind being too successful in clamping down feeling anything, and then those feelings exploding through on occasion. A seesaw that I have learned to be very careful with.

I do not trust blind faith. It is not that I am "minimizing" ~ I have a narrow-eyed caution when it comes to massive marketing claims by companies where billions upon billions of dollars are poured into a technology where the world's RAM, SSDs, hard drives and more have gone through the roof in pricing, companies like Nvidia gloating that they control the market, and more. It reeks of one massive bubble where a handful of individuals are getting rich at the expense of everyone else.

LLMs cost far too much, and are far too unreliable.

I can help you to inform yourself, but I'm not going to pretend that you already are.

And how precisely do you know that I am "uninformed"?

As for dialogue: I responded to each of the video's claims. What more do you want?

You gave your response, but it doesn't feel like you really addressed the video properly.

I'm not bothering to point out how the arguments made in the video fail against this fact (and related ones), but I can share those thoughts if somebody particularly wants me to.

Please, go ahead.

As for its first two claims, "LLMs don't Ponder, they Process" and "LLMs don't Reason, they Rationalize", the video largely indulges in cherry-picking to make its case, pointing out some peculiar failure modes, while ignoring the much more frequent successes, some of them rather spectacular. While, yes, LLM intelligence is to some extent "jagged", for the most part, you're not going to experience those failure modes unless you go looking for them, and instead will mostly experience impressive cognitive power, or at least an impressive simulation of it, to the extent that the distinction is even meaningful.

The examples aren't "cherry-picked" nor are they "failure modes" ~ they are examples of how the LLM is just doing so much pattern-matching and even cheating by looking for the answers in the prompt. The "successes" aren't even such ~ "failure" and "success" make no difference to an algorithm. The point of the examples is to expose that LLMs indeed do not "reason" ~ because they just fall over if you give different sets of otherwise identical logic problems, hinting that there is no reasoning going on, just a convincing simulation of such.

That foreshadows this: the distinction in the first claim between "pondering" and "processing" is, in the operational terms according to which I defined intelligence a few posts back, pretty meaningless and even academic in the pejorative sense.

Such a definition of "intelligence" is something I disagree with ~ it is the reduction of a poorly-understood mental power that is something only conscious beings have to something that can be turned into just a problem of computation, which is not "intelligence" at all in the original sense. It feels like when you conflate two entirely distinct things with the same word, you end up unwittingly confusing the two as identical or similar or overlapping.

I am not blaming you for the redefinition ~ I am blaming malicious "AI" snake-oil salesmen who want to make money by convincing investors that machines can be "intelligent" so they will pour money into it. And that is precisely what has appeared to happen. People have been fooled en masse, and it worries me immensely. I do not believe that there is anything innately "intelligent" about an algorithm ~ no algorithm can be innately intelligent, as intelligence as a creative mental power is not algorithmic in nature.

Re its second claim: that LLMs might sometimes rationalise rather than reason is no more remarkable than that many (most? all?) humans do on occasion (too). If, though, that was all that they did, then they would have no hope of achieving the mathematical and software development feats of which they are quite clearly capable. Again, this is cherry-picking.

Humans genuinely rationalize based on actual beliefs ~ that is, we filter new information through existing belief systems, rather than trying to understand the new information in itself.

In the video, he does note that LLMs do not actually "rationalize" ~ it's just a metaphor for the fact that LLMs will just make stuff up to sound convincing, because that is what the algorithm has been designed to do. Same with "hallucinating".

Curiously, before moving on to make its biased case, the video briefly touches, at 4:18, on the crucial point that I've also been making in this thread, where the lecturer says: "My surprise as a researcher has been the number of tasks that can be modelled as just 'predict-the-next-word'."

This is in regard to the mimicking of language patterns. The whole purpose of LLM algorithms is to predict a chain of plausible words based on prior training data and input context. But there's no intent behind the chain ~ no concepts, no thoughts, ideas, beliefs, anything. Just pattern-matched tokens. It's all very soulless ~ but convincing to the unwary user.

[Laird]

And how precisely do you know that I am "uninformed"?

I told you: by the way you minimise that which an informed person responds to with either astonishment, terror, or a mixture of both.

Additionally, by this:

I don't have to use one myself

That figures. If you'd actually experimented and engaged with current LLMs, testing them with challenging tasks, you wouldn't be poo-pooing them like you are. What they can do is truly remarkable, and reducing it to mere "pattern-matching" is laughably ignorant.

[Laird]

Please, go ahead.

His main argument was based on model collapse: training an AI on its own output, then doing the same to the newly-trained model, etc, leads progressively to poorer performance.

My response: so what? That doesn't prove that the original models can't generate new information. To the contrary, their ability to produce novel software applications, and generate novel proofs/solutions to mathematical theorems/problems (etc etc) demonstrates that they very much can.

In any case, additionally, I'd want to know how they're prompting this output on which they're training descendant models. How are they ensuring that they're producing an adequate recapitulation of the information base on which the base model was trained?

His claim based on model collapse was that therefore, the information quality produced by an LLM is inferior to that produced by a human, but (1) this doesn't follow, and (2) it wouldn't anyway prove his base claim (that LLMs can't produce endless information) even if it was true: poorer quality new information is still new information so long as it's of sufficient quality, which it very much is.

He then asked whether information quantity can make up for information quality. Again, this assumes in the first place that the new information being produced by the base model is of poor quality, which it demonstrably isn't.

In any case, he expressed a principle of conservation of information for LLMs (my editing brackets): "The amount of information we save by using the good [token probability] distribution is offset by the amount of information we need to find a good distribution in the first place."

Again: so what? These are generative models. They're not simply recapitulating the information on which they're trained; they're generating novel mathematical/software solutions, novel texts, etc.

Finally, he references the Turing Machine of Babel: outputting correct information is easy; the difficulty is getting a good correct:incorrect ratio. This again is irrelevant, because anyone who's used a current LLM meaningfully knows that this ratio is already absurdly high.

I told you: by the way you minimise that which an informed person responds to with either astonishment, terror, or a mixture of both.

You know what? "AI" terrifies me ~ because it is destroying peoples' mental health in alarming ways. The real danger is the psychosis it inflicts due to the sycophancy designed into the algorithm.

That figures. If you'd actually experimented and engaged with current LLMs, testing them with challenging tasks, you wouldn't be poo-pooing them like you are. What they can do is truly remarkable, and reducing it to mere "pattern-matching" is laughably ignorant.

I am not being "reductionist". I am being realistic ~ that is precisely what the algorithms do, so how else am I to interpret it? I refuse to project anything onto them. I can easily learn from the experience of others by examining how they use LLMs, how they report on it.

And frankly, an LLM would be dangerous for me, knowing my own state of mind and how much energy I put into just staying stable.

So much for my previous long reply...

I guess I will post videos by psychologists and others who have dealt with cases of LLM-induced psychosis. They can make my point perhaps better than I can.

AI Is Slowly Destroying Your Brain

Timestamps
00:00 - Introduction
01:51 - Folie a deux
05:52 - Mechanisms of AI psychosis
10:05 - What makes the mind healthy?
13:24 - Behavioral manifestation
14:48 - Testing different models
17:56 - How to use AI safely

[Laird]

So much for my previous long reply

Uh, no, dude. You're projecting. I've pointed out how the demonstrable prowess of LLMs in arenas such as software development and mathematics totally override the sort of minimising arguments provided in the video to which you linked. Your response to that? Crickets. You know nothing about them. Yet you have the nerve to insinuate that I am ignoring your supposedly decisive arguments. You don't have any. You have therefore been reduced to:

1. Ad-homs: "true believers", "snake-oil salesmen", etc.
   
2. Red herrings: AI psychosis and mental health concerns; the dangers of cognitive offloading (which I'm not disputing anyway).
   

There's no point to engaging beyond what I've already done. Pearls before swine, etc. Like I said, I can help you to inform yourself, but you don't seem to want that.

[Laird]

P.S. Because we posted at around the same time, I'm flagging that I'd responded again, in post #523, to your "Please, go ahead" request.

There's no point to engaging beyond what I've already done. Pearls before swine, etc. Like I said, I can help you to inform yourself, but you don't seem to want that.

And that's why I lose interesting in debating with you about it ~ you have decided whatever I have to share is meaningless, so you have appeared to only misunderstand and so misrepresent my statements, so nothing I say seems to matter, so why do I bother even trying any further? My current attempts also go nowhere, so I am even less incentivized.

Videos from better-spoken individuals than I are all I have left, as my own words don't seem to mean anything.

[Laird]

His main argument

To add to what I wrote in that post, I think it's worth noting that nowhere in his talk did he define what he means by "information" in this context.

And that's why I lose interesting in debating with you about it ~ you have decided whatever I have to share is meaningless, so you have appeared to only misunderstand and so misrepresent my statements, so nothing I say seems to matter, so why do I bother even trying any further? My current attempts also go nowhere, so I am even less incentivized.

Videos from better-spoken individuals than I are all I have left, as my own words don't seem to mean anything.

"...Defenceless under the night
Our world in stupor lies;
Yet, dotted everywhere,
Ironic points of light
Flash out wherever the Just
Exchange their message..."
 -Auden, 9/1/1939
 

More from this article, especially relevant to the exchange in this thread:

About that Matt Shumer post that has nearly 50 million views

Gary Marcus

All morning people have been asking me about a blog post by Matt Shumer that has gone viral, with nearly 50 million views on X.

It’s a masterpiece of hype, written in the style of the old direct marketing campaigns, with bold-faced call outs like “I know this is real because it happened to me first” and “I am no longer needed for the actual technical work of my job”. It’s chock full of singularity vibes...

Shumer didn’t cite the new Caltech/Stanford article that reviews a wide range of reasoning errors in so-called reasoning models [or the Apple reasoning paper or the ASU mirage paper, etc]. The picture he sells just isn’t realistic, however much people might wish it were true. I should add that Shumer is the guy who was once famous for apparently exaggerated claims about a big model of his that didn’t replicate and that many people saw a fraud; he likes to sell big. But that doesn’t mean we should take him seriously.

The security of the autogenerated code is very much in question, as for example this article last week noted (precisely as this substack has long warned)