This post addresses one of the aspects for discussion that I suggested in this thread's opening post:
The extent to which current AI systems model ("understand") the world, and whether/how this "understanding" might be improved.
I found an interesting insight on this from Geoffrey Hinton in an interview with him on CBS seven months ago titled (on YouTube)
Full Interview: "Godfather of artificial intelligence" talks impact and potential of AI. His insight comes at
33m11s into the interview, when he starts off this exchange with the interviewer:
GH: You can ask me the question, "Some people say that these big models are just autocomplete"...
Interviewer: Well, on some level, the models
are autocomplete. We're told that the large language models - they're just predicting the next word. Is that not so simple?
GH: No, that's true, they are just predicting the next word, and so they're just auto-complete - but, ask yourself the question of what do you need to understand about what's been said so far in order to predict the next word accurately, and, basically, you have to understand what's been said to predict what comes next - so
you're just autocomplete too.
He goes on to provide an interesting example of the understanding required of an AI to translate a particular simple sentence into French.
I later came across a couple of research papers which demonstrate that modern AI systems model reality in specific ways, suggestive of the possibility that language models like, say, ChatGPT, model, and, in some sense - as Geoffrey Hinton seems to suggest - "understand" reality more broadly.
The first is
Implicit Representations of Meaning in Neural Language Models by Belinda Z. Li, Maxwell Nye, and Jacob Andreas, 1 June, 2021. Here's the abstract:
Quote:Does the effectiveness of neural language models derive entirely from accurate modeling of surface word co-occurrence statistics, or do these models represent and reason about the
world they describe? In BART and T5 transformer language models, we identify contextual word representations that function as models of entities and situations as they evolve throughout a discourse. These neural representations have functional similarities to linguistic models of dynamic semantics: they support a linear readout of each entity’s current properties and relations, and can be manipulated with predictable effects on language generation. Our results indicate that prediction in pretrained neural language models is supported, at least in part, by dynamic representations of meaning and implicit simulation of entity state, and that this behavior can be learned with only text as training data.
Their paper comes with the caveat that their
"experiments do not explore the extent to which LMs encode static background knowledge, but instead the extent to which they can build representations of novel situations described by novel text."
Nevertheless, it's very suggestive.
The second paper is
EMERGENT WORLD REPRESENTATIONS: EXPLORING A SEQUENCE MODEL TRAINED ON A SYNTHETIC TASK by Kenneth Li, Aspen K. Hopkins, David Bau, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg, 27 February, 2023. Here's the abstract:
Quote:Language models show a surprising range of capabilities, but the source of their apparent competence is unclear. Do these networks just memorize a collection of surface statistics, or do they rely on internal representations of the process that generates the sequences they see? We investigate this question in a synthetic setting by applying a variant of the GPT model to the task of predicting legal moves in a simple board game, Othello. Although the network has no a priori knowledge of the game or its rules, we uncover evidence of an emergent nonlinear internal representation of the board state. Interventional experiments indicate this representation can be used to control the output of the network. By leveraging these intervention techniques, we produce “latent saliency maps” that help explain predictions.
Again, the paper is limited to a specific type of representation in AI systems, but suggestive of AI system representation - modelling/"understanding" - in general.
Even if, though, AI systems are modelling reality, they nevertheless are prone to making stuff up, i.e., "hallucinating". In the latest Ezra Klein Show podcast linked in my post above ("A.I. Could Solve Some of Humanity’s Hardest Problems. It Already Has."), Demis Hassabis of Google DeepMind suggests some possible ways to avoid this problem, including assigning probability to predictions, as DeepMind's AlphaFold system does, as well as incorporating a mechanism to look up and corroborate with references online.
I'm reconsidering, then, the sentiments that I expressed in
this earlier post: that the current deep learning models seem to fundamentally lack a facility for (mimicry of) understanding, and that simply scaling them up cannot remedy this.
I now wonder whether higher cognitive skills such as mathematical and logical reasoning
might be achievable simply by scaling up these deep-learning systems, without any need to marry them with more explicitly-coded symbolic reasoning modules. It's a very interesting possibility.
Further resources:
Gary Marcus (as heard in that Ezra Klein Show episode "A Skeptical Take on the A.I. Revolution" which prompted me to express those earlier sentiments in that earlier post) again:
Why Robot Brains Need Symbols on Medium on 30 October, 2019.
Geoffrey Hinton (et al) again (as seen in the YouTube video above) in a review article (as referenced by Gary in his Medium article linked just above):
Deep learning in Nature on 28 May, 2015.
