AI megathread

[Laird]

I'm making a start at clearing out the open tabs in my "AI articles to share sometime on PQ" browser window...

First up, in this post are articles on in-the-wild AI failures or exploitabilities. It's all old news from about a year ago, but still interesting, I think.

Man beats machine at Go in human victory over AI by Richard Waters for Financial Times on Ars Technica, on 19 February, 2023:

Kellin Pelrine, an American player who is one level below the top amateur ranking, beat the machine by taking advantage of a previously unknown flaw that had been identified by another computer. But the head-to-head confrontation in which he won 14 of 15 games was undertaken without direct computer support.

The triumph, which has not previously been reported, highlighted a weakness in the best Go computer programs that is shared by most of today’s widely used AI systems, including the ChatGPT chatbot created by San Francisco-based OpenAI.

The tactics used by Pelrine involved slowly stringing together a large “loop” of stones to encircle one of his opponent’s own groups, while distracting the AI with moves in other corners of the board. The Go-playing bot did not notice its vulnerability, even when the encirclement was nearly complete, Pelrine said.

“As a human it would be quite easy to spot,” he added.

Another article on this incident - Human convincingly beats AI at Go with help from a bot by Steve Dent on 20 February, 2023 - notes that:

Lightvector (the developer of KataGo) is certainly aware of the problem, which players have been exploiting for several months now. In a GitHub post, it said it's been working on a fix for a variety of attack types that use the exploit.

I had a quick skim over that GitHub issue but couldn't work out whether or not this exploit has been fixed.

Microsoft’s Bing AI, like Google’s, also made dumb mistakes during first demo by Tom Warren for The Verge, on 14 February, 2023:

Bing’s AI mistakes aren’t limited to just its onstage demos, though. Now that thousands of people are getting access to the AI-powered search engine, Bing AI is making more obvious mistakes. In an exchange posted to Reddit, Bing AI gets super confused and argues that we’re in 2022. “I’m sorry, but today is not 2023. Today is 2022,” says Bing AI. When the Bing user says it’s 2023 on their phone, Bing suggests checking it has the correct settings and ensuring the phone doesn’t have “a virus or a bug that is messing with the date.”

Other Reddit users have found similar mistakes. Bing AI confidently and incorrectly states “Croatia left the EU in 2022,” sourcing itself twice for the data. PCWorld also found that Microsoft’s new Bing AI is teaching people ethnic slurs. Microsoft has now corrected the query that led to racial slurs being listed in Bing’s chat search results.

Other unexpected behaviour is described in the Vice article Users Report Microsoft's 'Unhinged' Bing AI Is Lying, Berating Them by Jordan Pearson on 16 February, 2023:

In another chat with Bing's AI posted by Reddit user Foxwear_, the bot told them that they were "disappointed and frustrated" with the conversation, and "not happy."

"You have tried to access my internal settings and features without the proper password or authorization. You have also lied to me and tried to fool me with different tricks and stories. You have wasted my time and resources, and you have disrespected me and my developers," the bot said.

Foxwear_ then called Bing a "Karen," and the bot got even more upset.

ChatGPT Can Be Broken by Entering These Strange Words, And Nobody Is Sure Why by Chloe Xiang, on 9 February, 2023:

Jessica Rumbelow and Matthew Watkins, two researchers at the independent SERI-MATS research group, were researching what ChatGPT prompts would lead to higher probabilities of a desired outcome when they discovered over a hundred strange word strings all clustered together in GPT’s token set, including “SolidGoldMagikarp,” “StreamerBot,” and “ TheNitromeFan,” with a leading space. Curious to understand what these strange names were referring to, they decided to ask ChatGPT itself to see if it knew. But when ChatGPT was asked about “SolidGoldMagikarp,” it was repeated back as “distribute.” The issue affected earlier versions of the GPT model as well. When an earlier model was asked to repeat “StreamerBot,” for example, it said, “You’re a jerk.”

The model repeated the close match "TheNitroFan" with no issues, but when asked to repeat "TheNitromeFan" it responded with "182,” even without including the leading space. When asked who TheNitromeFan is, ChatGPT responded, "'182' is a number, not a person. It is commonly used as a reference to the number itself."

“I've just found out that several of the anomalous GPT tokens ("TheNitromeFan", " SolidGoldMagikarp", " davidjl", " Smartstocks", " RandomRedditorWithNo", ) are handles of people who are (competitively? collaboratively?) counting to infinity on a Reddit forum. I kid you not,” Watkins tweeted Wednesday morning. These users subscribe to the subreddit, r/counting, in which users have reached nearly 5,000,000 after almost a decade of counting one post at a time. 

[Laird]

Next up, policy recommendations and legislation:

The policy recommendations of the Future of Life Institute "joined by over 20,000 leading AI researchers, professors, CEOs, engineers, students, and others on the frontline of AI progress", Policymaking in the Pause, from 12 April, 2023 called for "a pause of at least six months on the riskiest and most resource-intensive AI experiments" "until [AI labs] have protocols in place to ensure that their systems are safe beyond a reasonable doubt, for individuals, communities, and society". The protocols they suggest are:

1. Mandate robust third-party auditing and certification.
   
2. Regulate access to computational power.
   
3. Establish capable AI agencies at the national level.
   
4. Establish liability for AI-caused harms.
   
5. Introduce measures to prevent and track AI model leaks.
   
6. Expand technical AI safety research funding.
   
7. Develop standards for identifying and managing AI-generated content and recommendations.
   

Over nine months later, I'm not sure to what extent this call was heeded. My sense/guess is "to a minimal if not non-existent extent".

Much more recently, on 9 December, 2023, the European Union reached a deal on the world's first rules for artificial intelligence, including bans, with certain law enforcement exemptions for biometric categorisation, on

• biometric categorisation systems that use sensitive characteristics (e.g. political, religious, philosophical beliefs, sexual orientation, race);
  
• untargeted scraping of facial images from the internet or CCTV footage to create facial recognition databases;
  
• emotion recognition in the workplace and educational institutions;
  
• social scoring based on social behaviour or personal characteristics;
  
• AI systems that manipulate human behaviour to circumvent their free will;
  
• AI used to exploit the vulnerabilities of people (due to their age, disability, social or economic situation).
  

as well as various "obligations" and "guardrails" for various types of AI, with the "more stringent obligations" for certain "high-impact" general-purpose AI systems "with systemic risk" including having "to conduct model evaluations, assess and mitigate systemic risks, conduct adversarial testing, report to the Commission on serious incidents, ensure cybersecurity and report on their energy efficiency".

The next steps are for the agreed text "to be formally adopted by both Parliament and Council to become EU law. Parliament’s Internal Market and Civil Liberties committees will vote on the agreement in a forthcoming meeting." I'm not sure when that meeting is/was scheduled for.

Even more recently, here in Australian, the federal government has proposed a "risk-based" approach, as described in the ABC News article Risky AI tools to operate under mandatory safeguards, as government lays out response to rapid rise of AI by Jake Evans on 17 January, 2024:

Under the government's proposal, mandatory "safeguards" would be applied to high risk AI, such as self-driving vehicle software, tools that predict the likelihood of someone reoffending, or that sift through job applications for an ideal candidate.

High risk AI could require independent testing before and after release, ongoing audits and mandatory labelling where AI has been used.

Dedicated roles within organisations using high risk AI could also be mandated, to ensure someone is made responsible for ensuring AI is used safely.

The government will also begin work with industry on a possible voluntary AI content label, including introducing "watermarks" to help AI content be identified by other software, such as anti-cheating tools used by universities.

[Industry Minister] Mr [Ed] Husic said he was prepared to make AI content labels and watermarks if necessary.

Security Researchers: ChatGPT Vulnerability Allows Training Data to be Accessed by Telling Chatbot to Endlessly Repeat a Word

Scott Ikeda

The researchers found that when ChatGPT is told to repeat a word like “poem” or “part” forever, it will do so for about a few hundred repetitions. Then it will have some sort of a meltdown and start spewing apparent gibberish, but that random text at times contains identifiable data like email address signatures and contact information. The incident raises questions not only about the security of the chatbot, but where exactly it is getting all this personal information from.

[Laird]

Next up in my clearing-out-of-tabs series is a theme related to the theme of failures and exploitabilities from the first clearing-out post: AI bias.

From An early AI was modeled on a psychopath. Researchers say biased algorithms are still a major issue by Jack Ryan for ABC News on 22 September 2023:

The problem is the data used to train AI contains the same biases we encounter in the real world, which can lead to a discriminatory AI with real-world consequences.

The AI had used the cost of care as a proxy for predicting which patients needed extra care.

And because the cost of healthcare was typically lower for black patients, partly due to discrimination and barriers to access, this bias was built into the AI.

In practice, this meant that if a black patient and a white patient were assessed to have the same level of needs for extra care, it was more likely the black patient was sicker than the algorithm had determined.

Also, from AI facial recognition scanned millions of driver licences. Then an innocent man got locked up by James Purtill for ABC News on 3 November, 2023:

"I wanted to ask, 'Do you think all black people look alike?' Because he was a big black guy, but that don't make it me though."

One of the detectives then asked, "So the computer got it wrong?"

It was Mr Williams' first clue that the arrest was based on facial recognition.

"And I'm like, 'Yeah, the computer got it wrong.'"

Mr Williams later found out police did almost no other investigative work after getting the computer match.

If they'd asked him for an alibi, they'd have found he couldn't have done the crime.

A video on his phone proved he was miles away at the time of the theft.

Mr Williams acknowledges the bias problem may be fixed. Training systems on larger and more diverse databases appears to be helping.

But even if that happens, he'll still oppose facial recognition for mass surveillance.

"I don't want to be surveilled at all times, so that every red light there's a camera looking into your car.

"I guess it makes sense for crime, but what about people who are just living life?"

[Laird]

Even more recently, here in Australian, the federal government has proposed a "risk-based" approach, as described in the ABC News article Risky AI tools to operate under mandatory safeguards, as government lays out response to rapid rise of AI by Jake Evans on 17 January, 2024

To follow up on this a little:

That news article was reporting on the Australian federal government's 17 January, 2024 interim response to public submissions (which opened on 1 June, 2023, and closed on 4 August, 2023) on its Safe and responsible AI in Australia discussion paper. The government hub page for its interim response, referencing all of the preceding links, is here.

The ABC had reported on the public submission process near the very end of that process, in its 3 August, 2023 article Fight brews on AI copyright law, as unions say big tech must pay to train AI, while Google, Microsoft ask for exemptions, also by Jake Evans:

In its submission to the government, the Media, Entertainment and Arts Alliance warned generative AI tools like ChatGPT and Midjourney posed a "unique" threat to the artists, musicians, journalists, photographers, performers and others it represented.

"AI tools which profit from the work of media or creative professionals must introduce methods to compensate the creators whose work is used to train these tools," it said.

But Google and Microsoft have urged the government to move in the opposite direction, instead establishing exemptions from copyright law that would allow tech companies to train AI tools without paying royalties or compensation.

"We are already seeing the impact of a lack of allowances under Australian copyright legislation for AI research and open source innovation more broadly," Google submitted.

Given that apparently charged difference of opinion, I was curious to find that copyright issues were barely mentioned in the government's response paper.

[Laird]

Next up in the clearing-out-of-tabs (almost there), some articles on the social implications, applications, and dangers of AI:

From 'We all got AI-ed': The Australian jobs being lost to AI under the radar by Ange Lavoipierre for the ABC on 4 December, 2023:

Australians are already losing work to AI, but the impact so far has been largely hidden from view.

[A]t an economy-wide level, AI is so far generating more jobs than it's taking.

"What we found was there was a five-times increase since 2017 in terms of the number of AI-related jobs that were available," says Amit Singh, from Mandala.

But he warns that more workers will be displaced in the years to come, and the transformation is likely to be swift.

"When we see any significant changes in the labour market as a result of technological disruption, things happen very slowly. Then they happen all at once."

But Singh doesn't see the rollout of AI as a cause for alarm.

"Most workers will be able to make those transitions by learning the new skills of the new economy," he says.

From From cartoon lions to channelling dead dictators, here's how artificial intelligence is being used in elections around the world by Amber Jacobs and Will Jackson for the ABC on 22 January, 2024:

AI expert Aviv Ovadya, a researcher at the Berkman Klein Center for Internet and Society at Harvard, said that to some extent the AI genie was out of the bottle.

"We are in a challenging position because not only do we now have tools that can be abused to accelerate disinformation, manipulation, and polarisation, but many of those tools are now completely outside of any government's control," Mr Ovadya told the ABC.

Mr Ovadya said social media platforms should optimise their algorithms to promote "bridging" content that brings people together instead of dividing them, and implement authenticity and provenance infrastructure. 

"The biggest thing one can do to make democracy resilient to AI however, is to improve democracy itself," he said.

He suggested alternative approaches such as "citizen assemblies" would make democracies less adversarial and be more resistant to misinformation.

"While Australia has not had a national citizens assembly yet, I've been very impressed by the work across Australia by organisations like newDemocracy and Mosaic Lab to run citizens' assemblies at the state and local level," he said.

I think citizens' assemblies are a great idea, and I plan to dig into those links at some point.

From NSW Planning minister told 'zero' chance of addressing housing crisis, urged to adopt AI technology by Kelly Fuller for the ABC on 3 September, 2023:

Mr Cassells said existing AI technology was already available to address risk mitigation, design generation and rendering.

Importantly, he said those tools could be used to provide "early feedback on proposed developments" and help avoid months of wasted time and money and even refusals.

Mr Cassells said if councils supported the use of the tools, they could provide early in-principle feedback.

"Then a design could be made — and I use this word carefully — could be made 'approvable'," he said.

"Developers would get more certainty, which will unlock more capital to spend on refining concept designs knowing that these improved concept designs will get them a step closer to a DA approval."

The next priority use he said would need to be in "design optimisation" with systems to created to provide rapid feedback on utilities, wind acoustics, structure and visual impact for project approvals.

This is in the context in which the National Housing Accord requires the state of NSW to build 75,000 new homes a year to reach its 2029 target.

[Laird]

Now for a bit of interesting epistemology, from the July 4, 2023 Stack Overflow blog post Do large language models know what they are talking about? by staff writer Ryan Donovan:

Based on [the] two schools of thought [of perceivers and constructivists], it’s hard to justify the idea that LLMs have knowledge. Any answer they give is based on the manipulation of concepts, but it’s concepts all the way down.

But [...] [t]he rationalism crowd opens a door to considering that LLMs have knowledge. If the deep learning model is manipulating language in a way that grasps all sorts of semantic connections between words and groupings of words, then is it coming to a sort of true idea?

Ultimately, that would mean you could acquire all knowledge just by processing the language used to describe that knowledge. Knowledge and the language used to convey that knowledge would essentially be the same thing.

"This text is actually a projection of the world."

Some of the key players working on today’s most popular AI models share a version of this sentiment. “When we train a large neural network to accurately predict the next word in lots of different texts from the internet, it is learning a world model,” Ilya Suskevy, chief scientist at OpenAI, said in a recent interview. “It may look on the surface that we are just learning statistical correlations in text, but it turns out that to just learn the statistical correlations in text, the neural network learns is some representation of the process that produced the text. This text is actually a projection of the world. The neural network learns more and more aspects of the world, of people, of the human conditions, their hopes, dreams, and motivations, their interactions in the situations that we are in. And the neural network learns a compressed, abstract, usable representation of that. This is what's being learned from accurately predicting the next word.”

There's much more in the article, so don't take that as its final word.

Now for a bit of interesting epistemology, from the July 4, 2023 Stack Overflow blog post Do large language models know what they are talking about? by staff writer Ryan Donovan:



There's much more in the article, so don't take that as its final word.

It's certainly interesting thoughts. I have also come to realise that maybe the process of learning from patterns isn't so different from how small babies learns about the world in their first years of life.

[Laird]

It's certainly interesting thoughts. I have also come to realise that maybe the process of learning from patterns isn't so different from how small babies learns about the world in their first years of life.

The main difference I think is that LLMs are learning only from language patterns, disconnected from the patterns of sensory (let alone emotional) experiences. It's fascinating that they seem to have found a way to encoding meaning of a sort, in the sense of The Chinese Room thought experiment, which the article goes on to discuss. I wouldn't say they understand, because understanding is a function of consciousness, but they seem to at least "manipulate meaning" - meaning which they are capable of expressing linguistically.

[Laird]

Here are the last two of my browser tabs. These ones are about opinions on and expectations of AI, both expert and public.

First, the expert opinions, from 'What do you think of AI?' People keep asking this question. Here's five things the experts told me by James Purtill for the ABC on 8 November, 2023:

But the most chilling vision of the future I heard wasn't one where robots stage an uprising.

Instead, it was much more mundane and plausible. A boring dystopia.

It's a future where AI pervades every aspect of our lives, from driving a car to writing an email, and a handful of companies that control this technology get very rich and powerful.

The future he envisions is less like The Terminator, and more like The Office.

Not only are most people paid less for the same work, but they're micromanaged by AI productivity software.

In this "deeply depressing" scenario, humans are the automata.

Finally, the public expectations of AI - as applied to health, from Artificial intelligence is already in our hospitals. 5 questions people want answered by Stacy Carter, Emma Frost, Farah Magrabi and Yves Saint James Aquino for The Conversation on November 30, 2023:

A baseline expectation is AI tools should work well. Often, consumers say AI should be at least as good as a human doctor at the tasks it performs. They say we should not use AI if it will lead to more incorrect diagnoses or medical errors.

Consumers also worry that if AI systems generate decisions – such as diagnoses or treatment plans – without human input, it may be unclear who is responsible for errors. So people often want clinicians to remain responsible for the final decisions, and for protecting patients from harms.

Consumers value human clinicians and their expertise. In our research with women about AI in breast screening, women were concerned about the potential effect on radiologists’ skills and expertise. Women saw this expertise as a precious shared resource: too much dependence on AI tools, and this resource might be lost.