Epic AI Fails — A List of Failed Machine Learning Projects

I don't know if this is the right thread for this.

This Montreal newspaper tested ChatGPT for the provincial bar (law) exam. It got 12%!

Article translated here:
https://www-lapresse-ca.translate.goog/a..._tr_pto=sc

It appears the ignorance and stupidity demonstrated by ChatGPT on this occasion is easily explainable, as there being grossly insufficient human writings on the subject on the Internet for the AI large language model to process. The AI just needed much more Internet-derived human-written information on Canadian law, than the relatively small amount available. Whereas, the Internet has a huge amount of information on US law, so as expected the AI turns out to excel at the American Bar exam.

This goes even more to show that such AI systems can be nothing more than complex zero consciousness mechanisms, not sentient or aware.

From the article:

"...similar experiments have been carried out in the United States. There, ChatGPT had shone in the American Bar exam, earning a mark that would have placed him in the top 10% of students. OpenAI also claims that its robot ranked in the top 10% of scores in several legal tests. How to explain this gap?

"....the major problem comes from the fact that the robot is much more fed by American law than by Quebec law. “He has peripheral knowledge of Canadian law,” says Mr. Anctil. It's like asking an American lawyer to practice the Civil Code of Quebec. »

“He has access to much, much more American material,” adds Laurent Charlin, associate professor at the École des Hautes Etudes Commerciales (HEC) of the University of Montreal and member of the Chair in Artificial Intelligence of Canada."

"(Also), these tests are carried out using an extremely precise protocol, which is not the case with the La Presse experiment , which is more "artisanal", notes Dave Anctil, professor of philosophy at Collège Jean -de-Brébeuf and researcher affiliated with the International Observatory of the Societal Impacts of Artificial and Digital Intelligence at Laval University. Under optimal conditions, the robot might have increased its rating. “You have to be careful with this type of artisanal testing."

I don't know if this is the right thread for this.

This Montreal newspaper tested ChatGPT for the provincial bar (law) exam. It got 12%!

Article translated here:
https://www-lapresse-ca.translate.goog/a..._tr_pto=sc

I think it’s a bit misplaced as the issue mentioned in the article doesn’t touch upon any fundamental issues with LLM. Obviously there are areas in which it lacks sufficient training. I have also asked it a few questions about danish law (how to do accounting for a rented out apartment) on which it couldn’t provide sufficient enough details. When people talk about AI replacing jobs it isn’t about “the chatGPT” but rather a ton of special purpose trained AIs that’s going to evolve in the following years.

When people talk about AI replacing jobs it isn’t about “the chatGPT” but rather a ton of special purpose trained AIs that’s going to evolve in the following years.

It's been "on the cards" since robots taking over factories were threatened in the 1980s and it has come no further since then.  Factories are still manned by overworked, underpaid and underappreciated human beings. Machines will never take over the world because they will never be creative thinkers.  The thinking is always done by human beings.