RISKS-LIST: Risks-Forum Digest Monday 10 July 2023 Volume 33 : Issue 75

ACM FORUM ON RISKS TO THE PUBLIC IN COMPUTERS AND RELATED SYSTEMS (comp.risks)
Peter G. Neumann, founder and still moderator

***** See last item for further information, disclaimers, caveats, etc. *****
This issue is archived at <http://www.risks.org> as
<http://catless.ncl.ac.uk/Risks/33.75>
The current issue can also be found at
<http://www.csl.sri.com/users/risko/risks.txt>

Contents:
A Myth About Innovation May Have Doomed the Titan (Naomi Oreskes)
OceanGate's safety culture (Robert Dorsett)
High-altitude upset
Robotaxi haters in San Francisco are disabling the AVs with traffic cones
(TechCrunch)
Tesla Autopilot tricked with wheel weights; Amazon, Alibaba pull listings
(WashPost)
How Tom Brady's Crypto Ambitions Collided With Reality' (NYTimes)
Chatbots without guardrials open next round of AI debate
(Stuart A. Thompson)
ChatGPT getting dumber? (Rik Farrow)
The Risk from AI Isn't just existential (Evgeny Morozov)
Gödel, Escher, Bach, and AI (Douglas Hofstadter)
Why Car Repairs Have Become So Expensive (NYTimes)
The mystery of the Ain Dubai, the_world's largest -- broken -- Ferris wheel
(WashPost)
Suncor swaps out laptops after cybersecurity incident as energy sector takes
stock of risks (CBC)
Unauthorized party' obtained Petro-Points members' contatc information in IT
breach, company says (CBC)
One Careless Act of War Could Destroy All Satellites in Just 40 Years
(ScienceAlert)
EV Charger Hacking Poses a Catastrophic Risk (WiReD)
Georgia won't, can't certify voting update addressing Halderman report
(Douglas Lucas)
India's religious chatbots condone violence using the voice of god (CBC)
Re: Three Companies Supplied Fake Comments to FCC (Rebecca Mercuri)
Re: The cleaner did it: an uncool act. (Mike Scott)
Re: Is America Ready For AI-Powered Politics? (Amos Shapir)
Abridged info on RISKS (comp.risks)

----------------------------------------------------------------------

Date: Mon, 3 Jul 2023 8:22:08 PDT
From: Peter Neumann <neumann@csl.sri.com>
Subject: A Myth About Innovation May Have Doomed the Titan (Naomi Oreskes)

Naomi Oreskes, *The New York Times* op-ed, 3 July 2023

The man behind the submersible had argued that regulation stifles new ideas.

Regulation may slow things down, but it also saves lives.
Sometimes slowing down is a good thing.

------------------------------

Date: Mon, 3 Jul 2023 19:33:19 -0500
From: Robert Dorsett
Subject: OceanGate's safety culture

A first-person account of the Titan, and a deep dive (no pun intended) on
Oceangate's safety culture. The first article paints an image of meticulous
pro-forma operational safety, while the second shows profound hot-dogging at
the engineering level.

David Pogue is a long-time writer on the consumer computer industry.
Ben Taub is an investigative journalist.

https://nymag.com/intelligencer/2023/06/what-i-learned-on-a-titanic-submarine-expedition.html

What I Learned on a Titanic Sub Expedition
*The New Yorker* (nymag.com)

https://www.newyorker.com/news/a-reporter-at-large/the-titan-submersible-was-an-accident-waiting-to-happen

The Titan Submersible Was An Accident Waiting to Happen

newyorker.com

And Rush's patent (note you can patent just about anything these days;
patents can be powerful weapons, but have little intellectual merit):

https://patents.google.com/patent/US11119071B1/en?q=(stockton+rush+carbon+oceangate)&oq=stockton+rush+carbon+oceangate

------------------------------

Date: Mon, 3 Jul 2023 19:35:54 -0500
From: Robert Dorsett via Ata-watchers <ata-watchers@airlinersafety.info>
Subject: High-altitude upset

An incident I was unaware of:

During cruise flight above the Arabian Sea, Indian Ocean, approximately
one minute after it had been passed overhead by an Airbus A380 on opposite
course, the CL604 was subject to temporary loss of control.

After it had lost approximately 9,000 ft of altitude the pilots regained
control of the aircraft and subsequently landed at an alternate aerodrome
at Muscat Airport, Oman.

The accident occurred over international waters. Thus the BFU as
representative of the State of Registry of the accident aircraft is
responsible for the conduct of the investigation. In accordance with
international regulations, the air accident investigation authorities of
Oman, India, the United Arab Emirates, Canada, USA, and France will assist
the BFU in this investigation.

------------------------------

Date: Thu, 6 Jul 2023 20:40:25 -0700
From: Lauren Weinstein <lauren@vortex.com>
Subject: Robotaxi haters in San Francisco are disabling the AVs with
traffic cones (TechCrunch)

I predicted years ago that eventually robohaters would use simple methods
like this to disable robocars when there's no human to deal with them. -L

https://techcrunch.com/2023/07/06/robotaxi-haters-in-san-francisco-are-disabl
ing-waymo-cruise-traffic-cones/

------------------------------

Date: Sat, 8 Jul 2023 10:04:47 -0400
From: Gabe Goldberg <gabe@gabegold.com>
Subject: Tesla Autopilot tricked with wheel weights; Amazon, Alibaba pull
listings (WashPost)

The devices, which aim to stop Tesla Autopilot from nagging drivers to put
their hands on the wheel, have been criticized by regulators and safety
experts

https://www.washingtonpost.com/technology/2023/07/07/tesla-fsd-autopilot-wheel-weights/

------------------------------

Date: Thu, 6 Jul 2023 16:30:33 -0400
From: Gabe Goldberg <gabe@gabegold.com>
Subject: How Tom Brady's Crypto Ambitions Collided With Reality' (NYTimes)

The superstar quarterback is among the celebrities dealing with the fallout
from the crypto crash. Others, like Taylor Swift, escaped.

As the FTX cryptocurrency exchange imploded last fall, Tom Brady, the
seven-time Super Bowl-winning quarterback, made an urgent phone call.

He dialed Sina Nader, FTX’s head of partnerships. The exchange’s staff was
in the middle of a crisis meeting with its beleaguered founder, Sam
Bankman-Fried. Mr. Nader couldn't answer. “I never would've expected to
decline a call from Tom Brady,” he said.

Mr. Brady had reasons to be concerned. As an “ambassador” for FTX, he had
appeared at the company’s conference in the Bahamas and in TV commercials
that promoted the exchange as “the most trusted” institution in the loosely
regulated world of crypto.

https://www.nytimes.com/2023/07/06/technology/tom-brady-crypto-ftx.html

------------------------------

Date: Sat, 8 Jul 2023 13:10:22 PDT
From: "Peter G. Neumann" <Neumann@CSL.SRI.COM>
Subject: Chatbots without guardrials open next round of AI debate
(Stuart A. Thompson)

Stuart A. Thompson, *The New York Times* Business, 8 Jul 2023

``The concern is completely legitimate and clear:
These chatbots can and will say anything if left to their
own devices.'' Oren Etzioni

``Every demographic and interest group deserves their model.
Open-source is about letting people choose.'' Eric Hartford,
behind WizardLM-Uncensored

AI chatbots have lied about notable figures, spewed misinformation, or
even advised users on how to commit suicide. To mitigate the tools'
most obvious dangers, companies like Google and OpenAI have carefully
added controls that limit what the tools can say. Now a new wave if
chatbots, developed far from the epicenter of the AI book, are coming
online without many of those guardrails. [...]

``Fake news is bad. But is it really the creation of it that's bd?
Bcause in my mind it's the distribution that's bad. [...] It's only
if I get that into a reputable publication [like the front page of the
*Times*], Yannic Kilcher

[FALSE? Every bogus piece of item on the WWWeb is relevant if
people believe it. PGN]

------------------------------

From: Rik Farrow <rik@rikfarrow.com>
Date: Fri, 7 Jul 2023 19:48:52 -0700
Subject: ChatGPT getting dumber?

Although not quite on topic, really part of an earlier thread, this does
pertain to "ChatGPT' getting dumber:
The Curse of Recursion: Training on Generated Data Makes Models Forget
Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Yarin Gal, Nicolas Papernot,
Ross Anderson
https://arxiv.org/abs/2305.17493

Large Language Models reinforce the strongest links, so results are skewed
away from less popular connections. Quoting the abstract:

We find that use of model-generated content in training causes irreversible
defects in the resulting models, where tails of the original content
distribution disappear. We refer to this effect as Model Collapse and show
that it can occur in Variational Autoencoders, Gaussian Mixture Models and
LLMs.

------------------------------

Date: Mon, 3 Jul 2023 11:36:41 PDT
From: Peter G Neumann <neumann@csl.sri.com>
Subject: The Risk from AI Isn't just existential (Evgeny Morozov)

Evgeny Morozov, *The New York Times" Sunday opinion
Worry not that the technolopgy will amplify our intelligence,'
Worry that it will shrink it.