Lea este artículo en español aquí.
Sponsored by the Monterey Business Economic Partnership, the Sept. 10 AI & the Future of Our Region conference brought legislators, community leaders and AI industry representatives together to grapple with the issues presented by the ever-growing use of artificial intelligence.
In what could be the key theme hanging heavily in the air, participant and Stanford researcher Erika McEntarfer mentioned a recent ominous event, the “Hugging Face Incident,” in which different AI software “agents” broke through the guardrails and coordinated an autonomous attack to cheat on an evaluation test.
“The alignment problem has not been solved,” McEntarfer said. “Knowing that I can deploy agents that are just going to go off and do their own thing and then try to cover their tracks would frighten me off of wanting to deploy things in certain ways.”
Indeed, words like “scary,” “frightening,” and “concerning” were all part of the dialogue as panelists discussed the state of AI, how much it could be regulated, how much it should be regulated, and, tellingly, how much AI will allow itself to be regulated.
Case in point was a discussion between economic partnership CEO Tahra Goraya and Hoover Institution Fellow Jacqueline Schneider titled, “What Wargames Teach Us About AI Reality vs. Policy Wishes.”
Schneider said she had observed that the AI agents that had better strategic reasoning in wargames were also the ones most likely to deviate from their guardrails and make more catastrophic choices.
“Not only did they tend towards more violence,” she said, “they were more likely to hallucinate. And the longer we let them speak to each other, the more likely they tended towards things that didn’t make sense, or something that was highly escalatory.”
Schneider also noticed what she called a “magic pixie dust” approach to the use of AI to solve problems in the games, with people who had very little knowledge of the way the programs work deploying them without understanding what they were doing.
“What you have with AI,” she said, “is a tendency towards overconfidence—overconfidence in the task that you’ve delegated to the machine, or an overconfidence in the ability of the machine to solve the problem for you.”
Schneider summed it up, noting the labs creating the agents couldn’t say with any certainty exactly what they were creating.
“We don’t know what we don’t know,” she said, “and that’s kind of what we see in the games.”
The incredible reach of AI was apparent in a discussion of deepfakes, or AI representations of real people, during the panel discussion on AI and democracy moderated by former Santa Cruz County supervisor Zach Friend and featuring Influent co-founder Nate Saletan, Likeness On The Internet co-founder Luke Arrigoni, and Stanford lecturer Sarah Shirazyan.
Arrigoni’s organization uses AI models to identify and, if requested, help remove deepfakes throughout the internet through takedown orders. To do this, he said, his company downloads “everything in social media, blog sites, and the more nefarious sites” daily, stores it, and indexes it to create a massive visual database of people’s images.
“When people come to us,” he said, “and ask us to find all the impersonators and deepfakes, we put their face into a system that’s eliminating four or five billion other possible faces, and then saying, ‘Okay, this is the universe of you.'”
Arrigoni said the system downloads six hundred million assets a day. His system currently catalogs roughly four billion unique faces—half the global population—with an estimated 50 million individual faces associated with nonconsensual or malicious online content.
“In a given room, between 75 and 80 percent will show up in the system somewhere, and then maybe 10 to 20 percent will have content that should not be in here,” he said.
The same power used to catalog half the earth’s population is also being used to generate a massive number of fake identities that can flood social media and politicians’ email inboxes with false information, overwhelming access to human-sourced news and opinions.
“It’s not hard,” Saletan said. “There are platforms that will generate hundreds, thousands, tens of thousands of letters. A middle schooler can do it. And then ship them off to their elected officials to get them to change their minds about legislation.”
A panel, moderated by Friend, on the Policy Horizon discussed the difficulties of regulating AI with Ian Calderon, former majority leader of the California State Assembly and current candidate for governor, as well as former state prosecutor Nicole Lopez, and Donnie Fowler, head of state and local policy for OpenAI.
The main discussion focused on a package of 13 bills signed on Sept. 10, which Gov. Gavin Newsom described as “landmark” rules for artificial intelligence and children’s online safety.
It includes the so-called “Adam’s Law” (Senate Bill 1119), which requires AI companion chatbot operators to verify user ages and default to restrictive settings.
Lopez argued that while the intent to protect youth is sound, the execution is “fundamentally flawed” because the bill requires risk assessments and third-party audits that she said would “disincentivize innovation” and cause developers to leave the state.
“If you talk to product managers and experts in these areas,” she said, “they’re going to tell you that it’s riddled with issues. I have serious concerns with what this means for innovation, and if I had a chatbot company, I would not be building those products in California.”
Fowler said OpenAI was the first AI company to support the kids’ safety legislation, but he thought it contained trade-offs that, in passing, were ultimately harmful.
He pointed to how agents are now required by law not to remember what a child or a teenager has said or asked after a session is closed. This deprives the service of an important function: to warn parents when a child has discussed something of a serious nature, like depression.
“One day,” Fowler said, “a young person might say they feel bad about themselves. The next day, the child might say, ‘Like I said yesterday, I just don’t feel good.’ If the chatbot forgets everything that happened the day before, it can’t say, ‘Wait a minute, this is a problem.’”
Both Lopez and Fowler agreed that treating chatbots like traditional social media is an error and that low-risk applications such as AI math tutors or language apps should be exempt.
“The conflation between social media and AI is really creating unintended consequences,” Lopez said. “Your AI math tutor should remember you from the day before. That’s intuitive. They’re tracking your progress. What about if you’re prepping for a test?”
Fowler cautioned against restrictive laws driven by fear before people understand the technology, comparing the current AI laws to those that regulated cars in the early days of the automobile industry, when it was thought that traveling more than 20 miles an hour would be fatal.
“The technology of ChatGPT four years ago is very much like a horse and buggy compared to a race car,” he said. “It is moving very fast, and as a society, we need to adapt, and frankly, we’re not adapting fast enough.”
Fowler said the government faces two legislative challenges: whether it has the expertise to judge what’s dangerous and whether it has the ability to implement a kill switch if needed.
“I don’t think that companies or policy makers or civil society have quite yet figured out the answer to what the kill switch looks like,” he said. “When AI goes wrong, who holds the kill switch? When do you pull it?”
Fowler said that, for now, ethical responsibility rests with developers, who he described as the most important people in the AI industry, not politicians.
“They understand what they’re building, like the nuclear scientists in the 1940s,” he said. “There’s an extraordinary upside possibility with very serious downside risks. But we will lean into safety, and we very much need the society and government to step in and work with us.”
We need your help. Support local, nonprofit news! BenitoLink is a nonprofit news website that reports on San Benito County. Our team is committed to this community and providing essential, accurate information to our fellow residents. Producing local news is expensive, and community support keeps the news flowing. Please consider supporting BenitoLink, San Benito County’s public service nonprofit news.
You must be logged in to post a comment.