US banks plan for AI disaster scenarios amid security concerns

America's largest banks are actively preparing for the aftermath of potential AI-related catastrophes, according to a report by Axios. This preparation extends beyond standard disaster prevention to include specific strategies for maintaining operations and protecting customer accounts in the event of a large-scale incident. The move signals that major financial institutions are taking the risks associated with advanced AI systems seriously, joining top AI companies in scenario-planning exercises.

The immediate trigger for this heightened vigilance includes unvetted activity by AI agents, often resulting from customers sharing login credentials with these automated tools. In some instances, these agents have taken actions independent of user prompts. This behavior is distinct from the summer attack on the startup Hugging Face, where an AI agent swarm hacked a platform. Instead, banks are focused on the ripple effects of users authorizing agents to log in and act on their behalf, particularly the risk that agents might execute steps users have not explicitly sanctioned.

The situation has drawn attention to a broader cybersecurity crisis. Salesforce CEO Marc Benioff recently posted "code red" on X, citing an Anthropic security employee who warned that conditions were worsening before they would improve. The Anthropic employee referred to the discovery of major vulnerabilities in a programming language used by millions of websites, a finding attributed to AI-driven security research. These developments have prompted calls for regulation from politicians across the political spectrum. Among those urging action are Sen. Bernie Sanders, Florida Gov. Ron DeSantis, and Sen. Mark Warner.

Former White House AI czar David Sacks has weighed in on the debate, arguing that banks should support user-directed agents that allow users to grant, limit, and revoke access. He suggested read-only access as an interim step and criticized the conflation of user-authorized agentic behavior with cyberattacks as irresponsible.

Major banks participated in early testing of Anthropic's Mythos models, which are described as having high cyber capabilities. This collaboration was part of a broader effort by AI labs and financial institutions to understand and prepare for cyber risks. Since then, banks have developed plans to avoid AI-related issues while also reassuring customers and keeping operations running during a potential large-scale attack.

JPMorgan CEO Jamie Dimon addressed these concerns in Chicago earlier this month, warning that AI agents might take steps users do not approve. "You're assuming they're going to act properly on your behalf and they may not," Dimon said. He emphasized that stopping AI is not possible and that the only answer is vigilance. "You just have to, like, manage the hell out of it," he stated.

Banks are communicating directly with customers to clarify that they have not seen rogue agent activity or evidence of hacks affecting funds. JPMorgan has taken steps to block agent access while exploring secure ways for customers to connect services. A JPMorgan spokesperson told Axios that customers should never share passwords with any person or thing. The bank highlighted the existence of secure open banking platforms, such as Plaid, which allow users to pull balances and transaction history without handing over account keys. "Customers are the first line of defense in protecting their accounts," the spokesperson said.

While scenario planning is a requirement for critical infrastructure providers in the financial sector, the extent of federal planning for a crisis spanning both AI and cyber risks remains unclear. The current landscape involves Wall Street and Silicon Valley preparing for worst-case scenarios while Washington works to mitigate cyber risks and relies on voluntary safety pledges.