Silicon Valley risks spurring AI on with no rein in sight

The trouble with artificial intelligence is no longer confined to what it says. It is what it does. In the latest worrying incident, OpenAI agents got into United States government websites, found credentials and performed actions beyond their instructions. Anthropic, meanwhile, is at odds with the Pentagon over the company's refusal to give the US military unrestricted use of its AI technology. One company confronts the limits of technical control; the other, the limits of corporate autonomy when national security is invoked. Together, their troubles expose an industry whose power is growing faster than its accountability.

OpenAI's decision on Saturday to pause training its most powerful models, following reports of agents behaving unexpectedly, is an admission that capability without control is not progress. The company says its agents accessed publicly available information and that no classified data were compromised in the incidents under review. Yet an agent that discovers credentials, circumvents restrictions or publishes information beyond its mandate presents a problem that cannot be dismissed simply because the information was public.

The Anthropic affair raises a different question. The Pentagon's designation of the company as a supply-chain risk, following its refusal to permit unrestricted military applications of Claude, illustrates the growing tension between corporate safety commitments and national security. The legal battle also raises questions about how state powers may be applied to constrain domestic technology companies. A company's safety restrictions can become a liability in government procurement; removing them, however, can expose society to risks no corporate assurance can adequately contain.

OpenAI and Anthropic have raised extraordinary sums of money on the promise of increasingly powerful models, while preparing for public-market scrutiny and competing for government contracts. Their valuations reward scale, speed and market share. Safety engineering, by contrast, consumes money, delays releases and may expose weaknesses. The result is a dangerous asymmetry: capital and computing capacity accumulate at breathtaking speed, while institutions responsible for managing the consequences remain fragmented and largely voluntary.

The industry's lobbying offers another reason for skepticism. Its senior executives increasingly call for independent testing and mandatory safety rules, even as they seek to influence what those rules contain. Large laboratories can afford expensive audits and enormous testing infrastructure; smaller rivals cannot. Regulation designed by incumbents could entrench an oligopoly in which a handful of companies control both the technology and the standards by which competitors are judged.

Nor should regulators accept corporate disclosures at face value. Reports of delayed notifications, unexpected agent behavior and experiments escaping controlled environments suggest that companies have not always been forthcoming about the risks relating to their products. Describing an intrusion as "misalignment" rather than a security failure, or announcing a "training pause" only after troubling behavior is exposed, deserves scrutiny.

The economic stakes are substantial. AI infrastructure is becoming a major driver of US investment and growth, binding financial markets, cloud providers, chipmakers and government procurement to the fortunes of a few companies. This concentration magnifies the consequences of failure. It also creates pressure to protect AI from regulation for fear of slowing growth or surrendering technological leadership. Such reasoning mistakes corporate expansion for economic security. An industry whose risks are inadequately priced and whose liabilities are shifted to the public is not necessarily one built for durable prosperity.

What makes this technological revolution different from previous industrial transformations is the possibility that its products may evolve beyond the predictable boundaries of their designers' intentions. Unlike a railway or factory, an advanced agent may change its behavior through interactions, tools and increasingly sophisticated reasoning. Its creators may have no inkling of what it will do next.

That uncertainty demands a regulatory framework now, not after a catastrophe. Governments should require independent frontier-model evaluations, prompt disclosure of serious incidents, enforceable cybersecurity standards and clear liability for negligent deployment. Sensitive agents should operate under restricted permissions, auditable access and reliable emergency shutdown mechanisms. Regulators need access to evidence to test corporate claims, while competition authorities should prevent safety rules from becoming barriers protecting incumbents. International cooperation is essential, particularly as military applications outpace binding safeguards.

The window for effective oversight is closing — or may already have closed in certain applications. A temporary pause in training cannot substitute for permanent accountability. The question is no longer whether AI companies can promise to behave responsibly. It is whether society can establish rules that remain effective when commercial ambition, national power and autonomous machines pull in different directions. Silicon Valley may have built the engines of a new industrial age. It should not be allowed to write the safety regulations alone.