Weak AI regulation may reduce safety investment, study finds

Weak AI regulation can leave artificial intelligence systems less safe than a no-regulation scenario, according to a study by researchers at Cornell University and Carnegie Mellon University. The paper, published in the Proceedings of the National Academy of Sciences, uses theoretical economics and game theory to examine how rules change company spending on AI safety.

The researchers found that rules aimed at the wrong part of the AI supply chain, or set at too low a level, can encourage firms to reduce their own safety spending and rely on other companies to absorb the burden. That finding complicates policy debates that often treat regulation as a single lever, rather than a set of obligations that can alter incentives across developers and users of AI systems.

How can weak AI regulation make systems less safe?

The study distinguishes between two groups: companies that build general-purpose AI models, such as OpenAI, Google and Anthropic, and companies that use those models in specific products or services, including medical diagnostics, e-commerce and customer service chatbots.

Application-level regulation can appear attractive because many risks arise when AI is used in a particular setting, Gizmodo reported in its coverage of the study. The researchers found, however, that concentrating obligations on downstream users can change the behavior of model developers. If deployers must ensure that final products satisfy regulatory requirements, model providers may have less reason to invest in measures such as third-party safety audits.

Benjamin Laufer, the study’s principal author, described the effect as free riding. “The regulation acts as a tool for the general provider to offload the safety burden onto the downstream specialist,” Laufer said.

In economic terms, the model examines how companies allocate safety spending when each actor knows that another part of the supply chain also affects the final product. If one firm expects another to make the necessary investment, it may cut its own costs. The study compares that dynamic to the prisoner’s dilemma, a game theory problem in which each participant has a reason to act in its own interest even though coordinated behavior would produce a better collective result.

The researchers define utility as a company’s share of revenue minus investment costs. Their model suggests that strong and well-placed rules can improve both end-product safety and the utility earned by general-purpose AI developers and downstream specialists from their investments.

What does the study imply for U.S. AI policy?

The findings speak to a split in the U.S. approach to AI oversight. Federal policymakers have focused much of their attention on advanced or frontier model developers, including safety testing and national security questions. State-level efforts have more often addressed uses of AI in employment, healthcare, insurance and other high-impact decisions.

The study argues that those layers should be assessed together. Obligations imposed on one part of the supply chain can affect investment decisions elsewhere, including by reducing incentives for another actor to spend on safety.

According to the researchers, the best outcome in their model occurs when regulators require enough safety investment from both model developers and downstream companies. That structure reduces uncertainty over who will pay for safeguards and supports complementary investments across the chain.

The paper also adds detail to the broader U.S. argument over AI rules, where critics warn that heavy regulation could slow innovation and weaken competitiveness with China, while supporters say market incentives alone may not address AI risks. Laufer said effective oversight requires attention to the range of firms involved in building and deploying the technology. “To regulate in a thoughtful way, we need to consider the whole supply chain, not just a single provider or entity,” he said.

This story draws on original reporting from PYMNTS.