Frontier AI and the question of governance

By Andrew Bailey.

Much has been written about the risks posed by the rapid advance of frontier artificial intelligence and the need for some form of regulation. The risks are real and increasingly significant. But we should not automatically turn to the question of regulation. Instead, we should first ask a more fundamental question: what precisely is the problem we are trying to solve?

Frontier AI combines two powerful capabilities. The first is its ability to capture and synthesise an immense stock of accumulated human knowledge. That is not entirely new – the internet has already transformed our ability to access and organise information on a vast scale.

The second is different and novel. Through recursive learning, these systems increasingly use their own outputs to refine and improve their performance. They learn not only from human-generated information but from the products of their own reasoning. The result is a feedback process that can become increasingly self-referential. Without effective means of intervention, this process resembles a closed loop in which the model progressively governs itself.

Many proposals for regulation implicitly assume that society can intervene in this process when necessary. But before we decide who should intervene, or how and when they should do so, we must establish why intervention is needed.

The answer, I would argue, lies in a principle that sits at the heart of every functioning society. Human beings do not live as isolated actors pursuing their individual objectives without constraint. We operate within a framework of shared norms, obligations and responsibilities. We accept that there are points where individual action must be moderated in the interests of society as a whole.

This is, in no small part, why governments exist. The 19th-century philosopher TH Green captured the point succinctly when he observed that no individual can make a conscience for himself, he always needs a society to make it for him.^{footnote [1]} Individual freedom and social responsibility are not opposing concepts. They are mutually dependent.

The challenge posed by frontier AI is that, in its most advanced forms, it threatens to operate outside this framework. A sufficiently powerful system functioning within a self-reinforcing loop risks reducing society’s ability to exercise meaningful oversight and intervention. The greater the capability of the system, the more important this question becomes. That is why the issue has acquired such urgency.

None of this implies that we should prohibit frontier AI. On the contrary, the potential benefits are immense. These technologies could drive scientific discovery, enhance productivity and contribute substantially to economic prosperity.

But if we are to realise those benefits safely, we must answer one critical question: should society retain the ability to intervene, to establish the boundaries within which these systems operate and to revise those boundaries as the technology evolves?

To my mind, the answer is unequivocally yes. The question may appear self-evident. Yet in the excitement surrounding AI development, there is a risk that we move too quickly to debates about regulatory architecture before establishing where the failure exists.

Once that has been established, attention can turn to the practical challenge. How do we make sure that intervention is possible and effective? A sensible starting point is rigorous model testing, conducted before and after deployment. Such testing is essential if we are to understand the behaviour of increasingly complex systems, identify vulnerabilities and establish confidence in the safeguards that are intended to contain them. It should also form an important part of appropriate standards for deploying these models.

Important work in this area is already underway. The UK has made a strong start through the establishment of the AI Security Institute, which is helping to develop the scientific and technical foundations of AI assurance. But the pace of progress must accelerate. The capabilities of frontier models are advancing quickly and our understanding of them needs to keep pace, especially as they are put to use outside controlled test environments.

We should also proceed with a degree of humility. Testing will not eliminate failures. Models will behave unexpectedly. They will expose weaknesses in our assumptions. That is not evidence that testing has failed, it is evidence of why testing is necessary. We will learn through experience and adaptation. The same approach should apply to learning from incidents, including near misses, when the models are in development, deployed and in use.

One should not see testing as an alternative to future regulation, nor as a complete solution. Over time, a more formal regulatory framework may well emerge. But regulation is not, in my view, the right place to start. Understanding, testing and establishing credible points of intervention must come first.

Why should this matter to a central bank governor? Because frontier AI has important implications for financial stability. Most immediately, it materially increases the scale and sophistication of cyber threats facing the financial system. We can no longer consider the resilience of payments networks, financial market infrastructures, banks and other critical institutions separately from AI advances. These firms will be using AI within their own operations. Rigorously testing the capabilities and features of the models they use will help us, collectively, to understand how best to deploy AI safely, whether for cyber defence or for agentic trading and payments. In time, that understanding could be codified into a set of standards that will help to ensure consistency in approach across the financial system and perhaps more broadly across the economy.

Central banks have a responsibility to safeguard the stability of the whole system. We cannot stand aside and assume that technological progress will resolve these questions on its own. The public interest requires that we engage with them now, before the risks become more difficult to contain.

The challenge before us is, therefore, not whether to embrace AI. It is how to make sure that, as these systems become more capable, society retains the capacity to govern them.