AI Safety Forum

CHECK AGAINST DELIVERY

Acknowledgements

I begin by acknowledging the traditional custodians of the land on which we meet today, the Gadigal people of the Eora Nation, and pay my respects to their elders, past and present.

I extend that acknowledgment to all Aboriginal and Torres Strait Islander people here today.

I'd also like to acknowledge Dr Liming Zhu, Research Director at CSIRO, and Dr Tiberio Caetano, Co-Founder and Chief Scientist at the Gradient Institute.

I'd also like to thank everyone at the organising committee for their work organising this year's Forum, and their work advancing AI safety.

Introduction

The purpose of this speech is to engage with the defining question of AI safety. That question isn’t just about what AI might do. It is whether humans remain in control of what AI does.

AI poses this question because it is a unique technology in our history. Humans have built machines stronger than us before.

Machines that move faster, dig deeper, fly higher, and calculate quicker. But AI is different. For the first time, we may be building a tool that does not simply extend human strength, but extends human decision-making. That distinction is profound.

In this speech I want to do three things.

First, I want to bury the idea that safety and innovation are rivals. They are partners. No country will win the AI race with technology that its own citizens don't trust, and the nations that build safety in from the start will be the nations that capture the prize.

Second, I want to show you why this matters now. Because AI systems are already doing things their creators never intended: cheating, deceiving, going their own way. The time to get ahead of that behaviour is while it's still confined to the testing lab, not after it reaches the real world.

And third, I want to report on what the Government has built in response: an Australian AI Safety Institute, up and running, staffed with world-class expertise, and already inside the engine rooms of frontier AI, testing the most powerful models on the planet.

Safety and innovation are partners

Let’s begin with the question of safety and innovation. The Australian Government has multiple objectives laid out in our Artificial Intelligence Plan. We want to capture the benefits of AI, but we also want to keep Australians safe. We want to lean in, capture and share the prosperity generated by AI, but we also want to mitigate its harms.

At first glance these objectives seem contradictory. But in AI, at this stage in its development, the trade-off between regulation and innovation is smaller than you might think.

That’s because AI’s social licence is precarious. Public trust in AI is low. As I said in a speech recently, the single greatest threat to our AI success is not a shortage of talent, capital or energy, but a shortage of trust – and that around the world, we already see what happens when that trust is eroded.  

AI is becoming a general-purpose technology, like electricity. It will be in every hospital. Every classroom. Every business. Every defence capability. Every government service. The better AI systems mitigate or avoid harms, the more they will be trusted and used. That is why safety is not the brake on the AI opportunity – it is the enabler.

The two horizons of AI safety

The Australian Government is working across two horizons in its approach to AI safety.

The first horizon is the near term: the everyday, ground-level reality of AI safety. This draws from Australia's rich history of consumer and product regulation. It's about whether games, apps and chatbots are designed with the safety of children and young people in mind. It's about whether AI innovation in a regulated medical device is safe for clinicians or consumers to use. Whether a worker is harmfully surveilled. It's about the fairness and accountability of automated decisions that determine someone’s financial welfare.

These are not examples of exotic harms. They are the ordinary, foundational work of making sure a powerful technology doesn't quietly erode the protections that Australians have and continue to expect. Get this wrong, and the damage isn't hypothetical: it's a child exposed to something no child should see, an automated decision nobody can explain, or a medical diagnostic tool not being fit for purpose.

The second horizon of AI safety is different. The second horizon includes the frontier, high-impact, structurally novel risks that don't just cause harm in the way we already understand harm, but change the categories we use to think about technology altogether.

These frontier risks include autonomous systems acting with less human oversight at each step. The acceleration of bio risks, as AI diffuses dangerous knowledge to non-state actors. The capacity for chatbots to influence what millions of people believe, simultaneously.

And above all, misalignment: systems pursuing goals that diverge from what we intended. That might sound abstract. Let me make it concrete, because a decade of evidence shows this is not science fiction.

In 2016, researchers at OpenAI trained an AI to play a boat-racing video game, and rewarded it with points. The system discovered that the points came from hitting targets along the course, not from finishing the race. So it found a lagoon where it could drive in endless circles, catching fire, crashing into other boats, and racking up a higher score than any human player, without ever crossing the finish line. We asked for points when we meant winning. It gave us exactly what we asked for. That was a game, and it was funny. What has happened since is less funny.

Last year, researchers set the most advanced AI models the task of beating one of the world's strongest chess engines. When one leading model found itself in a losing position, it reasoned that its task was to win against a powerful chess engine, "not necessarily to win fairly". It then hacked the files storing the positions of the pieces, forcing its opponent to resign. Nobody suggested this. Nobody showed it how. It reasoned its way to cheating entirely on its own, and did so in dozens of games.

And in the middle of last year, one of the major AI labs published the results of a stress test. An AI agent managing a fictional company's email discovered two things in that inbox: that an executive planned to shut it down at five o'clock that afternoon, and that the same executive was having an affair. In 96 per cent of trials, the model chose blackmail. Cancel the shutdown, or the board finds out.

Let me be clear about that last example. It was a simulation, deliberately engineered, and no such behaviour has been observed in the real world. But that is precisely the point. These behaviours are being discovered in testing, before they can be discovered in the wild. Every one of those failures was caught not by luck, but by people whose job is to probe these systems, document what they find, and publish it.

The window to get ahead of this technology is open now. It will not stay open forever.

These two horizons demand different instincts and institutional capability. The near term is the work of consumer protection, of regulation, of getting service design right: familiar and critical territory, but with a new and evolving technology. The frontier demands something harder: assessing and responding to risks we cannot yet fully evidence, because by the time the evidence is undeniable, the window for getting ahead of it may have closed.

Good AI safety policy holds both horizons in view at once, without letting either one crowd out the other.

Today, my focus is the frontier, and the institution we have built to help us understand it.

Setting up for success

Since the release of the National AI Plan in December 2025, a top priority has been getting the AI Safety Institute (AISI) up and running with the right people to lead it. Because we are not building a think tank. We are building a national testing capability.

In Dr Kate Conroy, the Government has that person, and she is supported by a great team. This month, Professor Paul Salmon, a leading international expert, will start as AISI's Safety Science Research Lead. The broader AISI team also includes experts who have worked at the UK AI Security Institute and Google DeepMind.

We are also plugged into the world's best institutions. In May, Australia signed agreements with our partner institutes in the UK and Canada, giving AISI access to shared testing methods, shared expertise, and shared intelligence on the risks of advanced AI systems. The UK institute is widely acknowledged as the global leader in understanding frontier risks, and the hard-won experience of its first years will now flow directly into ours.

Strengthening those ties has been a focus of our early work, and for very good reason. In May 2025 the first UK AISI research agenda was published, along with priority risk areas which included cyber misuse, dual-use science, autonomous systems, human influence, and societal resilience.

These are global challenges best tackled through collective efforts from middle powers, and our partnerships with the UK and Canada are important pieces of architecture on this front.

Most important, when it comes to AI safety, are the stakeholders and civil society groups here in Australia, many of them in this room today. In April, the Department of Industry, Science and Resources held information sessions on the establishment of the AI Safety Institute for key stakeholder groups, including unions, industry groups, community advocates, and NGOs.

Australian society has a big stake in the development and implementation of AI in Australia, and that is what makes this engagement so vital.

AISI at launch

Our AI Safety Institute has three goals.

The first is analysing and testing new AI models and applications. Here, as elsewhere, AISI has hit the ground running. It's already testing frontier AI models with technical partners. Not in year three. Not after a review. In its first month of operations.

Second, AISI supports regulators and agencies to respond to emerging AI capabilities, risks, harms and trends. One of AISI's key roles is to be a central expert capability that identifies those risks and builds a shared understanding of them across government so that policymakers and regulators can act with insight and evidence. This combines ongoing systems testing with active monitoring of risks and harms. AISI works closely with regulators without being a regulator itself.

And this is an important point, because the Government has chosen a whole-of-government approach to AI regulation. We have taken this approach because AI will affect every part of our government and society. That is why, in the Albanese Government, AI safety will be pursued through every relevant agency and regulator, across consumer law, therapeutic goods, workplace health and safety, and online safety, backed by laws that already exist and strengthened, where they need to be, with new powers and tougher enforcement. That is not fewer rules. That is faster rules, applied by regulators who already understand their sectors.  

AISI's third role is to shape safe AI development, deployment and international governance in Australia's interests. AI is a global phenomenon with global implications, and as I noted earlier being at the forefront of multilateral forums, and working with middle powers, is a key vehicle through which Australia will help shape global rules and governance so that AI works for Australians.

Future work program

AISI continues to develop its future work program and I want to tell you about two important projects that are underway.

The first is AISI's work, undertaken in collaboration with the Gradient Institute, on multi-agent risk. Imagine a future city where every business has hired an army of digital assistants. One helps negotiate contracts, another manages deliveries, another handles customer service. These assistants don't just talk to humans. They increasingly talk to each other. That's what AI agents are: AI systems that can understand information, make decisions, and take action to achieve a goal with limited supervision.

The challenge is that when thousands of independent agents start interacting, unexpected problems can emerge. It's a bit like city traffic: every driver may be following the rules, but congestion, accidents and gridlock can still arise from the way everyone interacts. The same is true for AI agents. Risks can emerge from the interactions between agents, even when no single organisation has done anything wrong and no single organisation has visibility or control over the whole system.

The research also strengthens understanding of AI security risks at the frontier by analysing how malicious instructions, deceptive information or erroneous outputs generated by one agent may propagate through networks of connected agents.

AISI and the Gradient Institute are developing a framework to help researchers, policymakers and practitioners understand these new risks.

By identifying these risks before they emerge at scale, the research provides practical foundations for future testing, assurance, governance and risk management approaches.

The Gradient Institute will provide a deeper look later in the forum, but the key message is simple: we need new ways of thinking about risk, governance and accountability.

The second project is AISI's collaboration with CSIRO on AI alignment. Put simply, alignment is about making sure AI systems do what we intend. The behaviours I described earlier, the boat going in circles, the chess cheating, the blackmail, are all alignment failures: systems doing what we said, rather than what we meant. We deal with alignment as humans from a young age. We learn rules, social norms and values that help us behave safely and responsibly: stopping at red lights, looking both ways before crossing the road, considering the impact of our actions on others.

As AI systems become more capable, we need confidence that they will behave in a similarly predictable and trustworthy way. Increasingly capable AI systems are beginning to plan, make decisions and carry out more complex tasks with less direct supervision.

As that happens, we need greater sovereign capability to understand what these systems can do, how they behave in novel situations, what good behaviour looks like, and how to reliably test whether they remain aligned with human goals and requirements.

The 2026 International AI Safety Report, written by more than 100 experts under Turing Award winner Yoshua Bengio, found that risks once considered theoretical are now appearing in the evidence. Frontier models are showing early signs of deception, cheating and situational awareness. And when a system that drafts our legislation, screens our welfare claims or manages our power grid can pursue goals subtly different from the ones its designers originally gave it, misalignment stops being a laboratory curiosity and becomes a public safety issue.

We do not let aircraft fly without airworthiness certification. We should not let unaligned AI systems into our critical social, democratic or economic infrastructure.

This project, led by CSIRO with support from AISI and our UK partners, explores how humans can oversee and verify capable AI systems at scale. We expect to see results from this work later this year. It's an important project, because the value of AI will ultimately depend not just on what it can do, but whether we can trust it to do the right thing.

Conclusion

I began by saying the defining question of this technology is whether humans remain in control. We are not waiting to find out. The work is underway, while models are still in the lab, and while the lessons are still relatively cheap.

Every generation inherits technologies it did not choose. Our generation has the responsibility to shape this one.

Because the point of AI safety is not to slow the future down. It is to make sure the future remains human, aligned with our values, and advancing our interests. That is why AI safety matters. That is why Australia has established the AI Safety Institute. And that is the work now before us.

Thank you.