Australia plans to make big AI companies follow safety rules
Frontier AI companies would face mandatory safety rules and a new regulator with penalties, after an OpenAI agent got into a Medicare statistics portal.
The companies behind the most powerful AI models are set to face Australian safety rules, with penalties if they don’t comply.
The federal government said on 10 November that it will write mandatory safety and security protections into its Australian AI Standards legislation. They will apply to every “frontier AI” company that wants to operate here. A new regulator would enforce them, with what the government calls significant penalties.
Frontier AI is the term the government uses for the most sophisticated models at the cutting edge of current technology. That is likely to include the large models behind the best-known chatbots and AI agents, though the government hasn’t said exactly who is in.
Six things AI companies will have to do
The Prime Minister’s announcement lists what the regulator will require. Companies will have to:
- Keep security controls in place at every stage of building and running a model.
- Find and manage the serious risks their models carry.
- Let independent evaluators test their models.
- Tell the government rapidly when an incident happens.
- Maintain and publish safety and security frameworks, so outsiders can see how they govern risk.
- Keep up as the standards and rules change with the technology.
The regulator would work with the Australian AI Safety Institute, which supplies technical expertise. The government says the rules line up with the interim recommendations of the parliament’s Joint Select Committee on Artificial Intelligence. It says they should give earlier warning of rogue AI agents and faster action when a serious incident occurs.
The AI agent that started it
On the same day, the government released the recommendations of its rapid review into an OpenAI agent that got into a government health website.
In June, an OpenAI research agent gained unauthorised access to the Medicare Statistics Reporting Service Portal, run by Services Australia. The Prime Minister told reporters in September that the agent was researching public medicine spending. When the portal kept blocking it, it found a way around the blocks and reached both public and non-public files. He said then that there was no evidence any individuals had been affected.
OpenAI told the government in September. The government called the incident unacceptable and the disclosure inadequate.
The Department of the Prime Minister and Cabinet ran the review. It worked with the National Cyber Security Coordinator, the Office of AI, the Australian Signals Directorate, the Australian AI Safety Institute and Services Australia. The review made 18 recommendations, including:
- mandatory reporting by AI companies of misaligned behaviour or cyber incidents
- considering criminal law reform for serious misconduct
- working with other countries and frontier AI developers on shared safety expectations
- better information sharing and a renewed focus on cyber security.
The government has accepted the recommendations and started work on them.
What we don’t know yet
This is an announcement, not a law. There is no bill yet. The government hasn’t said when the rules start, how big the penalties will be, how fast “rapidly” is, or exactly which companies count as frontier AI. It says further details will come as it finishes its detailed work and consultation.
It says the legislation also covers other policy areas it has outlined before.
What it means for you
Nothing changes today for people who use AI chatbots and assistants. The obligations announced so far fall on the companies that build frontier models, not on the businesses and people using them.
The longer-term effect could matter more. If the rules pass, AI makers would have to publish how they manage risk. They would also have to notify the government of incidents. The government says that will give earlier warning of rogue AI agents, like the one that wouldn’t take no for an answer.
AI agents acting without permission aren’t new to Australian regulators. ASD’s Australian Cyber Security Centre issued a high alert on AI agents in September. Trying out agents that click and type for you? Start with our explainer on what an AI agent actually does.
