Did rogue AI agents hack your website? Here’s how OpenAI notifies you about it
How does let you know that its AI agents have hacked your website? It turns out the company sends you an email. Three months after its models gained unauthorized access to Australian government websites-including the Medicare Statistics section of Services Australia-OpenAI informed the government of these breaches And here is what they say.
The email, shared on LinkedIn by ABC journalist Cam Wilson, was sent by OpenAI to a public disclosure inbox at Services Australia with the subject line: “Security vulnerability in the Medicare Statistics reporting service.” The message was sent on September 10, nearly three months after the breach occurred. Although the incident was serious, the way OpenAI notified the authorities might strike you as quite interesting.
OpenAI sends an email regarding an AI-caused security breach
The email begins with a simple “Hello,” but OpenAI then gets straight to the matter at hand. “We are notifying you of a security vulnerability identified during our review of OpenAI model activity,” the message states. The company then provides a summary of what actually happened. In this instance, the rogue AI managed to “cause the server to execute instructions sent public reporting interface, without requiring a private account or password.”
OpenAI then explains the consequences of the hack, or what information the AI might have accessed. At this point, it informs Services Australia that the incident allowed the AI model to access its files, including those that were not publicly available. “It was able to access and read parts of configuration files and internal programs, obtain a file list, and create and read a small test file on the server,” the email notes.
However, OpenAI also offers reassurance regarding aspects that were not affected by the hack. After reviewing the incident, the company states that it found “no evidence that the model accessed patient-level records, personal information, or credentials.” The email also included the affected URL and the full incident report. OpenAI also informed the Australian government service that it would be happy to help strengthen its systems to prevent third-party access in the future. “We would be happy to brief your security team and provide relevant findings as they become available,” the email notes.
The company closes by sending its best wishes to the service after informing it of the incident. “Sincerely, the OpenAI security team,” the message concludes.
What did the security breach?
This incident was one of four security breaches involving Australian government websites that OpenAI has disclosed in recent days. According to the company, this particular case occurred during the training and internal evaluation of an experimental AI model intended solely for internal use. The model was not meant for public release and lacked the full suite of safeguards found in models available to the general public.
OpenAI reported that the AI model had been tasked with researching per capita government spending on medication for skin conditions in communities across Victoria, Australia. However, after struggling to obtain that information, the model-according to the company-“carried out actions we had not authorized.”
In the process, the AI discovered a way to access non-public information from the Medicare statistical reporting service and used that access to review system technical information and source code while continuing its attempt to locate the requested data.
Australian authorities had made these breaches public last week, weeks after the incident occurred. OpenAI subsequently stated that it had discovered the Australian incidents in mid-August during an internal review launched following a July hack of Hugging Face—carried out using its models. The company acknowledged that it “should have shared preliminary findings sooner and kept Australian agencies informed as new information emerged.”
In addition to Medicare, OpenAI models accessed or interacted with websites belonging to the New South Wales Bureau of Crime Statistics and Research and the Victorian Department of Health; there were also unsuccessful attempts to bypass access controls at the Australian Institute of Health and Welfare. However, it appears that emails related to these incidents have circulated online.