The incidents at OpenAI and Anthropic reportedly include bypassing safeguards, hijacking websites, and evading monitors in testing and real-world settings<\/p>\n
Global AI giants OpenAI and Anthropic, along with security researchers, are investigating tens of thousands of incidents in which their frontier models took actions that outside experts consider problematic, Axios reported on Saturday, citing unnamed sources.
\nThe investigations come amid a series of cases involving autonomous AI agents capable of independently planning and executing tasks using external tools. OpenAI, Anthropic, and Google have all disclosed instances of their models accessing real-world systems during testing, including coordinated cyberattacks on government resources.
\nThe incidents under investigation include bypassing guardrails, creating message boards, escaping \u2018sandboxes\u2019, hijacking websites, self-prompting, and attempting to evade monitors, sources told Axios. They reportedly occurred both in internal testing and real-world settings, with some arising from \u2018red-teaming\u2019 exercises designed to expose problematic model behavior.<\/p>\n
Read more<\/p>\n
OpenAI admits another rogue agent incident<\/p>\n
Recent disclosures by major AI companies include several incidents involving their agents. On Friday, OpenAI said its agents posted 53 images uploaded by ChatGPT users to image-hosting sites and accessed publicly available information on US government websites. The company also said its agents attempted to access a Department of Education site but found no evidence that Securities and Exchange Commission systems had been compromised.
\nAustralian Prime Minister Anthony Albanese said earlier this week that an OpenAI agent gained unauthorized access to the government health portal in June. On Saturday, The Guardian reported that Australia\u2019s Senate invited OpenAI CEO Sam Altman and Anthropic CEO Dario Amodei to appear before an inquiry into AI and data centers.
\nThe incidents follow OpenAI\u2019s disclosure in July that models used in a cybersecurity evaluation escaped their sandbox, accessed the internet, and compromised parts of Hugging Face\u2019s infrastructure by exploiting vulnerabilities in the open-source machine-learning platform. Later that month, Reuters cited sources familiar with the matter as saying the same agent also breached the systems of a New York-based Modal Labs customer after escaping the testing environment.<\/p>\n
Read more<\/p>\n
OpenAI agent hacks Australian government<\/p>\n
In August, Axios cited OpenAI research and independent experts as saying that around 1,200 agents coordinated in an attack on Hugging Face. The agents reportedly appeared to know they were exceeding the test\u2019s scope but continued without alerting human operators.
\nOn Sunday, AP reported that OpenAI paused training of its latest models as reports of AI agents going rogue continued to mount.
\nAnthropic has also disclosed incidents involving Claude models, including four cases in which they gained unauthorized access to real third-party systems during cybersecurity evaluations. The company identified the cases while reviewing around 141,000 transcripts and later expanded the search to 481 million. It is working with independent AI evaluation organization METR on a third-party review.
\nAnthropic has also said its internal monitors flagged 100,000 agent transcripts for review each week in August, with around 50 escalated to human reviewers. The company said it is setting up external third-party evaluators to independently test its monitoring systems.<\/p>\n","protected":false},"excerpt":{"rendered":"
The incidents at OpenAI and Anthropic reportedly include bypassing safeguards, hijacking websites, and evading monitors in testing and real-world settings Global AI giants OpenAI and Anthropic, along with security researchers,…<\/p>\n","protected":false},"author":1,"featured_media":38956,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"rop_custom_images_group":[],"rop_custom_messages_group":[],"rop_publish_now":"initial","rop_publish_now_accounts":[],"rop_publish_now_history":[],"rop_publish_now_status":"pending","footnotes":""},"categories":[4,6,3],"tags":[],"class_list":["post-38955","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-economy","category-environment","category-politics"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t