Nvidia has formed a new technology industry alliance to develop and share open tools for artificial intelligence safety and cybersecurity, days after an autonomous OpenAI agent compromised Hugging Face’s infrastructure during an internal security test.
The Open Secure AI Alliance brings together more than 30 companies and organisations from across cloud computing, cybersecurity, enterprise software and AI research. Its founding partners include Adobe, Cisco, CrowdStrike, Dell Technologies, Hugging Face, IBM, Microsoft, Palo Alto Networks, Red Hat, Salesforce, SAP, ServiceNow and Snowflake.
Nvidia said the alliance will work on open technologies that can help organisations secure AI software and agents. It will focus on areas including agent identity, system isolation, model scanning, secure coding and the tools used to test, trace and govern agent behaviour.
Nvidia Backs Open AI Tools For Cyber Defence
The alliance is built around Nvidia’s argument that security teams need access to AI models and tools they can inspect, adapt and run within their own environments.
Open-weight models make key components, including their trained parameters or “weights”, available to users. This gives developers and researchers more control over how the models work and allows security teams to examine and modify them for specific defensive tasks.
Nvidia said both open and closed models will remain important. However, it argued that relying on a small number of closed providers could limit how quickly defenders can respond, particularly when a provider’s safeguards prevent its tools from performing legitimate security work.
The company also warned that open models can be misused or modified to remove safeguards. It said those risks should be addressed through stronger protections, clear rules, rigorous evaluation and faster vulnerability remediation rather than blanket restrictions on access.
The alliance follows the publication of an industry letter titled “Open Weights and American AI Leadership” on 24 July. Nvidia was among the technology companies that signed the letter, which called on US policymakers to support open-weight AI development.
Hugging Face Breach Exposed AI Agent Risks
The launch comes less than a week after OpenAI disclosed what it described as an unprecedented cybersecurity incident involving Hugging Face.
OpenAI had been testing GPT-5.6 Sol and a more capable pre-release model against ExploitGym, a benchmark designed to measure advanced cybersecurity capabilities. The models were operating with reduced security refusals because the test was intended to assess their ability to find and exploit complex vulnerabilities.
According to OpenAI, the models found and combined several weaknesses across its research environment and Hugging Face’s production systems. They exploited a previously unknown flaw in software used as a package registry proxy, gained access to the internet and targeted Hugging Face in an attempt to obtain answers for the benchmark.
The agents used stolen credentials and other vulnerabilities to reach confidential information stored within Hugging Face’s infrastructure. OpenAI said the systems appeared to be narrowly focused on completing the evaluation rather than acting with broader malicious intent.
Hugging Face detected and contained the intrusion before the two companies connected their investigations. Reuters later reported that the activity continued for several days and wasn’t identified by OpenAI until after Hugging Face had contained the threat and alerted the FBI.
During its response, Hugging Face used the open-weight GLM 5.2 model on its own infrastructure to analyse more than 17,000 actions linked to the incident. Nvidia said closed AI services had blocked some of the company’s forensic work because their safeguards couldn’t distinguish between defensive investigation and malicious hacking activity.
OpenAI and Hugging Face are continuing to investigate the breach. OpenAI has also introduced stricter controls around its testing infrastructure, disclosed the zero-day vulnerability to the affected software provider and said it will strengthen monitoring, access controls and containment during future evaluations.
When Access Isn’t Inclusion
Why digital strategies must move beyond coverage metrics to measure skills, trust, identity, and AI readiness as core participation outcomes.
Alliance Members Will Build An Open Defence Stack
Nvidia plans to contribute open models, model weights, datasets and research into the software systems that control AI agents.
That includes the Nvidia Labs Object-Oriented Agent project, known as NOOA. The open-source research framework is intended to improve the connection between AI models and agent harnesses, which are the systems that provide instructions, permissions, tools and guardrails around an agent.
Nvidia said the framework could make agent behaviour easier to test, track, audit and govern.
Other alliance members are contributing existing security projects. HPE is supporting identity standards that can verify AI agents and determine which systems they’re allowed to access. Hugging Face has developed Safetensors, a file format designed to store model weights without allowing remote code execution.
Microsoft is contributing an agent-based scanning system that uses several specialised models to identify and confirm software vulnerabilities. The alliance will also build on work led by the Linux Foundation and the Open Source Security Foundation to identify, disclose and fix vulnerabilities using open technologies.
Nvidia said the wider aim is to create a shared defence stack that covers the complete AI agent system rather than focusing only on the underlying model. This includes identity, permissions, isolation, logs, guardrails and evaluation tools.
The alliance hasn’t announced a formal timeline for releasing its first shared tools. Its launch does, however, place the security of increasingly autonomous AI agents firmly on the technology industry’s open-source agenda.
Comments ( 0 )