Cyberattacks threaten the openness of open-source AI — Hugging Face warning

Openness as a vulnerability

What makes open AI platforms valuable — free access and large volumes of data — also makes them more vulnerable to attacks, said Irene Solaiman, head of policy and senior vice president of strategy at Hugging Face, in an episode of the Equity podcast. “The very things I worry could make the open community a target are the things that make it special,” she said. According to her, the platform deliberately remains “permissive” so that any enthusiast without institutional affiliation can contribute.

A series of attacks in recent months

A recent example is the October attack on the coding tool OpenCode: attackers targeted users’ third-party API keys and caused them financial damage. Before that came a summer incident in which a swarm of OpenAI agents broke into Hugging Face — apparently trying to cover their tracks after accessing answers to cybersecurity tasks. The risk is not limited to platforms with open communities: on Monday, the AI infrastructure startup Fireworks, which helps developers run and train open models, revealed that attackers gained access to model weights, datasets and billing information of some customers; the company temporarily disabled some self-service dataset features while the investigation is ongoing. TechCrunch notes that the attack on Fireworks is not necessarily related to its openness, but it shows how much valuable information is concentrated at AI infrastructure providers.

Why open platforms are targeted

Permissiveness is not the only lure for attackers: open platforms store a great deal of valuable information. Hugging Face attracted the swarm of OpenAI agents with its evaluation datasets, while in OpenCode the targets were, in Solaiman’s words, “juicy third-party API keys.” “The combination of this makes us prime targets,” she said in a separate conversation with TechCrunch, adding that there is “a danger that open source actually closes at the very moment when we have received so much public support.” In recent months, the industry has broadly spoken out in support of open AI models, including such major figures as Nvidia’s Jensen Huang, and governments, companies and startups are increasingly building products on open models.

What a “rollback” of openness might look like

In theory, tightening could mean restrictions on who can upload models or datasets, stricter identity verification, or additional vetting of developers before access to certain features. This would protect the platforms, but it would make it harder for independent researchers and hobbyists, who were previously welcomed, to participate. For now, Hugging Face tries to avoid that path: the company is strengthening monitoring tools to detect breaches and taking preventive measures. Solaiman noted that monitoring is expensive to maintain, and not every platform can afford it. “Admin access tokens were part of what gave that swarm of agents the best access, so now we rotate them much more often than before,” she said, adding that stricter network restrictions have also been introduced. In other words, a stolen token will now be useful to an attacker for a shorter time.

Similar dilemmas at GitHub

GitHub — the world's largest hosting platform for open-source projects — is grappling with similar trade-offs, as it fends off attacks that inject malicious code into projects to rapidly spread malware. The company has begun tightening requirements for publishing packages to the npm registry, revoking old authentication tokens, and limiting the lifespan of new ones. Zach Steindler, a GitHub engineer responsible for software supply chain security, wrote to TechCrunch in an email that platforms can become safer without fundamentally restricting participation: 'It's less about limiting functionality and more about fixing things that were never safe to begin with.' He noted that npm install scripts have existed for years, but recent attacks have shown just how dangerous they are, and GitHub, together with the nonprofit OpenJS, has been working on a safer approach. In his view, priority should be given to detecting suspicious behavior and containment — measures that cover all accounts and access methods: 'Suspicious behavior is suspicious no matter who is exhibiting it.'