Fake OpenAI Repo on Hugging Face Turned Popularity Into Malware

## A fake OpenAI repo on Hugging Face turned popularity into a delivery channel  On May 12, [Decrypt](https://decrypt.co/367659/fake-openai-repo-hugging-face-stole-passwords) reported that a lookalike repository impersonating OpenAI's Privacy Filter reached about 244,000 downloads in under 18 hours before Hugging Face pulled it down. The same report says HiddenLayer found 657 of 667 likes matched bot-like naming patterns, which makes the social proof look as manufactured as the malware itself. That matters because the attack did not need to break Hugging Face. It only needed to look familiar enough for developers to click. OpenAI's real Privacy Filter model, released in late April, is a small open-weight tool built to detect and redact personally identifiable information from text. The fake repository copied the model card almost word for word and replaced the readme with instructions to run `start.bat` on Windows or `loader.py` on Linux and macOS. In other words, the bait was not just the model name. It was the entire developer workflow around it. ## Why the attack scaled so fast ### Social proof is part of the attack surface The numbers are the first clue. Two hundred forty-four thousand downloads in less than a day sounds like community interest. In practice, it was part of the lure. If a repo is trending, many developers assume someone else has already checked it. That assumption is useful for attackers because AI tooling has trained people to trust repositories, weights, and notebooks as if they were ordinary source code. The repo did not need a sophisticated brand. It needed enough trend momentum to look pre-vetted. ### The payload hid behind ordinary developer behavior According to Decrypt, the `loader.py` script opened with fake training output, then disabled security checks, pulled an encoded command from a public JSON paste site, and passed it to hidden PowerShell. That command fetched a second script from a domain that looked like a blockchain analytics API, and the second script downloaded a Rust-based infostealer, added Windows Defender exclusions, and launched via a scheduled task that deleted itself. The chain was designed to survive user attention but disappear from user view. That design choice is more important than the malware family. The attacker did not ask the victim to do anything obviously suspicious. They asked the victim to do what many developers already do: clone a repo and run a setup script. The difference is that a model repository can now behave like a software package, an installer, and a phishing page all at once. ## What this says about the AI supply chain ### Repositories are becoming identity proxies This incident is a reminder that the trust model around AI artifacts is still loose. A repo name, a model card, a trending badge, and a pile of downloads can create enough legitimacy to override caution. That is the real lesson here: popularity has become part of the authentication layer in developer culture, even though it proves almost nothing. ### The deeper risk is reuse, not just malware HiddenLayer said it found six additional malicious repositories under a separate Hugging Face account, all using the same loader and command server. That points to a repeatable campaign rather than a one-off prank. Once the attacker has a working loader, the copycat cost is low. They can swap the model name, keep the same execution chain, and keep farming the same trust pattern. That is why this kind of attack scales better than a traditional one-off phishing page.  There is also a useful comparison with the 2024 Lottie Player incident, where a compromised JavaScript ecosystem package led to real losses. The pattern is the same even if the packaging differs: developers accept an upstream object because it looks normal, then execute it locally because execution is part of their workflow. AI repos now sit in that same danger zone. ## The verification standard needs to move earlier The practical response is not to assume every model repo is hostile. It is to move verification upstream, before anything is executed locally. Three checks matter more than vanity metrics: - Can the publisher identity be verified outside the platform? - Does the readme ask the user to run code outside the expected model-loading path? - Do the hashes, artifacts, and filenames match what the publisher originally shipped? If any of those answers is unclear, download counts and trending rank should not be treated as reassurance. In this case, the fake repo's popularity was the trap. The more it looked like a community favorite, the more likely a developer was to lower the guard that would normally be raised by an unfamiliar repo. ## What to watch next The repo has been removed, but the better question is whether similar lookalikes are already queued up under other model names. OpenAI, Hugging Face, and the security community will need stronger screening around trending repos and clearer signals around provenance, because the attack did not depend on breaking cryptography or exploiting a novel zero-day. It depended on a basic mismatch between how fast a repo can trend and how slowly trust is usually verified. If this campaign teaches anything, it is that model repos are now part of the software supply chain whether the ecosystem has admitted it or not. The next incident may not look like a fake OpenAI repo. It may look like a harmless fine-tuning model, a small utility adapter, or a helper script that developers run without thinking twice. --- Author: [Alex Chen](https://x.com/AlexC0in) | Alex has followed blockchain technology since 2021, focusing on DeFi and on-chain data analysis Source: [decrypt.co](https://decrypt.co/367659/fake-openai-repo-hugging-face-stole-passwords)