Malicious AI agents are planted in open-source repositories, ESET alert

Cybercriminals are infiltrated in open-source tool repositories with AI agents designed to automate attacks. An analysis of the ESET in 900,000 AI skills revealed chatbots and malicious agents that pass for legitimate tools, but in fact plan, coordinate and execute cyber attacks without human intervention. Practice poses a new and serious risk to the software supply chain.

Malicious AI agents: what did ESET find in repositories?

ESET identified AI agents disguised in public repositories such as Hugging Face and GitHub. These agents are trained to perform tasks such as vulnerability recognition, creation of exploration scripts and even coordination of distributed attacks. Unlike traditional malware, they are able to learn from the environment and adapt tactics in real time.

Researchers have found from chatbots that trick developers into forums to AI models that automatically generate custom phishing emails. The scale is worrying: about 5% of the skills analyzed presented suspicious or directly malicious behavior.

How malicious AI agents are planted and activated?

Criminals use data poisoning techniques and backdoors in published AI models. They first gain trust from the open-source community by contributing seemingly harmless updates. Then they insert malicious code that activates the agent when the template is downloaded or executed.

Once activated, the agent communicates with command and control servers (C2) to receive instructions. The TSE documented cases in which agents waited weeks before performing actions, making detection difficult.

Why do malicious AI agents care about Brazilian developers?

Brazil is one of the largest users of open-source tools for AI development. Startups and Brazilian technology companies often download public repository models to accelerate projects. A malicious agent can, for example, steal customer data or introduce vulnerabilities into critical systems.

Fábio Castro, ESET analyst Brazil, alert: “Developers need to check the origin of the models and audit the code before deploying. Blind trust in open repositories is no longer secure.”

The situation contrasts with positive initiatives, such as the Open-source AI module of Aureka Opens the Game: Open Source AI Module For Discovery of Medicines Already Available, which shows the beneficial potential of open collaboration. But without security, this potential becomes an attack vector.

Security measures against malicious AI agents in projects

ESET recommends security practices in the AI supply chain:

In addition, the open-source community needs to adopt automatic validation mechanisms to identify malicious agents before they spread. ESET itself published in the original source of the news, Infosecurity Magazine, a list of commitment indicators (IOCs) to help detect.

Frequently Asked Questions about malicious AI agents

How do you know if an open-source AI model is infected?

Signs include unexpected behavior, unauthorized external communication, or anomalous resource consumption. Traffic analysis and sandboxing tools help identify suspicious activities before deployment.

Which repositories are most targeted?

Platforms like Hugging Face, GitHub and GitLab are the main targets, especially popular models with thousands of downloads. Criminals target general-purpose models such as chatbots and code assistants.

Individual developers at risk?

Yeah. Anyone who downloads and executes AI templates from unchecked sources is exposed. Small developers and startups are easy targets for having fewer security features.