Cisco releases open source tool that reveals the origin of AI models – understand the impact

Cisco introduced Model Provence Kit, an open source tool that helps organizations track the origin of third-party AI models from training to deployment. The goal is to strengthen the integrity of the AI supply chain and accelerate response to security incidents. The kit is already available at GitHub and promises to be a critical ally for security and compliance teams.

What is Model Provence AI Model Origin Kit?

The Model Provence Kit is an open source tool developed by Cisco that allows you to verify and document the origin of artificial intelligence models. It collects metadata on the model’s life cycle — such as used datasets, versions, training settings, and digital signatures — and stores them unchangingly. With this, companies can audit whether a model has undergone malicious changes, such as data poisoning, or whether it meets regulatory requirements such as the EU AI Act.

Why is the provenance of AI models essential for safety?

Third party AI models are increasingly used in critical applications, but their supply chain is rarely transparent. An attacker can inject corrupted data during training or replace a legitimate model with an adulterated version. According to Cisco, the kit allows tracking each step from the original dataset to the deploy, creating an auditable record. In a scenario where vulnerabilities in AI libraries — such as Critical Vulnerability in LiteLLM is Exploited Soon After Public Disclosure — are quickly exploited, having a provenance trail is essential to contain damage.

How does the kit mitigate risks like data poisoning in AI models?

The tool uses hash techniques and digital signatures to create a model’s “care chain”. Each component — dataset, training script, checkpoint — is hashed and linked to the next. If a dataset is contaminated, the hash will not match, and the kit alerts automatically. In addition, it integrates with IC/CD systems and model repositories such as Hugging Face, allowing continuous verification. For companies operating in regulated sectors, such as health and finance, this reduces the risk of sanctions for the use of non-compliant models.

What is the impact of the kit for developers and companies in Brazil?

In Brazil, the adoption of corporate AI grows rapidly, but the lack of transparency in the origin of the models is a bottleneck. The Model Provence Kit can be used by MLOps teams to validate models before production, without licensing cost — as it is open source. It also facilitates response to incidents: in case of suspected attack, it is possible to quickly isolate the compromised model and track its origin. Cisco has provided documentation and examples in the official repository, with Python support and common formats like ONNX and TensorFlow.

FAQ: questions about the origin of AI models and Cisco kit

Is the kit really free?

Yes, Model Provence Kit is fully open source and is available on Cisco GitHub without licensing costs or commercial usage restrictions.

What types of AI models does it support?

The tool supports templates in the most common formats such as TensorFlow, PyTorch and ONNX, and can be extended to other frameworks through plugins.

How to start using Model Provence Kit?

Simply access the official Cisco repository in GitHub, follow the installation instructions via pip and integrate the kit into the existing MLOps pipeline. No special infrastructure required.

Source: SecurityWeek