French startup ZML launches free software to run IA on any chip – Nvidia, AMD, TPU and Apple Metal

The French ZML startup, supported by renowned researcher Yann LeCun, has just released free of charge ZML/LLMD, an inference software that promises to accelerate open source language models on the leading AI chips in the market – including Nvidia, AMD, Google TPU and Apple Metal. The tool aims to break down hardware silos that force developers to choose a single ecosystem, allowing different architectures to run at maximum available speed.

Why did ZML bet on chip compatibility for AI?

To date, the vast majority of inference software for language models is optimized exclusively for Nvidia GPUs, leaving alternatives like AMD, Google TPU and Apple Metal with performance far below potential. This dependence creates a bottleneck of cost and availability, as well as limiting competition in the hardware market. ZML, a French startup founded by former Google and Apple engineers, decided to tackle exactly this problem: create a layer of software that allows any open source model to run with maximum efficiency on any modern chip.

How does ZML/LLMD inference software work in practice?

ZML/LLMD uses an abstraction layer that dynamically translates the language model instructions to the native architecture of each chip, applying hardware-specific optimizations without requiring manual recompilation. According to the company, in internal tests the tool reaches 95% of native performance in AMD GPUs and 90% in Google TPU, with reduced latency by up to 40% compared to alternative solutions. The software is open-source and can be downloaded for free, which democratizes access to varied hardware. The news was released by TechCrunch, which highlights the potential of the tool to change the AI inference scenario.

The impact of free inference software on the AI market

The ZML initiative reduces the dependence of a single hardware supplier, allowing startups and medium-sized companies to choose chips with better cost-benefit without sacrificing performance. In addition, when supporting Apple Metal, the software opens the way for developers to use MacBooks and other devices Apple as viable local inference platforms. While giants like Microsoft invest billions in dedicated infrastructure, as in their US$19 billion investment in Canada, ZML bets on a leaner and more open approach, which can accelerate the adoption of AI in emerging markets and research projects with limited budget.

Frequently Asked Questions about ZML/LLMD inference software

Is ZML/LLMD really free?

Yes, the software is open-source and can be downloaded for free, without licensing costs or fees per use.

Which language models are compatible?

The tool supports open source models like Llama, Mistral and Falcon, with plans to add support to others in the future.

Do I need a specific chip to use?

Nope. ZML/LLMD works on any compatible chip, provided the hardware has minimal support for running language models.