Google Gemma 4: Open AI model uses speculative decoding and gets up to 3x faster

Google has just launched Gemma 4 models with an innovative technique called speculative decoding, which promises to accelerate text generation by up to three times in relation to traditional methods. The approach, named Multi-Token Prediction, allows the model to predict several tokens at once, drastically reducing inference time without sacrificing quality.

What is speculative decoding on Gemma 4?

Speculative decoding is a method that combines a lighter “drake” model with the main model to generate tokens in parallel. While the main model validates predictions, the draft advances rapidly, resulting in speed gains of up to 3x in benchmarks released by Google. The technique was already known in academic research, but this is the first time it has been implemented in a large open model such as Gemma.

Why does Gemma 4 matter to Brazilian developers?

For the AI community in Brazil, Gemma 4 represents an opportunity to execute powerful models with fewer computational resources. With speculative decoding, tasks such as chatbots, summarization, and code generation can be executed locally on more modest hardware or on cloud at reduced costs. Companies using generating AI can reduce latency in customer service applications, for example, without having to invest in expensive GPUs.

How does Multi-Token Prediction work in Gemma 4?

Instead of predicting one token at a time, the model is trained to predict token sequences simultaneously. During inference, the model generates a draft of several tokens, which is checked in parallel. If verification fails at any point, the system only goes back to that token, maintaining accuracy. This reduces the number of sequential steps and accelerates generation.

What are the benchmarks released from Gemma 4?

According to the source Ars Technicala, tests indicate gains of up to 3x in long text generation tasks. In shorter tasks, the gain is lower, but still significant. The model maintains quality equivalent to traditional methods.

Is Gemma 4 really an open model?

Yes, Google launched Gemma 4 under an open license, allowing developers and researchers to download, modify and deploy the weights of the model. This includes the version with speculative decoding as an experimental resource. The decision contrasts with other great players who keep their models closed, expanding access to cutting-edge AI.

What is the impact of Gemma 4 on the AI ecosystem?

Open models like Gemma 4 can democratize the development of AI applications in Brazil. With the acceleration provided by speculative decoding, startups can create more responsive products. In addition, the technique can be applied to other models, inspiring more research on efficiency. For comparative contexts, see also NVIDIA and OpenAI announce IA infrastructure megaproject: the largest in history.

Frequently Asked Questions about Gemma 4

Does speculative decoding affect the quality of the generated text?

No, speculative decoding is designed to maintain the same quality as the main model validates draft predictions, correcting errors before the final output.

Which Gemma 4 family models support Multi-Token Prediction?

The feature is available as experimental in all sizes of Gemma 4, including variants 2B, 7B and 27B, according to Google's ad.

Is it possible to use Gemma 4 with speculative decoding in local hardware?

Yes, the technique has been optimized for running on GPUs and CPUs, allowing you to run on consumer hardware such as an RTX card, with noticeable speed gains in long generation tasks.