Google researchers have developed a new technique called “speculative cascades” designed to make large language models (LLMs) significantly faster, cheaper, and more efficient. But if it isn’t, time is wasted waiting for it to finish, only to then start the large model’s process from scratch. Speculative Cascades: A ‘Best of Both Worlds’ ApproachGoogle’s new method, speculative cascades, offers a hybrid solution that merges these two ideas. This rule dynamically decides, on a token-by-token basis, whether to accept the small model’s draft or defer to the large model. It can also perform a comparative check, deferring if the large model is significantly more confident in a different answer.