Introducing Turbo: 5x faster than Yarn & NPM, and runs natively in-browser 🔥
Note: This is part of a talk I’m giving at Google’s Mountain View campus on Wednesday, December 6th — come join!
After four months of hard work, I’m excited to finally announce Turbo! 🎉
Turbo is a blazing fast NPM client originally built for StackBlitz that:
- Installs packages ≥5x faster than Yarn & NPM 🔥
- Reduces the size of
node_modulesup to two orders of magnitude 😮 - Has multiple layers of redundancy for production grade reliability 💪
- Works entirely within your web browser, enabling lightning fast dev environments ⚡️
Why?
When we first started working on StackBlitz, our goal was to create an online IDE that gave you the same feeling as being behind the wheel of a supercar: that giggly delight of receiving instantaneous responses to your every command.
This contrasts with the experience NPM & Yarn can provide locally. Since they’re designed to handle massive dependencies backend codebases require for native binaries & other assets, their install process requires something more akin to a
semi-truck than a supercar. But frontend code rarely relies on such massive dependencies, so what’s the problem? Well, those dependencies still slip into the install process as dev & sub-dependencies and are downloaded & extracted all
the same, resulting in the infamous black hole known as node_modules:
This is the crux of what prevents NPM from working natively in-browser. Resolving, downloading, and extracting the hundreds of megabytes (or gigabytes) typical frontend projects contain in their node_modules folder is a challenge browsers
just aren’t well suited for. Additionally, this is also why existing server side solutions to this problem have proven to be slow, unreliable, and cost prohibitive to scale.
So, if NPM itself can’t work in the browser, what if we built a brand new NPM client from the ground up that could?
The solution: A smarter, faster package manager built specifically for the web 📦
Turbo’s speed & efficiency is largely achieved by utilizing the same techniques modern frontend applications use for snappy performance—tree-shaking, lazy-loading, and plain ol’ XHR/fetch with gzip.
Retrieves only the files you need, on-demand 🚀
Instead of downloading entire tarballs, Turbo is smart and only retrieves the files that are directly required from the main, typings and other relevant fields. This eliminates a surprising amount of dead weight in individual packages and even more so in larger projects:
So what happens if you import a file that’s not required by the main field, like a Sass file for example? No problem—Turbo simply lazy-loads it on-demand and persists it for future use, similar to how Microsoft’s new GVFS Git protocol works.
Robust caching with multiple failover strategies 🏋️
We recently rolled out a Turbo-specific CDN that hydrates any NPM package in one gzipped JSON request, providing massive speed boosts to package installations. This concept is similar to NPM’s tarballs which concats all files in a package and gzips them, whereas Turbo’s cache intelligently concats only the files your application needs and gzips them.
Every Turbo client runs standalone in-browser and automatically downloads the appropriate files on-demand directly from jsDelivr’s production grade CDN if a package fails to be retrieved from our cache. But what if jsDelivr goes down too? No sweat—it then switches over to using Unpkg’s CDN instead, giving you three separate layers of redundancy for ultra reliable package installations 👌
Lightning fast dependency resolution ⚡️
To ensure minimal payload sizes, Turbo uses a custom resolution algorithm to aggressively resolve common package versions whenever possible. It’s also insanely fast & redundant: the serverless version of the resolver has access to NPM’s entire dataset in-memory and resolves any package.json in <85ms. Should Turbo have any problems connecting to the serverless resolver, it gracefully fails over to running completely in-browser and retrieves all required metadata for resolution.
Doing dependency resolution on the client also opens up some new & exciting possibilities, like the ability to install missing peer dependencies in just one click 😮:
Proven to work at scale 📈
Turbo is now reliably handling tens of millions of requests every month with negligible overhead cost, and we’re also excited to announce that Google’s Angular team recently chose StackBlitz to power all the live examples that millions of developers use in their docs!
Now in technology preview 🙌
Turbo is live on StackBlitz.com and during the technology preview phase we’ll be running a ton of tests & releasing speed, efficiency, and reliability improvements, all of which your feedback is critical on — so please don’t hesitate to file any issues you run into or to chat with us in our Discord community! 🍻
While Turbo was originally designed for production grade usage in a real IDE, parts of it have already found their way into a handful of online playgrounds & folks in our community have even started prototyping a way to enable script type=module to work with Turbo (how cool is that??). We can’t wait to see all the other amazing stuff people come up with, so once our API churn smooths out we’ll be open sourcing all of it (and many other parts of StackBlitz) in our our Github repo for the world to use 🤘
Lastly, we want to give a huge thanks to Google’s Angular team for deciding to take a bet on our technology & to the Google Cloud team for their amazing service + sponsoring the servers that Turbo runs on! ❤️
As always, please feel free to tweet me @ericsimons40 or @stackblitz with any questions, feedback, ideas, etc :)
PS — If you’re interested in supporting our work, please consider grabbing a Thinkster Pro subscription! We’re creating a brand new series on how we built Turbo & StackBlitz, as well as revamping our entire content catalog :)
Rock on & I hope to see y‘all in Mountain View on Wednesday, December 6th!
