Jetpack revisited: Even faster Serverless packaging and deploys
Two weeks ago we introduced the serverless-jetpack plugin, a drop-in replacement for Serverless Framework packaging. We thought that we had sped up
serverless CLI packaging and deployments as much as possible.
It turns out we were wrong.
We discovered we could make it even faster...
Reexamining the problem
As we discussed previously,
serverless CLI packaging slows to a crawl when reading large
node_modules directories on disk, only to later exclude most files from application packaging as
Our solution with
serverless-jetpack was to avoid any introspection of
node_modules by doing a
npm production installation in a temporary directory. This takes a bit of time, but often far less than globbing on disk with built-in
In the course of our initial release, we received great feedback from the community, including some undiscovered issues and new ideas as to how to tackle the speed problem.
Back to the drawing board
The existing design of Jetpack offers an easy means of replacing built-in
serverless packaging with a new heuristic of our choosing. Jetpack also has a full benchmark and integration test suite to ensure that the plugin is faster than
serverless while still correctly producing identical packaging output.
...so maybe we can take a different approach?
After a seemingly promising false start, we reexamined the problem. We need to not read
devDependencies. How else could we do it?
Using techniques pioneered in
inspectpack, we created a small, focused library named inspectdep, which efficiently reads
node_modules and discovers the on-disk locations of production modules. By hooking
inspectdep into Jetpack we were able to:
- Quickly identify production dependencies before globbing.
- Use that information to skip disk I/O for
devDependenciesby reading only production dependencies in a project's
In effect, Jetpack is able to do pretty much the same thing that
serverless does, just much, much faster.
A faster, more powerful Jetpack
With this refactor, the
severless-jetpack plugin is much simpler.
There are no
There are no more configuration options! The plugin "just works" with existing
include|exclude Serverless packaging configurations. If it works with
serverless, it should work with
So, how much faster is it? Let's check out some new benchmarks!
This time around, we have Windows VM timings from our new AppVeyor CI. The speedup for Windows seems to be even more dramatic—for the
serverless-jetpack is over 15 times faster than built-in
serverless packaging in our CI setup.
Fire it up!
We've been very pleased with the reception of serverless-jetpack since its introduction. Our internal refactoring should now make it better and easier to integrate in Serverless Framework applications. We hope the newer, shinier Jetpack speeds up your
serverless packaging even more! 🚀