Jetpack: blazingly fast Serverless packaging and deploys
14 May 2019
We love the Serverless Framework at Formidable. We get to focus on application development while it takes care of details like packaging and deployment—all with a rich plugin ecosystem and a great technical community!
However, an oft-recurring pain point for Serverless projects is that packaging and deploying can be really, really slow.
We're thrilled to introduce the serverless-jetpack plugin, a drop-in replacement for built-in
What slows down Serverless packaging?
A few months ago, a co-worker noticed that Serverless Framework deploys for a client project were taking more than 10 minutes in just the packaging stage. This made the developers miserable, and even worse, significantly hindered our production deployment speed.
- Read nearly all of
node_modules(and other sources) from disk into a preliminary file list.
- Then exclude files from the list detected as
devDependencies. (And, perform additional disk I/O in
node_modulesto infer what are
This approach hit us hard, as our client project had a git monorepo with many individually packaged Lambda functions, meaning lots of large
node_modules directories driven primarily by
A simple, faster idea
So boiling the problem down: the vast bulk of files in
node_modules read during packaging end up being excluded later because they're
devDependencies. Then we thought, what if we didn't have to read all of those
devDependencies? What if we somehow had the
serverless CLI read just the production ones?
Well, we would need a tool to quickly get us production dependencies... But, as it turns out we have not one, but two! Modern and , the ubiquitous package managers your project already uses, install production dependencies very quickly when used with a lockfile.
We postulated a theory: the cost of an additional, temporary
yarn|npm production install would be far less than the current time the
serverless CLI uses in excluding extraneous
node_modules files post-read.
How it works
With this hunch in hand, we tried a simple packaging experiment:
- Do a fresh production
yarn|npminstall into a temporary directory.
- Do normal exclude/includes on project sources except
node_modules, instead matching against the production one in our temporary directory.
The results were amazing. Across various packaging scenarios, we were able to see consistent, impressive packaging speedups. And the speedups increased as development dependencies increased.
Some timings for relative comparison from our project benchmark:
(The Mode column has clickable links to scenario projects on GitHub. For Type,
baseline is built-in Serverless Framework packaging,
jetpack is our new plugin. Time is in milliseconds.)
Try it out!
... and thus the serverless-jetpack plugin was born!
Installation should be a breeze for most projects. Jetpack works seamlessly with built-in Serverless packaging configuration, including service and individual function-level configurations with
npm users should add this to
plugins: - serverless-jetpack
npm users will also need:
custom: serverless-jetpack: mode: npm # (default `yarn`)
And that's mostly it!
We take correctness seriously at Formidable. In addition to the usual tests and style checks, we validate that Jetpack's output in each scenario identically matches that of built-in Serverless packaging.
As a few parting thoughts, you should be using
yarn with a
yarn.lock file or
email@example.com+ with a
package-lock.json to get a full speedup. And, there are a few esoteric complexities and additional configuration options that you may want to review before integration.
All in all, if Serverless Framework packaging/deployment speed is cramping your style, hook up a jetpack and see if your deploys take off! 🚀