Jetpack: blazingly fast Serverless packaging and deploys
14 May 2019
However, an oft-recurring pain point for Serverless projects is that packaging and deploying can be really, really slow.
We're thrilled to introduce the plugin, a drop-in replacement for built-in
What slows down Serverless packaging?
A few months ago, a co-worker noticed that Serverless Framework deploys for a client project were taking more than 10 minutes in just the packaging stage. This made the developers miserable, and even worse, significantly hindered our production deployment speed.
- Read nearly all of
node_modules(and other sources) from disk into a preliminary file list.
- Then exclude files from the list detected as
devDependencies. (And, perform additional disk I/O in
node_modulesto infer what are
This approach hit us hard, as our client project had a git monorepo with many individually packaged Lambda functions, meaning lots of large
node_modules directories driven primarily by
A simple, faster idea
So boiling the problem down: the vast bulk of files in
node_modules read during packaging end up being excluded later because they're
devDependencies. Then we thought, what if we didn't have to read all of those
devDependencies? What if we somehow had the
serverless CLI read just the production ones?
Well, we would need a tool to quickly get us production dependencies... But, as it turns out we have not one, but two! Modern and , the ubiquitous package managers your project already uses, install production dependencies very quickly when used with a lockfile.
We postulated a theory: the cost of an additional, temporary
yarn|npm production install would be far less than the current time the
serverless CLI uses in excluding extraneous
node_modules files post-read.
How it works
With this hunch in hand, we tried a simple packaging experiment:
- Do a fresh production
yarn|npminstall into a temporary directory.
- Do normal exclude/includes on project sources except
node_modules, instead matching against the production one in our temporary directory.
Some timings for relative comparison from our project benchmark:
(The Mode column has clickable links to scenario projects on GitHub. For Type,
baseline is built-in Serverless Framework packaging,
jetpack is our new plugin. Time is in milliseconds.)
Try it out!
npm users should add this to
plugins: - serverless-jetpack
npm users will also need:
custom: serverless-jetpack: mode: npm # (default `yarn`)
And that's mostly it!
As a few parting thoughts, you should be using
yarn with a
yarn.lock file or
email@example.com+ with a
package-lock.json to get a full speedup. And, there are a few esoteric and additional that you may want to review before integration.