trace-pkg: Package Node.js apps for AWS Lambda and beyond

Packaging Node.js applications into a single artifact (e.g. a zip file) for deployment shouldn't be a big hassle. Gather application sources and package dependencies, and then stick them all together, right? Unfortunately, the practical realities of this task can be crushingly slow and produce bloated packages full of unused code.

To address this problem, we are thrilled to introduce the trace-pkg tool for creating fast and tiny Node.js application packages for any production deployment environment and suitable for use with any cloud infrastructure tooling!

Packaging Node.js applications

In our various work over the years with large-scale Serverless Framework applications, we found that creating the application packages and deploying them to AWS Lambda was a cumbersome and inefficient process. Long story short, naively zipping up all the sources and production dependencies for a given application can produce big bundles and take a lot of time.

We detailed our discovery path in writing the serverless-jetpack plugin to address these problems in several blog posts to find the fastest technique for creating the smallest, correct application package, namely:

1. Take one or more application files as input.
2. Read all imports (e.g. require() calls and import statements) and discover dependent files.
3. ... repeat for each dependent file.
4. Bundle them all together.

With this approach, serverless-jetpack has been able to create packages that are 25x faster and 4x smaller than the built-in serverless packaging features.

Generalizing package tracing

While serverless-jetpack is great for serverless applications, there are a lot more production scenarios outside of the Serverless Application Framework. Many projects have bespoke packaging. For example, some may implement a Terraform or CloudFormation stack to target AWS Lambda (a service with stringent code size constraints). Other projects can target a container-based service like Kubernetes or AWS ECS and just need a fast, slim artifact packaging tool.

With these thoughts in mind, we are pleased to announce the trace-pkg command line tool for Node.js applications! trace-pkg offers:

• 📦 the smallest possible application package via import tracing
• 🚀 blazingly fast package creation
• 🔀 parallel builds using all CPUs
• ⚙️ flexible options and configuration

All you really need to get going is to install trace-pkg, create a configuration file, and then run the command line tool. For example, running:

$ trace-pkg -c trace-pkg.yml

with a configuration file trace-pkg.yml that looks something like:

options:
  concurrency: 0 # Use all CPUs

packages:
  # Trace an API server entry point.
  api:
    trace:
      - packages/api/src/server.js

  # Trace a web app server and add some CSS files too
  webapp:
    trace:
      - packages/webapp/src/server.js
    include:
      - packages/webapp/src/*.css

will produce slim, fast zip packages suitable for deployment in your production environment!

There are various command line and configuration options that may be appropriate for your projects. And, like serverless-jetpack, there are important sections to review about handling dynamic import misses and other complexities.

Try it out!

If you deploy to AWS Lambda or any other environment via zip application packages, we encourage you to try out trace-pkg and see if it can speed up and slim down your production deployment experience. We've seen enormous speedups and happier engineering teams and hope that you experience the same!