How to cache the RUN npm install instruction when docker builds a Dockerfile?

Introduction

When building a Docker image, one of the most time-consuming and resource-intensive steps is running the "RUN npm install" instruction. This instruction installs all the dependencies listed in your package.json file. Caching the results of this instruction can greatly improve the build time of your image. In this article, we will explore different strategies for caching the "RUN npm install" instruction in a Dockerfile.

Caching Strategies

There are several strategies for caching the "RUN npm install" instruction in a Dockerfile. These include −

• Using a .dockerignore file − This strategy involves excluding the node_modules directory from being copied into the image. This is done by creating a .dockerignore file and listing the node_modules directory in it. When the image is built, the files in the node_modules directory are not copied, and the "RUN npm install" instruction is executed again. This strategy is simple to implement, but it has a drawback, which is that it increases the size of the image, as it will contain all the dependencies.

• Multi-stage builds − This strategy involves using multiple FROM statements in a Dockerfile to create multiple images, each with a specific purpose. The first stage of the build installs the dependencies and copies the node_modules directory. The second stage of the build copies only the necessary files from the first stage, such as the application code. This strategy reduces the size of the image, but it can be more complex to set up and maintain.

• Using a package-lock file − This strategy involves using a package-lock file, such as npm-shrinkwrap.json or yarn.lock, to ensure consistent installs. These files lock the versions of the dependencies so that the same versions are installed each time the image is built. This strategy is simple to implement and helps to ensure consistency, but it can lead to issues if the dependencies are updated and the lock file is not.

• Using an npm cache volume − This strategy involves creating a volume for the npm cache and using it for the "RUN npm install" instruction. This will make the npm install faster as the packages are already cached, but it requires additional configuration and management of the volume.

• Using BuildKit − BuildKit is a feature added in Docker version 18.09 that improves the build performance. It allows caching the npm packages across multiple stages and images, reducing the time of the npm install instruction.

• Using --no-cache flag − This strategy involves using the --no-cache flag in the "RUN npm install" instruction, which tells Docker to not use the cache when running the command. This can be useful if there are issues with the cache or if the dependencies have been updated. However, this will increase the time of the build.

Implementing the cache in a Dockerfile

Here are examples of how to implement some of the strategies discussed above in a Dockerfile −

• Using a .dockerignore file −

# .dockerignore file
node_modules

# Dockerfile
COPY . .
RUN npm install

• Multi-stage builds −

# Dockerfile
FROM node:12 as builder
COPY package.json package-lock.json ./
RUN npm ci
COPY . .
RUN npm run build

FROM node:12
COPY --from=builder node_modules node_modules
COPY --from=builder dist dist
CMD ["npm", "start"]

• Using a package-lock file −

# package-lock.json or yarn.lock file
...

# Dockerfile
COPY package.json package-lock.json ./
RUN npm ci
COPY . .

• Using an npm cache volume −

# Dockerfile
VOLUME /root/.npm
COPY package.json package-lock.json ./
RUN npm ci --cache /root/.npm
COPY . .

• Using BuildKit −

# Dockerfile
COPY package.json package-lock.json ./
RUN --mount=type=cache,target=/root/.npm npm ci
COPY . .

• Using --no-cache flag −

# Dockerfile
COPY package.json package-lock.json ./
RUN npm ci --no-cache
COPY . .

Best practices for caching the RUN npm install instruction in Docker

• Use a package-lock file (such as npm-shrinkwrap.json or yarn.lock) to ensure consistent installs and to lock the versions of the dependencies.

• Use a multi-stage build to only copy the necessary files from the previous build, such as the node_modules directory, which can help to reduce the size of the image.

• Use a .dockerignore file to exclude the node_modules directory from being copied into the image, this can help to speed up the build time, but it can increase the size of the image.

• Use an npm cache volume to cache the npm packages across multiple stages and images, reducing the time of the npm install instruction.

• Use BuildKit, which is a feature added in Docker version 18.09 that improves the build performance and allows caching the npm packages across multiple stages and images.

• Use the --no-cache flag cautiously, as it tells Docker to not use the cache when running the command and it can increase the time of the build.

• Monitor the cache regularly, inspect the cache and clear it when necessary, if you notice that the cache is not working as expected.

• Keep your dependencies up-to-date, if you are using a package-lock file make sure to update the lock file accordingly.

• Always test your builds with the cache enabled and disabled, to ensure that the cache is working as expected.

• Consider using a separate container or service to handle the npm install step, this can help to speed up the build process and ensure consistency across multiple builds.

Conclusion

Caching the "RUN npm install" instruction in a Dockerfile can greatly improve the build time of your image. There are several strategies for caching, including using a .dockerignore file, multi-stage builds, a package-lock file, an npm cache volume, BuildKit and --no-cache flag. Each strategy has its own pros and cons, and the best strategy for your use case will depend on your specific requirements. To troubleshoot and maintain the cache, you can use the Docker cache command and clear the cache when needed.