TransWikia.com

How is a reproducible build guaranteed with version ranges in NPM?

Software Engineering Asked by Finlay Weber on January 9, 2021

I know with NPM caret, tilde and some logical operators can be used to specify version ranges. This post explains a bit on how this works.

The problem now is I find it hard to reconcile the use of version ranges with the idea of having reproducible builds. Version ranges for dependencies means that you are not specifying a requirement for a particular version but a range of versions, which might change between builds (e.g. a patch release of a dependency was released between the last and current build).

Reproducible builds seek to remove variability in the environment ensuring that every repeated build would always be the same.

How it is ever possible to have reproducible builds with version ranges when using NPM?

3 Answers

It's possible by having two distinct lists of dependencies, one with ranges, and one with specific versions, known as a lockfile.

Version ranges are helpful for libraries so that bug patches in your libraries' dependencies can be included without the libraries needing to be updated. They also give you the option of choosing a library version that meets the requirements of multiple packages.

Reproducible builds are generally more useful for apps than libraries. So by committing the lockfile to source control and including it in your deployment system, you can ensure that the precise versions of all dependencies you tested locally will be used when deploying, whether to one server or a thousand.

Npm keeps the lockfile as a separate file, so that you can choose whether to use it or not. If you always want the latest versions, you can have them. If you want reproducible builds, you can have them by distributing the lockfile.

Correct answer by curiousdannii on January 9, 2021

The problem now is I find it hard to rectify the use of version ranges with the idea of having reproducible builds.

It is obviously impossible.

how it is ever possible to have reproducible builds with version ranges when using npm

You can't. What makes you think you can?

This doesn't even have anything to do with NPM. It is just basic logic. "Reproducible build" means all versions are exactly specified, "version range" means, at least one version is not exactly specified, ergo you cannot have reproducible builds with version ranges, and you cannot have version ranges with reproducible builds.

That's just a simple logical contradiction. Nothing to do with NPM or any other packaging system.

If you want reproducible builds, you need to give up on using version ranges. Nobody is forcing you to use version ranges, and nobody is forcing you to use reproducible builds. You can choose to use either one of them, or neither one of them. You can not choose to use both, because they logically contradict each other.

What you can do, however, is the following:

Most newer package management systems split dependency resolution and package build into two separate steps. RubyGems, NPM, Yarn, and others do this, for example.

In NPM, RubyGems, and Yarn, after the package management system has resolved the dependencies, it records the resolution it found. All three package management systems I mentioned call this a "lock file". The package build is then performed using the already resolved dependencies from the lock file.

So, you can use version ranges during dependency resolution, but then you can commit the lock file to the repository, and thus everybody who builds the package from the repository, will get the already resolved exact versions.

Again, that is your choice. You can distribute the lock file, but you don't have to.

Again, all of this is a choice. Some people like version ranges, some people don't. If you don't like 'em, don't use 'em. Some people like reproducible builds, some people don't. If you decide to have reproducible builds, then you can't have version ranges. Period.

I, personally, chose to re-resolve dependencies on every build for my own private packages.

Answered by Jörg W Mittag on January 9, 2021

You can have a reproducible build and version ranges for your dependencies, so long as each build does not reinstall dependencies.

The version range is a setting in a config file, but a build is more than a config file. If the code is exactly the same, including dependencies from NPM, then it is a reproducible build. The challenge is to only download those dependencies once, and reuse them for each build. As soon as you remove those files and the build does another npm install you are not guaranteed to have a perfectly reproducible build.

Version ranges are a convenience. Some projects do not need a perfectly reproducible build. Others do.

Personally I would be surprised to find version ranges for dependencies when a reproducible build is desired. I would expect exact version numbers, and any change to those versions be planned as part of the development process, which sets a new baseline for reproducible builds. By specifying a version range, you are relying on build logic alone to ensure reproducibility.

Even if you specify exact version numbers, the mere fact an npm install downloads the source files again is a bit of a red flag. Perfectly reproducible builds should not contain any code changes, and may also involve calculating a checksum of the files used to produce the build.

NPM is not something you control, so there is nothing truly preventing someone from deleting their package and then republishing it under the same name, same version, with different files. So even exact version numbers for dependencies are not fail-safe.

It depends on how you define "reproducible." With a narrow version range, you might only receive patches, but no new features. This is not exactly a terrible thing, nor is it particularly great. It depends on the project.

If "reproducible" means bit-for-bit any two builds are exactly the same, I would argue exact version numbers for dependencies are not enough, but a good start. You may also need to calculate a checksum based on the files used to build the application.

Answered by Greg Burghardt on January 9, 2021

Add your own answers!

Ask a Question

Get help from others!

© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP