"packages should build reproducibly" - after 4 years this work of many is in debian-policy now

This post was written roughly 44h ago and now that the fix for #844431 has been merged into the git master branch, I'm publishing it - hoping you'll enjoy this as much as I do!

So today is the last (official) day of DebConf17 and it looks like #844431: "packages should build reproducibly" will be merged into debian-policy today! So I'm super excited, super happy, quite tired and a bit sad (DebConf is ending…) right now! :-)

Four years ago Lunar held a BoF at DebConf13 which started the initiative in Debian. I only got involved in September 2014 with setting up continuous tests, rebuilding each package twice with some variations and then comparing the results using diffoscope, which back then was still called debbindiff and which we renamed as part of our efforts to make Reproducible Builds the norm in Free Software.

Many people have worked on this, and I'm also very happy how visible this has been in our talk here yesterday. You people rock and I'm very thankful and proud to be part of this team. Thank you everyone!

And please understand: we are not 94% done yet (which our reproducibility stats might have made you think), rather more like half done or so. We still need tools and processes to enable anyone to indepently verify that a given binary comes from the sources it is said to be coming, this will involve distributing .buildinfo files and providing user interfaces in APT and elsewhere. And probably also systematic rebuilds by us and other parties. And 6 or 7% of the archive are a lot of packages still, eg in Buster we currently still have 273 unreproducible key packages and for a large part we don't have patches yet. So there is still a lot of work ahead.

This is what was added to debian-policy now:

Reproducibility
---------------

Packages should build reproducibly, which for the purposes of this
document [#]_ means that given

- a version of a source package unpacked at a given path;
- a set of versions of installed build dependencies;
- a set of environment variable values;
- a build architecture; and
- a host architecture,

repeatedly building the source package for the build architecture on
any machine of the host architecture with those versions of the build
dependencies installed and exactly those environment variable values
set will produce bit-for-bit identical binary packages.

It is recommended that packages produce bit-for-bit identical binaries
even if most environment variables and build paths are varied.  It is
intended for this stricter standard to replace the above when it is
easier for packages to meet it.

.. [#]
   This is Debian's precisification of the `reproducible-builds.org
   definition `_.

For now violating this part of policy may result in a severity: normal bug, though I think we should still only file them if we have patches, else it's probably better to just take a note in our notes.git, like we did before the policy change.

Finally one last comment: we could do reproducible security updates for Stretch now too, for those 94% of the packages which are reproducible. It just needs to be done by someones and the first step would be publishing those .buildinfo files from those builds…