On doing 540 no-source-change source-only uploads in two weeks

So I've been doing 540 no-source-change source-only uploads in the last two weeks and am planning to do 3000 more in January 2021. We'll see how that goes ;)

Let me explain what I have been doing and why.

So, starting with the Bullseye release cycle the Release Team changed policy: only packages which were build on buildds are allowed to migrate to testing.

Which is pretty nice for reproducible builds as this also ensures that a .buildinfo file is available for anyone wanting to reproduce the binaries of that package.

However, there are many binary (and source) packages in Debian which were uploaded before 2016 (which is when .buildinfo files were introduced) or were uploaded with binaries until that change in release policy July 2019.

Then Ivo De Decker scheduled binNMUs for all the affected packages but due to the way binNMUs work, he couldn't do anything about arch:all packages as they currently cannot be rebuilt with binNMUs.

Ivo and myself discussed what could be done about the remaining packages and (besides long complicated changes to Debian's workflows) the only thing deemed possible was doing many many source uploads with just a new changelog entry:

  * Non maintainer upload by the Reproducible Builds team.
  * No source change upload to rebuild on buildd with .buildinfo files.

These packages are all inherently buggy, because Debian policy mandates that packages should be reproducible and without .buildinfo files one cannot reproducibly rebuild packages. So instead of filing many many bugs we've decided to just fix these bugs by doing a no-source-change source uploads. One nice aspect of these uploads is that there's no follow-up work imposed on the maintainer: whether they keep that changelog entry or whether they discard it, it does not matter.

So Ivo had developed an SQL query which showed 570 packages needing an update roughly two weeks ago, on December 18 and so I started slowly. This is the amount of NMUs I did in the last days:

for i in $(seq 18 30) ; do echo -n "Dec $i: " ; ls -lart1 done/*upload|grep -c "Dec $i" ; done
Dec 18: 12
Dec 19: 0
Dec 20: 3
Dec 21: 13
Dec 22: 13
Dec 23: 16
Dec 24: 4
Dec 25: 28
Dec 26: 0
Dec 27: 38
Dec 28: 198
Dec 29: 206
Dec 30: 9

About ten packages had FTBFS bugs preventing an upload and seven packages were uploaded by the maintainer before me. I've seen two cases of sudden maintainer uploads after 8 and 10 years of no activity!

So what did I do for each upload?

  • pre upload work:
    • test build with pbuilder in sid
    • check PTS for last upload date (and having arch:all binaries) and open RC bugs
    • modify d/changelog
    • check debdiff between two sources (should be only the changelog entry!)
    • upload
    • (some times filing bugs or modifying bug meta data etc)
  • post upload:
    • check PTS for testing migration, so for this I've had >500 browser tabs open and I'll keep each of them open until the packages migrates...

Much to my surprise I didn't get much feedback, there were like 6 people on the #debian-reproducible channel cheering and one on #debian-qa, though that person is a Release Team member so that was kind of important cheering. And I've seen some maintainer uploads to packages which haven't seen uploads since some years. And really nice: no-one complained so far. I hope this will stay this way with the plan to do 3000 more uploads of this kind:

Those 570 packages were only key packages but there are 3000 more source packages which have a binary in bullseye for which no .buildinfo file exists. So I plan to upload them all in January 2021 and you can help me doing so, by uploading your packages before me - and maybe fixing some other bugs in the process!

I've posted the list of packages (sorted by ddlist) to debian-devel@lists.d.o, see https://lists.debian.org/debian-devel/2020/12/msg00419.html.

Many thanks to Ivo and the whole Release Team for their support of Reproducible Builds and generally speaking for the many many enhancements to the release process we've seen over the years. Debian in 2021 will rock'n'roll more than ever! So thank you all, once again, for making Debian what it is and what it will be!