Downloader sometimes fails on external .dsc files signed with unusable
GPG keys:
- expired keys
- .dsc file signed with a key that is not in apt's GPG database.
These files are typically signed by individual package maintainers, not
the official Debian/binary repository key that APT uses for binary
package indexes.
Downloader uses 2 different methods to fetch source packages, which
behave differently w.r.t. GPG checks:
- with "archive" key present in meta_data.yaml, we download using the
"dget" utility, which in turn validates against a set of known keys
in /usr/share/keyrings/
- without "archive", we download using "apt-get source", which doesn't
validate GPG at all
This patch makes the downloader script ignore GPG signature errors, but
print a warning in the log. We also allow package maintainers to add an
additional option, "dsc_sha256", to check the .dsc file's checksum,
instead of its GPG signature (this will suppress the warning).
Rationale:
- it's difficult to make GPG verification accept expired keys
- we always verify sha256 checksums of the files making up the source
package (ie the files referenced by .dsc)
- as for the .dsc file itself, we only verify its checksum if it is
present in meta_data.yaml, "dsc_sha256". No packages do that as of
this writing.
CHANGES
=====================
- utils.py: add a slightly different version of run_shell_cmd that
captures and returns both STDOUT and STDERR
- debrepack.py:
* use "dscverify" (from "devscripts" package) to verify .dsc files
* if normal verification fails, try again with GPG check disabled
* new key in meta_data.yaml: dsc_sha256. If present, make sure .dsc
file's checksum matches.
* workaround for "dget" and "dscverify" falsely succeeding when
files referenced by .dsc are missing
* removed functions "download_check_dsc" and "check_dsc" as they are
no longer used after this change
HOW TO REPRODUCE
=====================
One example of a package signed with a key that is not in the current
(bullseye) version of debian-keyring, is golang-github-golang-jwt-jwt
[1]. The download fails on that package.
TESTS
=====================
- Remove golang-github-golang-jwt-jwt from /import/mirrors and reproduce
the download error (unknown GPG key)
- Apply this patch, remove all downloaded sources, re-run downloader and
make sure it succeeds with GPG-related warnings
- Simulate various problems with a .dsc file and make sure they are
detected (ie the script fails):
* .dsc URL in meta_data.yml returns http 404
* one of the files referenced by .dsc returns http 404
* one of the checksums in .dsc doesn't match
* .dsc checksum it self doesn't match "dsc_sha256" in meta_data.yaml
[1] 2b7ac3c340/golang-github-dev/golang-github-golang-jwt-jwt-dev/debian/meta_data.yaml
Closes-Bug: 2072650
Signed-off-by: Davlet Panech <davlet.panech@windriver.com>
Change-Id: I2c91a997eafdcfd546d79e575c81bf6f9530ca0a
e66aead87b