apt-offline not pulling Packages without extension

[ncheenu]

There is a discrepancy between apt-get update and apt-offline get for updates.

While the apt-get update works, apt-offline get does not work when the Packages file does not have any extension in the remote repository. Somehow, apt-get update seems to resolve. But not apt-offline.

$ sudo apt-get update --print-uris | grep google-cloud
'http://packages.cloud.google.com/apt/dists/google-cloud-ops-agent-bullseye-all/InRelease' packages.cloud.google.com_apt_dists_google-cloud-ops-agent-bullseye-all_InRelease 0
'http://packages.cloud.google.com/apt/dists/google-cloud-ops-agent-bullseye-all/main/binary-amd64/Packages.xz' packages.cloud.google.com_apt_dists_google-cloud-ops-agent-bullseye-all_main_binary-amd64_Packages 0
'http://packages.cloud.google.com/apt/dists/google-cloud-ops-agent-bullseye-all/main/binary-all/Packages.xz' packages.cloud.google.com_apt_dists_google-cloud-ops-agent-bullseye-all_main_binary-all_Packages 0
'http://packages.cloud.google.com/apt/dists/google-cloud-ops-agent-bullseye-all/main/i18n/Translation-en.xz' packages.cloud.google.com_apt_dists_google-cloud-ops-agent-bullseye-all_main_i18n_Translation-en 0

URL does not exist for Packages.xz

$ wget -O /dev/null 'http://packages.cloud.google.com/apt/dists/google-cloud-ops-agent-bullseye-all/main/binary-amd64/Packages.xz'
--2024-12-05 16:25:59--  http://packages.cloud.google.com/apt/dists/google-cloud-ops-agent-bullseye-all/main/binary-amd64/Packages.xz
Resolving packages.cloud.google.com (packages.cloud.google.com)... 142.250.195.110, 2404:6800:4007:824::200e
Connecting to packages.cloud.google.com (packages.cloud.google.com)|142.250.195.110|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2024-12-05 16:25:59 ERROR 404: Not Found.

It exists only for Packages

$ wget -O /dev/null 'http://packages.cloud.google.com/apt/dists/google-cloud-ops-agent-bullseye-all/main/binary-amd64/Packages'
--2024-12-05 16:26:34--  http://packages.cloud.google.com/apt/dists/google-cloud-ops-agent-bullseye-all/main/binary-amd64/Packages
Resolving packages.cloud.google.com (packages.cloud.google.com)... 142.250.195.110, 2404:6800:4007:824::200e
Connecting to packages.cloud.google.com (packages.cloud.google.com)|142.250.195.110|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 85577 (84K) [text/html]
Saving to: '/dev/null'

/dev/null                                 100%[===================================================================================>]  83.57K  --.-KB/s    in 0.08s

2024-12-05 16:26:35 (1.06 MB/s) - '/dev/null' saved [85577/85577]

apt-get update is able to resolve this and fetch the right file. But not, apt-offline. I have a patch that tries without a supported extension if everything fails.

diff --git a/apt_offline_core/AptOfflineCoreLib.py b/apt_offline_core/AptOfflineCoreLib.py
index ffa9518..e746ca4 100644
--- a/apt_offline_core/AptOfflineCoreLib.py
+++ b/apt_offline_core/AptOfflineCoreLib.py
@@ -1620,7 +1620,7 @@ def fetcher(args):

             # INFO: Handle the multiple Packages formats.
             # See DTBS #583502
-            SupportedFormats = ["bz2", "gz", "xz", "lzma"]
+            SupportedFormats = ["bz2", "gz", "xz", "lzma", None]

             # INFO: We are a package update
             PackageName = url
@@ -1665,10 +1665,13 @@ def fetcher(args):
                         pkgFileWithType.rstrip(pkgFileWithType.split(".")[-1]).rstrip(
                             "."
                         )
-                        + "."
-                        + Format
+                        + (("." + Format) if Format else "")
                     )
-                    NewUrl = url.replace(PackageFormat, Format)
+
+                    NewUrl = url.replace(PackageFormat, Format if Format else "")
+                    if not Format:
+                        NewUrl=NewUrl[:-1]
+
                     log.verbose(
                         "Retry download %s %s\n" % (
                             NewUrl, LINE_OVERWRITE_FULL)

[ncheenu]

This is on Debian 11 Bullseye

[rickysarraf]

Thank you for reporting. I'll check on this. Though it is quite surprising that in today's day, uncompressed payload is served.

[rickysarraf]

Note to self: The reason we do this compression type ping-pong is because we don't rely on the InRelease file to ascertain the URLs to be downloaded. Instead, we always rely on the client apt tool to report it. And then do the ping-pong cycle.

If we rely on the InRelease file solely, it'd mean we scrape and discard the remaining URL for --update, that were generated on the client with set --update.

Then, there's also set --no-checksum option. Possibly, it would conflict this as well.

So, for now, the current approach should suffice. We should only check on how best to handle the uncompressed apt metadata indices.