Skip to content

(CDPE-7204) Update to Ruby 3.2, PDK 3.4.0, and Debian bullseye and update tooling gems #156

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jun 5, 2025

Conversation

Magisus
Copy link
Contributor

@Magisus Magisus commented Jun 2, 2025

This commit updates PDK to current latest, 3.4.0. It also removes some
unnecessary rm lines from the Dockerfile: in PDK 3.2.0 the
dependencies that used to be used to analytics were dropped when the
analytics code was removed, and this included the httpclient gem,
which contained the files we used to delete because they triggered
security scanners. The gem is no longer included with the PDK package,
so the delete lines aren't needed.

Building this image with the latest nokogiri (1.18.8) ran into an issue
where they have split their gem platforms for the precompiled gem from
just linux to linux-gnu and linux-musl. However on Ruby 3.1 and
Debian Buster, bundler recognized the platform as just plain linux, so
it failed to grab the precompiled gems. We wanted to update to Ruby 3.2
to match what puppet-agent ships with anyway, and the docker images for
3.2 only support back through bullseye, not buster, so this commit
also updates to Debian Buster and Ruby 3.2. On this platform, bundler
correctly detects the gem platform and installs the correct pre-compiled
gems.

Updating to Ruby 3.2 required installing a newer version of rugged (a
dependency of octocatalog-diff), and this version depends on
pkg-config, as well as libssl-dev rather than just libssl1.1 for
HTTPS support, so this commit also installs those.

This PR also updates all of the other test gems and their deps. See commits for details.

@Magisus Magisus requested a review from a team as a code owner June 2, 2025 21:08
This commit updates PDK to current latest, 3.4.0. It also removes some
unnecessary `rm` lines from the Dockerfile: in PDK 3.2.0 the
dependencies that used to be used to analytics were dropped when the
analytics code was removed, and this included the `httpclient` gem,
which contained the files we used to delete because they triggered
security scanners. The gem is no longer included with the PDK package,
so the delete lines aren't needed.

Building this image with the latest nokogiri (1.18.8) ran into an issue
where they have split their gem platforms for the precompiled gem from
just `linux` to `linux-gnu` and `linux-musl`. However on Ruby 3.1 and
Debian Buster, bundler recognized the platform as just plain `linux`, so
it failed to grab the precompiled gems. We wanted to update to Ruby 3.2
to match what puppet-agent ships with anyway, and the docker images for
3.2 only support back through bullseye, not buster, so this commit
also updates to Debian Buster and Ruby 3.2. On this platform, bundler
correctly detects the gem platform and installs the correct pre-compiled
gems.

Updating to Ruby 3.2 required installing a newer version of rugged (a
dependency of octocatalog-diff), and this version depends on
`pkg-config`, as well as `libssl-dev` rather than just `libssl1.1` for
HTTPS support, so this commit also installs those.
@Magisus Magisus changed the title (maint) Update PDK to 3.4.0 (maint) Update to Ruby 3.2, PDK 3.4.0, and Debian bullseye Jun 2, 2025
@@ -51,7 +47,7 @@ RUN groupadd --gid 1001 puppetdev \
&& useradd --uid 1001 --gid puppetdev --create-home puppetdev

# Prep for non-root user
RUN gem install bundler -v 2.4.22 \
RUN gem install bundler -v 2.6.9 \
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Older versions of bundler have some weird bugs around the gem platform stuff. Just updating this to latest didn't fix the issue I was seeing with nokogiri, but it also didn't hurt anything, so figured I might as well update it.

@Magisus
Copy link
Contributor Author

Magisus commented Jun 2, 2025

Release note for nokogiri explaining the gem platform change: https://nokogiri.org/CHANGELOG.html#v1180-2024-12-25 Note that even rebuilding the puppet8 tag with the existing deps would hit this, since we always try to pull latest nokogiri right now. Even on new versions of bundler, and even trying to add the bundle lock --add-platform, I was not able to get Ruby 3.1+buster to register as x86_64-linux-gnu, which meant the install was falling back to the case described in the second paragraph, which failed trying to extract the bundled libraries they ship. Adding the xz-utils package (as described here) fixed that, but I figured since we wanted to bump Ruby anyway, I'd try that, and that fixed it the better way.

@Magisus
Copy link
Contributor Author

Magisus commented Jun 3, 2025

I've tested this with all of our built-in jobs and they all succeeded. But I'm not sure how useful the code I tested them on was.

@Magisus Magisus force-pushed the pdk-3.4.0 branch 3 times, most recently from 6f05dae to cabde57 Compare June 3, 2025 22:26
@Magisus Magisus changed the title (maint) Update to Ruby 3.2, PDK 3.4.0, and Debian bullseye (maint) Update to Ruby 3.2, PDK 3.4.0, and Debian bullseye and update test gems Jun 3, 2025
@Magisus Magisus changed the title (maint) Update to Ruby 3.2, PDK 3.4.0, and Debian bullseye and update test gems (maint) Update to Ruby 3.2, PDK 3.4.0, and Debian bullseye and update tooling gems Jun 3, 2025
@Magisus Magisus changed the title (maint) Update to Ruby 3.2, PDK 3.4.0, and Debian bullseye and update tooling gems (CDPE-7204) Update to Ruby 3.2, PDK 3.4.0, and Debian bullseye and update tooling gems Jun 4, 2025
@Magisus Magisus force-pushed the pdk-3.4.0 branch 2 times, most recently from 2fdc863 to cbe25da Compare June 4, 2025 23:28
Gemfile Outdated
@@ -2,29 +2,26 @@ source 'https://rubygems.org'

gem 'puppet', '~> 8.4'
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This currently pulls in puppet 8.10, which was the last gem release, but the latest puppet version is actually 8.12. I think this is probably related to the Puppet Core changes, I imagine they stopped shipping gems to Rubygems. I'm going to figure out what it would take for us to get the actual latest version of Puppet here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Getting a more recent gem than 8.10 is going to be a bit fraught, and might have security implications for the way we build and publish this image. For now, I might leave that for a separate effort while we discuss how to make it work. It seems like it might be valuable to get the rest of these changes published sooner rather than later.

This commit updates all of our gem dependencies to latest versions
whenever possible, and as new as possible to avoid dependency resolution
conflicts besides that. This list plays nicely together.

It also removes a few puppet-lint plugins that have since been included
in puppet-lint itself. This update required a couple of test changes:

* When the `legacy_facts` plugin was incorporated into core puppet-lint,
they changed the message it emits when a legacy fact is detected to
include the name of the fact. This commit updates the test to match the
new message.
* There was also a new check added that ensures there is only a single
space before an arrow `=>` when there is only a single parameter in a
resource. We had a resource with a second param commented out, where we
never adjusted the spacing for the one that was left. This now failed
the linter, so this commit updates the spacing to make it happy, since
that was in the `goodsyntax` test.
@Magisus Magisus merged commit c451ba3 into puppet8 Jun 5, 2025
2 checks passed
@Magisus Magisus deleted the pdk-3.4.0 branch June 5, 2025 20:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants