Gradle Verification Metadata doesn't get updated without verify-metadata or verify-signatures being set

[jamietanna]

Tell us more.

Context

Related to #32534, Renovate's Gradle Verification Metadata support appears to only be triggered when verify-metadata or verify-signatures are specified.

This generally makes sense, as the gradle/verification-metadata.xml file is generally only updated when there are checksums of metadata files i.e. pom.xmls.

However, Gradle's Verification Metadata support does also make it possible to store checksums of dependencies in the gradle/verification-metadata.xml even with these settings disabled.

This could be for dependencies that don't have a checksum distributed in their registry, or for some other reason.

This means that there are cases where using Renovate with Gradle's Verification Metadata does not work out-of-the-box, as you then need to enable verify-metadata or verify-signatures are specified.

Example

For instance, with Elasticsearch, we have Verification Metadata set up, but have neither verify-metadata or verify-signatures specified, but we do have other dependencies in the gradle/verification-metadata.xml.

In the case that both options are unset, Gradle will perform metadata verification for these direct dependencies.

When Renovate attempts to update this, however, the ./gradlew ... command is not executed.

Workaround

This would mean that - in the current implementation - we need to enable verify-metadata and/or verify-signatures to be able to get Renovate to update the Gradle Verification Metadata.

This is a reasonable short-term solution, but does introduce some trade-offs for the Gradle build.

Proposal

An ideal case would be that Renovate triggers a verification metadata update i.e. if there are any entries in the gradle/verification-metadata.xml.

[Churro]

If I understand you correctly, you are suggesting to unconditionally always update checksums in verification-metadata.xml even if they are not be actively verified during dependency resolution. If so, wouldn't that need to apply to PGP signatures too? Or in other words:

• Update digests for all components if any component declares them, regardless if <verify-metadata>true</verify-metadata>
• Update pgp values for all components if any of them declares them, regardless if <verify-signatures>true</verify-signatures>

Do you think users would expect this file to be updated with digests/signatures even if verification is disabled?

I'm not sure how widespread cases like Elasticsearch are, but without verification the practical value of digests/signatures is pretty limited. I can only imagine they are needed as audit trail/integrity documentation or for manual investigations (oh, hello xz utils).

[jamietanna]

If I understand you correctly, you are suggesting to unconditionally always update checksums in verification-metadata.xml even if they are not be actively verified during dependency resolution. If so, wouldn't that need to apply to PGP signatures too? Or in other words:

I'm not sure how widespread cases like Elasticsearch are, but without verification the practical value of digests/signatures is pretty limited. I can only imagine they are needed as audit trail/integrity documentation or for manual investigations (oh, hello xz utils).

So to give a bit more colour, I can confirm that the checksums in gradle/verification-metadata.xml are verified, even if verify-metadata and verify-signatures are unset, by:

• Taking Elasticsearch's repo (commit 5e16bc3fa615d76a5f188e0b722691da2981e633)
• Modifying the gradle/verification-metadata.xml to falsify all checksums
  • i.e. with :%s/value="..../value="1111
• Ran ./gradlew check

This then results in the following error:

FAILURE: Build failed with an exception.

* Where:
Settings file '/home/jamie/workspaces/elasticsearch/settings.gradle' line: 17

* What went wrong:
Error resolving plugin [id: 'com.gradle.develocity', version: '3.18.1']
> A problem occurred configuring project ':build-tools'.
   > Execution failed for task ':build-conventions:compileJava'.
      > Dependency verification failed for configuration ':build-conventions:compileClasspath'
        21 artifacts failed verification:
          - Saxon-HE-11.3.jar (net.sf.saxon:Saxon-HE:11.3) from repository MavenRepo
          - antlr4-runtime-4.10.1.jar (org.antlr:antlr4-runtime:4.10.1) from repository MavenRepo
          - checker-qual-3.12.0.jar (org.checkerframework:checker-qual:3.12.0) from repository MavenRepo
          - checkstyle-10.3.jar (com.puppycrawl.tools:checkstyle:10.3) from repository MavenRepo
          - commons-beanutils-1.9.4.jar (commons-beanutils:commons-beanutils:1.9.4) from repository MavenRepo
          - commons-collections-3.2.2.jar (commons-collections:commons-collections:3.2.2) from repository MavenRepo
          - commons-io-2.2.jar (commons-io:commons-io:2.2) from repository MavenRepo
          - commons-logging-1.2.jar (commons-logging:commons-logging:1.2) from repository MavenRepo
          - error_prone_annotations-2.11.0.jar (com.google.errorprone:error_prone_annotations:2.11.0) from repository MavenRepo
          - failureaccess-1.0.1.jar (com.google.guava:failureaccess:1.0.1) from repository MavenRepo
          - guava-31.1-jre.jar (com.google.guava:guava:31.1-jre) from repository MavenRepo
          - j2objc-annotations-1.3.jar (com.google.j2objc:j2objc-annotations:1.3) from repository MavenRepo
          - javassist-3.28.0-GA.jar (org.javassist:javassist:3.28.0-GA) from repository MavenRepo
          - jsr305-3.0.2.jar (com.google.code.findbugs:jsr305:3.0.2) from repository MavenRepo
          - listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar (com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava) from repository MavenRepo
          - picocli-4.6.3.jar (info.picocli:picocli:4.6.3) from repository MavenRepo
          - plexus-utils-3.2.1.jar (org.codehaus.plexus:plexus-utils:3.2.1) from repository MavenRepo
          - reflections-0.10.2.jar (org.reflections:reflections:0.10.2) from repository MavenRepo
          - slf4j-api-1.7.32.jar (org.slf4j:slf4j-api:1.7.32) from repository MavenRepo
          - xmlresolver-4.2.0-data.jar (org.xmlresolver:xmlresolver:4.2.0) from repository MavenRepo
          - xmlresolver-4.2.0.jar (org.xmlresolver:xmlresolver:4.2.0) from repository MavenRepo
        This can indicate that a dependency has been compromised. Please carefully verify the checksums.
        
        Open this report for more details: file:///home/jamie/workspaces/elasticsearch/build/reports/dependency-verification/at-1732706294509/dependency-verification-report.html

In this case it indicates that there is still verification that occurs when Gradle runs - it is not the additional verification that happens for metadata or signatures, if that makes sense?

Snippet of modified gradle/verification-metadata.xml

         <component group="com.microsoft.azure" name="msal4j-persistence-extension" version="1.3.0">
             <artifact name="msal4j-persistence-extension-1.3.0.jar">
-            <sha256 value="dfc41c817fbfa76057af6ffe4379dbca6a5e16b8e87df8bdda23f371756c2d09" origin="Generated by Gradle"/>
+                <sha256 value="11111c817fbfa76057af6ffe4379dbca6a5e16b8e87df8bdda23f371756c2d09"
+                    origin="Generated by Gradle" />
             </artifact>
         </component>
         <component group="com.microsoft.sqlserver" name="mssql-jdbc" version="6.2.1.jre7">
             <artifact name="mssql-jdbc-6.2.1.jre7.jar">
-            <sha256 value="9cfa259450ae3471d2e6e2c3d8aefcce236e3daef8b3734d21dc93c3a5bbe806" origin="Generated by Gradle"/>
+                <sha256 value="1111259450ae3471d2e6e2c3d8aefcce236e3daef8b3734d21dc93c3a5bbe806"
+                    origin="Generated by Gradle" />
             </artifact>
         </component>
         <component group="com.netflix.nebula" name="gradle-contacts-plugin" version="6.0.0">
             <artifact name="gradle-contacts-plugin-6.0.0.jar">
-            <sha256 value="ea60c14ca84b49c919e591ca718d977cdb5dcc5c5c384e72d56ad7dfd36cf5e5" origin="Generated by Gradle"/>
+                <sha256 value="1111c14ca84b49c919e591ca718d977cdb5dcc5c5c384e72d56ad7dfd36cf5e5"
+                    origin="Generated by Gradle" />
             </artifact>

[Churro]

Thanks, I missed that it would still verify the digests of JAR files with <verify-metadata>false</verify-metadata>. I checked gradle's DependencyVerifier.java but probably not thoroughly enough.