Automate chart version bumping

[hairmare]

Is your feature request related to a problem? Please describe.
Keeping Chart versions bumped when implementing features and preparing PRs for rollout can be cumbersome and a tad frustrating.

Describe the solution you'd like
A GitHub Action that takes care of keeping the versions how they should be for us.

• It should parse semantic versions and use them to decide what to bump
• It should support 0.x versions
• it should only bump versions as/where needed
• it should use semantic-commit style comments in commits
• it should create PRs for proposing changes to charts
• it should use derived branch names (ie chart-updater-action/<chartname>/<depname>)
• it should tell us what the update contains
• it could create combined PRs for each *-apps chart

Describe alternatives you've considered
We continue bumping manually and maybe start using tools like helm-local-chart-version as part of our local workflow.

Affected chart
none, CI

Additional context
This might save some time in those moments where a lot of bumps are ready for a given chart. This could also be the foundation for more automation like update PRs at a later stage.

[hairmare]

all my unpublished proof of concepting for this has been following the "write some code to do it" approach and i still have nothing to show that resembles a solution that is maintainable.

this post highlighted the idea of using an existing tool rather than having to write our own. The linked updatecli look like something we'd want to investigate closely.

[hairmare]

i spent some time with updatecli and it looks like it will be able to do alot of the work we'd like it to

the main caveat is that it's still missing some features/has some quirks that we'd probably want to wait for:

• if i run a helmChart update and helm-docs as their own step updatecli will create several commits, afaict upstream is already working on making it possible to squash all the commit from a manifest (tracked in Feature Request: Introduces the concept of "actions" updatecli/updatecli#465)
• because it uses go yaml.Marshal to write changes anything touched by updatecli would get canonicalized (imo we can live with this, the tradeoff wrt to reordering the contents of Charts.yaml is ok and this doesn't affect values.yaml)

i also haven't figured out how to configure how updatecli names branches, how to template commit messages, and how i can analyse an updated targetRevision to help deciding if the charts versions needs a major, minor, or patch bump. I'm still reading updatecli's actual code to figure some of these things out and might end up writing some PRs if the stuff we're missing isn't too large.

One thing that is sure is that we would be writing a lot of yaml to cover all the -apps charts, an example that partially updates argo-cd in infra-apps is below:

almost working updatecli.yaml

title: Update app of apps charts

scms:
  helm-charts:
    kind: "github"
    spec:
      branch: "main"
      user: "adfinisbot"
      email: "[email protected]"
      owner: "hairmare"
      repository: "adfinis-helm-charts"
      token: '{{ requiredEnv "GITHUB_TOKEN" }}'
      username: '{{ requiredEnv "GITHUB_ACTOR" }}'

sources:
  infra-apps-argocd:
    name: Get the latest argocd chart version
    kind: helmChart
    spec:
      url: https://argoproj.github.io/argo-helm
      name: argo-cd

targets:
  infra-apps-argocd:
    name: Update argocd targetRevision
    kind: helmChart
    scmid: "helm-charts"
    sourceid: "infra-apps-argocd"
    spec:
      name: "charts/infra-apps"
      file: "values.yaml"
      key: "argocd.targetRevision"
      versionIncrement: patch
  helm-docs:
    name: Update docs with pre-commit
    kind: shell
    depends_on:
      - infra-apps-argocd
    scmid: "helm-charts"
    spec:
      command: hack/helm-docs.sh

[olblak]

Hi here,

I accidentally found your discussion.
I hope you don't mind me jumping here.

I am the creator of Updatecli

First of all thank you for considering Updatecli.

Your use case is interesting. @dduportal has been pushing in the same direction than you
I just want to add that if you use the GitHub flow, then you could have two commits but one pullrequest.
That's what we do all the time, you need the following snippet in your manifest.

pullrequests:
  helm-charts:
    kind: github
    scmid: helm-charts
    spec:
      automerge: false
      labels:
        - dependencies

Another thing that appears to be confusing here is the difference between the scm of kind git and github.
the git scm would commit directly to the branch specified in your manifest while the "github" one create a temporary branch and is designed to be used with a pullrequest, so it open a pullrqeuest from that temporary branch to the one specified in the manifest.

So the github scm implements the GitHub flow, I got many feedback that it's not an intuitive behavior.
We had lengthy discussion about how this behavior should evolved, and I also have an open pullrequest but we never finished that discussion...

Meanwhile I have been focusing on another area.

To reduce the need to write and maintain manifest, I worked on a feature named "autodiscovery" where Updatecli would detected common behavior. It's still experimental as I am still playing with it to identify the best way to use it. And I would love some feedback

You can have taste with the latest version of Updatecli.
The following manifest should be enough to start with.

By the way manifests under the directory named "updatecli.d" are recognized by IDE(s) that support jsonschema store (like vscode), so you get validation, and autocompletion for free.

updatecli.d/autodiscovery.yaml

name: Update app of apps charts

scms:
  helm-charts:
    kind: "github"
    spec:
      branch: "main"
      user: "adfinisbot"
      email: "[email protected]"
      owner: "hairmare"
      repository: "adfinis-helm-charts"
      token: '{{ requiredEnv "GITHUB_TOKEN" }}'
      username: '{{ requiredEnv "GITHUB_ACTOR" }}'

pullrequests:
  helm-charts:
    kind: github
    scmid: helm-charts
    spec:
      automerge: false
      labels:
        - dependencies

autodiscovery:
  # scmid is applied to all crawlers
  scmid: helm-charts
  crawlers:
    helm:
      # To ignore specific path
      #ignore:
      #  # - path: <filepath relative to scm repository>
      #  # - path: chart/*
      #only:
      #  # - path: <filepath relative to scm repository>
      #  # - path: chart/*

Then run updatecli diff --experimental --config updatecli.d/autodiscovery.yaml

To see what would have to change.

I still believe that we need to execute more custom pipeline in parallel, hence the reason to allow to opt out using the ignore/only configuration. But it's not yet clear how much configuration I need from the autodiscovery versus writing the source/condition/target manifest

[hairmare]

Hey @olblak

Thanks very much for you input. I have in fact been keeping an eye out for the autodiscovery feature as i have a bunch of additional repos that will probably benefit from it.

I have been experimenting with kind: github and it seems to do what we need from it. I'll certainly take autodiscovery for a spin and check if it works for our use case.

What make our use case special seems to be that we want to update versions in random values in a helm chart rather than managing dependencies.

One area where i still see potential for improvement is when it comes to generating descriptions for PRs. In the best case i'd like to configure updatecli so it does something roughly similar to how dependabot adds autogenerated changelogs to it's PR descriptions. I'm still groking the updatecli code and plan on looking into maybe implementing something in this direction once i understand it.

[olblak]

What make our use case special seems to be that we want to update versions in random values in a helm chart rather than managing dependencies.

Tell me if I am wrong, but what you want to do is to update a random key such as in the following file where you want to automatically bump the value of the tag based on a value coming from let say a git tag, or a github release.

yourApp:
     image: ghcr.io/adfinis/yourApp
     tag: 1.0.0

If it's the case, then we do that all the time.
In this example

We want to update UI docker image version used by the Helm chart as soon as a developer publish a new tag but only if a docker image with the corresponding tag as been published.

The source information is a git tag, the target to update is a helm chart value. The kind "helmchart" provides more settings specific to helm chart update than the kind "yaml"

Something that is maybe not clear enough is that every condition or target spec has a default field set to the source out.
So for example this line could be removed. I guess it was there because at some point we had something like "'{{ source "ui-backend-tag" }}-beta"

One area where i still see potential for improvement is when it comes to generating descriptions for PRs. In the best case i'd like to configure updatecli so it does something roughly similar to how dependabot adds autogenerated changelogs to it's PR descriptions. I'm still groking the updatecli code and plan on looking into maybe implementing something in this direction once i understand it.

That's something difficult to solve because depending on the source the changelog is not alway available. So for example github releases usually have a changelog but from a docker registry, it can be difficult to identify... I have been scratching my head around this for a while, and I think we need some kind of standartisation. At the moment I use the pullrequest setting name "description" to provide custom message in my PR body.

Every Updatecli must implement the changelog interface as here but most of the time there is no standard way to retrieve that information.