Mix manager should set git versioning git dependencies

[1oglop1]

What would you like help with?

I think I found a bug

How are you running Renovate?

Self-hosted

If you're self-hosting Renovate, tell us which platform (GitHub, GitLab, etc) and which version of Renovate.

Gituhb 38.101.1

Please tell us more about your question or problem

This is a follow-up on Bad Tag name for git dependencies in Elixir project.

Problem

It looks like the mix manager handles git dependencies incorrectly and when "datasource": "git-tags", "versioning": "hex" renovate runs the hex versioning and reaches this line

renovate/lib/modules/versioning/hex/index.ts

Line 104 in 8b2d1fa

newSemver = `== ${newSemver}`;

As far as I understand mix does not support version ranges for git dependencies. And it looks like the extract.ts implementation is completely wrong.
Eg. ref: can be a tag, branch or digest. But the current implementation treats it as digest and replaces tag with SHA which is probably not the desired solution.

It produces an invalid newVersion from v.0.1 -> == v0.2 which fails to download the tag (because it does not exist) and mix deps.update fails.

Fix proposal

I think the fix can follow the logic of the existing Poetry manager, where it sets versioning to semver in case the dependency is a type of semver

renovate/lib/modules/manager/poetry/schema.ts

Lines 133 to 135 in 8b2d1fa

	if (dep.datasource === GitRefsDatasource.id && dep.currentDigest) {
	dep.versioning = gitVersioning.id;
	return dep;

Here are the use cases the mix manager should cover

    {:basic_dep, git: "https://github.com/user/repo.git"},  // skip not enough data
    {:branch_dep, git: "https://github.com/user/repo.git", branch: "main"}, // if semver support, else skip
    {:commit_dep, git: "https://github.com/user/repo.git", ref: "abc123"}, // if semver support, else skip
    {:make_dep, git: "https://github.com/user/repo.git", ref: "abc123", manager: :make}, // skip mix deps.update cannot run custom managers and rebar is not implemented
    {:tag_dep, git: "https://github.com/user/repo.git", tag: "v1.0.0"}, // if semver support, else skip

After doing more tests, it turns out the mix.exs file changes in an unexpected way, and for some reason mix.lock is not changed at all
https://github.com/jans-forks/renovate-reporoduction-31778/pull/3/files

I've made some modifications to the code and achieved this.
https://github.com/jans-forks/renovate-playground/compare/main..e5f0dce01dd3b321dd3d2e46c7bf09605cead7ed
I am still not sure about the desired behaviour for branch where renovate has changed ref: branch-name -> ref: SHA1, but I think this should only happen in a lock file not the mix.exs.

Besides the logic fix, I propose to update the docs page https://docs.renovatebot.com/modules/manager/mix/
To mention that git dependencies may require manager process may require extra git credentials. Which I achieved with customenvvariables.

Reproduction repo

• https://github.com/jans-forks/renovate-reporoduction-31778

Logs (if relevant)

Logs

DEBUG: branches info extended (repository=jans-forks/renovate-reporoduction-31778)
       "branchesInformation": [
         {
           "branchName": "renovate/mix",
           "prNo": 3,
           "prTitle": "Update mix to v0.2.0",
           "result": "done",
           "upgrades": [
             {
               "datasource": "git-tags",
               "depName": "git_lib_pkg",
               "displayPending": "",
               "fixedVersion": "v0.1.0",
               "currentVersion": "v0.1.0",
               "currentValue": "v0.1.0",
               "newValue": "v0.2.0",
               "newVersion": "v0.2.0",
               "packageFile": "el_app/mix.exs",
               "updateType": "minor",
               "packageName": "https://github.com/jans-forks/elixir_lib"
             },
             {
               "datasource": "git-tags",
               "depName": "lib_pkg",
               "displayPending": "",
               "fixedVersion": "v0.1.0",
               "currentVersion": "v0.1.0",
               "currentValue": "v0.1.0",
               "newValue": "== v0.2.0",
               "newVersion": "v0.2.0",
               "packageFile": "el_app/mix.exs",
               "updateType": "minor",
               "packageName": "https://github.com/jans-forks/elixir_lib"
             }
           ]
         }
       ]

[github-actions[bot]]

Hi there,

Get your discussion fixed faster by creating a minimal reproduction. This means a repository dedicated to reproducing this issue with the minimal dependencies and config possible.

Before we start working on your issue we need to know exactly what's causing the current behavior. A minimal reproduction helps us with this. Discussions without reproductions are less likely to be converted to Issues.

Please follow these steps:

1. Read our guide on creating a minimal reproduction.
2. Go to our minimal reproduction template repository.
3. Select the Use this template button to create a new repository based on the template.
4. Work on the minimal reproduction in your own repository.
5. Fill out the information in your repository's README.md.
6. Add the link to your reproduction to the first post of your Discussion. If you are not the original author, you can post a new comment with the link.

If you need help with running renovate on your minimal reproduction repository, please refer to our Running Renovate guide.
Good luck,

The Renovate team

[1oglop1]

@rarkins please lemme know if there is something else I am missing.

[1oglop1]

It seems to reproduce, but now needs dependencies reduced to minimal. The other dependencies don't appear to be necessary

I am not sure what you mean, there is only a single dependency on my dummy elixir package, but it is mentioned multiple times to reproduce all/most possible cases.

Would you suggest a different way of doing so?

[rarkins]

Remove all the other dependencies

[1oglop1]

I'm sorry but I am confused by your answers. What do you mean by "other dependencies", all lines in the link are relevant to the problem. Could you clearly point out what would you like to remove?
I am trying to help but I do not think that this process makes it any easier.
https://github.com/jans-forks/renovate-reporoduction-31778/blob/main/el_app/mix.exs#L22-L28

[rarkins]

Your reproduction has 6 PRs. Why are so many needed to demonstrate one problem?

[1oglop1]

It was one PR (with a group) reproducing a complex problem, but then you complained about too much config. The behaviour of mix manager is quite complex and the current buggy solution is simplistic and naive while not considering possible combinations. So fixing one combination can break the other.

It is possible to use different notation for git entries in mix.exs:
tag: "v0.2.0", ref: "v0.1.0" but ref can also be digest, tag or even a branch name.

They are then represented in the mix.lock like this
SHA1, [reference_type]

"284def38b9c7909f9eb99d5c19b7bb7eb80f90ef", [tag: "v0.2.0"]
"4888c25b27e47369296ebd66502a3d3ec6e955db", [ref: "v0.1.0"]
"83b72d4940b13cc68e70a32aa14eb624b29b880a", [ref: "main"]
"4888c25b27e47369296ebd66502a3d3ec6e955db", [ref: "4888c25b27e47369296ebd66502a3d3ec6e955db"]

in the end the the mapping between mix.exs -> mix.lock is

tag: "v0.2.0" -> "284def38b9c7909f9eb99d5c19b7bb7eb80f90ef", [tag: "v0.2.0"]
ref: "v0.1.0" -> "4888c25b27e47369296ebd66502a3d3ec6e955db", [ref: "v0.1.0"]
ref: "main" -> "83b72d4940b13cc68e70a32aa14eb624b29b880a", [ref: "main"]
ref: "4888c25b27e47369296ebd66502a3d3ec6e955db" -> "4888c25b27e47369296ebd66502a3d3ec6e955db", [ref: "4888c25b27e47369296ebd66502a3d3ec6e955db"]

The current behaviour incorrectly evaluates tags that look like a version and changes ref from anything to digest inmix.exs this is all covered in the PR example.
I believe this is incorrect, and this behaviour should have more logic.
Ideally, the change should be propagated in both mix.exs and mix.lock.
So I'd like to understand more about the strategies or behaviour patterns of managers.
One strategy I do not fully understand is update-lockfile introduced in #27924

I'd like to discuss possible approaches for implementation

[kivra-pauoli]

Is there a renovate pull request in this repository, that I could look at? I searched for one before opening #32539, but maybe I didn't use the right keywords (?)

[1oglop1]

@1oglop1, you'd mention that potentially the Mix documentation was wrong. Has that been addressed? The problem being that if the source of truth is incorrect it'll be difficult to "fix" something. Thanks for looking into it.

It is incomplete rather than incorrect. I could not find anything about mix.lock file and how it is created from deps.

[bernardo-martinez]

@1oglop1 probably it should set currentValue with either ref or branchOrTag depending on whats available, and then let git versioning compute the currentDigest? Even if ref is commit SHA, it should work?

[rarkins]

currentValue/currentDigest need to be extracted correctly by the manager. They are not determined by versioning

[paulo-ferraz-oliveira]

I could not find anything about mix.lock file and how it is created from deps.

I was under the impression that Renovate ran mix lock after it updated mix.exs. Is this not the case?

[rarkins]

That's the case. But that's also unrelated to versioning

[1oglop1]

@rarkins @paulo-ferraz-oliveira as discussed, I am marking this as resolved and will follow up in the next discussion.

#32815