Gitlab CI_JOB_TOKEN won't work in addition to Personal Access Token

[mnikhil21]

How are you running Renovate?

Self-hosted Renovate

If you're self-hosting Renovate, tell us which platform (GitHub, GitLab, etc) and which version of Renovate.

Gitlab, renovateVersion - 39.26.3

Please tell us more about your question or problem

I have setup a renovate project to monitor a repo (my-group/myrepo). I'm using personal access token with api, read_repository and write_repositories permissions so that renovate can create merge requests. The renovate bot is meant to be tracking terraform module dependencies.

We do not have a terraform "registry" as such. The terraform modules are hosted in different gitlab repos (cloudplatform/<module name>). While my PAT doesn't have access to these repos, the CI_JOB_TOKEN of my CI pipeline has read access to all the module repos.

I have set RENOVATE_TOKEN = ${PAT} in my CI file. MR creation works fine for public dependencies.

And I'm trying to setup renovate.json like so:

{
  "hostRules": [
    {
      "matchHost": "https://git.example.com/cloudplatform/",
      "token": "${CI_JOB_TOKEN}"
    }
  ]
} 

However, renovate isn't able to read the terraform dependencies. I keep receiving 401 Unauthorized error.

I also ran curl --header "Authorization: Bearer ${CI_JOB_TOKEN} "<sample module url> and it gave me successful result. So the token works. I'm just not able to use it in my renovate config.

Question: Is my use case even possible? I read in another thread about "one pipeline, one token" - #30013

Or if my use case is indeed achievable, pls let me know how I can setup authentication.
Thank you!

Logs (if relevant)

Logs

Replace this text with your logs, between the starting and ending triple backticks

[rarkins]

JSON files don't have access to env directly like you're trying. You could put this rule in a config.js file and then be able to access process.env that way.

[mnikhil21]

I tried config.js as well - but I'm still getting 401 error.

This is the hostRule I put in my config.js

..
..
..
hostRules: [
    {
      hostType: 'gitlab',
      matchHost: 'https://git.example.com/api/v4/packages/terraform/modules/v1/cloudplatform',
      token: process.env.CI_JOB_TOKEN
      ..
      ..
      ..,
    }
  ],

my ci file has this entry:
CI_JOB_TOKEN: ${CI_JOB_TOKEN}

[mnikhil21]

FYI - I added that matchHost entry because renovate converts the package url into API based host. I had previously tried setting the matchHost to "https://git.example.com/cloudplatform/" - but I still got the 401 error.

[rarkins]

Please provide full debug logs of a run

[github-actions[bot]]

Hi there,

This issue or discussion is missing some logs, making it difficult or impossible to help you. Depending on which situation applies follow one, some or all of these instructions.

No logs at all

If you haven't posted any log yet, we need you to find and copy/paste the log into the issue template.

Finding logs on hosted app

Select me to read instructions

If you use the Mend Renovate app (GitHub):

1. Log in to the Mend Developer Portal
2. Navigate to the correct organization and repository
3. Locate the appropriate log (it may not always be the latest one)
4. Copy/paste the log contents
5. Follow the steps in the formatting your logs section

Finding logs when self-hosting

Select me to read instructions

Read the Renovate docs, troubleshooting, self-hosted to learn how to find the logs.

Insufficient logs

Select me to read instructions

If you already gave us a log, and the Renovate team said it's not enough, then follow the instructions from the No logs at all section.

Formatting your logs

Select me to read instructions

Please put your logs in a <details> and <summary> element like this:

<details><summary>Select me to see logs</summary>

```
Copy/paste your logs here, between the starting and ending backticks
```

</details>

If you feel the logs are too large to paste here, please use a service like GitHub Gist and paste the link here.

Good luck,

The Renovate team