Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add support for cloning Git repository via SSH rather than HTTPS #300

Draft
wants to merge 23 commits into
base: develop
Choose a base branch
from

Conversation

boegel
Copy link
Contributor

@boegel boegel commented Feb 6, 2025

This is useful when the target repository of PRs that the bot should act on is a private repository.

Opt-in via this in bot configuration file (app.cfg):

[buildenv]
clone_git_repo_via = 'ssh'

(draft PR since this needs actual testing, and perhaps more changes)

tasks/build.py Outdated
@@ -407,6 +422,17 @@ def download_pr(repo_name, branch_name, pr, arch_job_dir):
error_stage = _ERROR_CURL
return curl_output, curl_error, curl_exit_code, error_stage

git_diff_cmd = ' '.join([
f"git fetch origin pull/{pr.number}/head:{pr.number}",
f"git diff HEAD pr{pr.number} > {pr.number}.diff",
Copy link
Contributor

@smoors smoors Feb 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if you're now creating the diff like this, shouldn't you remove the curl_cmd above?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, of course, I'll tackle that too

Copy link
Contributor

@smoors smoors Feb 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

edit: nvm

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@smoors made a good point (in Slack) about this change: this doesn't take into account that there may be changes in the main branch that are not included in the PR branch yet.

So, a better approach would be to:

  • fetch the PR branch
  • check out the PR branch
  • git merge main
  • swap back to main branch
  • git diff
  • git apply

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This procedure hasn't been implemented yet, right?

A few thoughts:

  • main should probably be the target branch of the PR, for example, for EESSI/2023.06 that would be 2023.06-software.eessi.io?
  • the git diff after the swap back to "main" is really just the plain git diff or rather git diff PR_branch?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to determine base branch for PR (the target branch), maybe git merge-base is useful...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll change this so the new approach based on git is only used when clone_via is set to 'ssh' (so nothing changes for existing bots that listen to a public repo).

That gives us some freedom to experiment with the situation where the target branch has been updated since the branch for the PR was created.

Copy link
Contributor Author

@boegel boegel Feb 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've updated this in a99ae60 to only use the git diff approach when clone_git_repo_via is used.

It's still doing only git fetch + git diff, so you could still have issues when the PR branch is way behind on the target branch, I'll try to look into that next.

@boegel boegel marked this pull request as ready for review February 6, 2025 16:37
@boegel
Copy link
Contributor Author

boegel commented Feb 6, 2025

This is working as expected now, so marking it ready-for-review.

Hopefully @trz42 has time to take a look?

Copy link
Contributor

@trz42 trz42 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works as intended. See trz42/software-layer#78

The optional settings have to be commented. The intention with listing them, is more to document that they are optional and it could be easily changed in the future if we wish so.

I think, it might be good to add some comments (README.md and app.cfg.example) about what is needed if the ssh key has not a standard name and if it uses a passphrase.

@boegel boegel force-pushed the git_clone_via_ssh branch 2 times, most recently from c03e52b to 299b9e8 Compare February 21, 2025 13:58
@boegel
Copy link
Contributor Author

boegel commented Feb 21, 2025

I haven't actually re-tested this after the changes I've made, and some more changes are required to the git diff approach to take into account that the target branch of a PR may have been updated since the PR was opened, so I'm marking this as work-in-progress for now...

@boegel boegel marked this pull request as draft February 21, 2025 16:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants