Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

🌱 Add KubeVirt support to Tilt dev workflow #11697

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

🌱 Add KubeVirt support to Tilt dev workflow #11697

wants to merge 2 commits into from
+321 −70

Conversation

johananl
Copy link
Member

@johananl johananl commented Jan 17, 2025 •
edited
Loading

What this PR does / why we need it:

This PR adds support for local development using KubeVirt as an alternative to CAPD.

This is useful in cases where CAPD can't be used for whatever reason, with one example being developing Ignition-related features (since Ignition runs in early boot and therefore can't be containerized easily).

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged): None

/area devtools

@k8s-ci-robot k8s-ci-robot added the area/devtools Issues or PRs related to devtools label Jan 17, 2025
@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Jan 17, 2025
@k8s-ci-robot k8s-ci-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Jan 17, 2025
@johananl johananl changed the title Add KubeVirt instructions to tilt.md 📖 Add KubeVirt instructions to tilt.md Jan 17, 2025
@sbueringer
Copy link
Member

Is there someone who is familiar with KubeVirt that could do a first review?

@nunnatsa
Copy link
Contributor

nunnatsa commented Jan 22, 2025 •
edited
Loading

Hi @johananl

Thanks for this PR. As far as I can see, this is based on the KubeVirt sections in the quickstart guide.

I just opened a PR to update it, as it didn't work for me now (I was working when I wrote it...). Maybe you want to take a look?

#11734

fabriziopandini
fabriziopandini reviewed Jan 27, 2025
View reviewed changes
Copy link
Member

@fabriziopandini fabriziopandini left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for writing this down.
It will be great if we can do a further step and re-use the existig scrpt for creating kind + registry as well as create a new script for automating most of the steps described in this doc

@johananl johananl force-pushed the tilt-kubevirt branch 9 times, most recently from a560441 to 7bbd562 Compare January 30, 2025 15:01
@johananl
Copy link
Member Author

@fabriziopandini I've refactored the PR and converted most of the instructions to code. We now have make kind-cluster-kubevirt which does almost everything. I've edited the guide accordingly.

Thanks for the suggestion 👍

@johananl johananl changed the title 📖 Add KubeVirt instructions to tilt.md 🌱 Add KubeVirt support to Tilt dev workflow Jan 30, 2025
mboersma
mboersma approved these changes Jan 31, 2025
View reviewed changes
Copy link
Contributor

@mboersma mboersma left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

The changes make sense to me. I tried this out with make kind-cluster-kubevirt and everything seems to work as advertised in Tilt. I did make serve to follow through the docs, and I think they're clear and the tab formatting works fine.

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 31, 2025
@k8s-ci-robot
Copy link
Contributor

LGTM label has been added.

Git tree hash: b986b25002289d3e30b1412e76e01616859efe9d

@k8s-ci-robot k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Feb 4, 2025
@k8s-ci-robot k8s-ci-robot requested a review from mboersma February 4, 2025 10:51
@sbueringer sbueringer added the tide/merge-method-squash Denotes a PR that should be squashed by tide when it merges. label Feb 4, 2025
@sbueringer
Copy link
Member

Thx!

/lgtm

/assign @nunnatsa @fabriziopandini

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Feb 4, 2025
@k8s-ci-robot
Copy link
Contributor

LGTM label has been added.

Git tree hash: 7ed72e93e4c435b3aa65698c9f8e2b0d42626dff

chrischdi
chrischdi reviewed Feb 4, 2025
View reviewed changes
hack/kind-install-for-capk.sh
Comment on lines +38 to +60
kubectl apply -f \
"https://raw.githubusercontent.com/projectcalico/calico/${CALICO_VERSION}/manifests/calico.yaml"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question: What feature is missing in Kindnet to require calico?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quoting from the quickstart:

KubeVirt is a cloud native virtualization solution. The virtual machines we’re going to create and use for the workload cluster’s nodes, are actually running within pods in the management cluster. In order to communicate with the workload cluster’s API server, we’ll need to expose it. We are using Kind which is a limited environment. The easiest way to expose the workload cluster’s API server (a pod within a node running in a VM that is itself running within a pod in the management cluster, that is running inside a Docker container), is to use a LoadBalancer service.

To allow using a LoadBalancer service, we can’t use the kind’s default CNI (kindnet), but we’ll need to install another CNI, like Calico.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure how calico is able to do LoadBalancer services, I guess metallb is doing that instead for the kubevirt setup? Normally the CPI does so

FYI: kind also has a CPI: https://kind.sigs.k8s.io/docs/user/loadbalancer

Copy link
Member Author

@johananl johananl Feb 4, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, MetalLB handles the LoadBalancer services. I took these instructions almost verbatim from the official CAPI quickstart (check the KubeVirt tabs).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To allow using a LoadBalancer service, we can’t use the kind’s default CNI (kindnet), but we’ll need to install another CNI, like Calico.

Nit: so that doc is simply not true, because metallb is doing the LoadBalancer service job. I guess metallb does not work with kindnet and that may be the reason at the end.

Feel free to resolve here, I'm okay with that. However, might be good to clarify the docs :-)

Copy link
Member Author

@johananl johananl Feb 21, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't looked into this in depth as my focus here is to contribute a KubeVirt-based dev workflow. IIUC, the need to swap the default CNI with Calico is due to the fact that there is some conflict between the default CNI and MetalLB.

Please feel free to look into this and open a PR to fix the quickstart if you find a mistake there.

@k8s-ci-robot k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Feb 4, 2025
@k8s-ci-robot
Copy link
Contributor

New changes are detected. LGTM label has been removed.

@johananl johananl force-pushed the tilt-kubevirt branch 4 times, most recently from 56bb5e7 to fe92230 Compare February 4, 2025 14:56
chrischdi
chrischdi reviewed Feb 18, 2025
View reviewed changes
Copy link
Member

@chrischdi chrischdi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just some comments on the doc.

Great work on the bash scripts!

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@johananl
Copy link
Member Author

I think I've addressed all the feedback. Happy to get a final review 🙏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@chrischdi chrischdi chrischdi left review comments

@jackfrancis jackfrancis Awaiting requested review from jackfrancis

@richardcase richardcase Awaiting requested review from richardcase

@mboersma mboersma Awaiting requested review from mboersma

@fabriziopandini fabriziopandini Awaiting requested review from fabriziopandini

@nunnatsa nunnatsa Awaiting requested review from nunnatsa

@sbueringer sbueringer Awaiting requested review from sbueringer

Assignees

@mboersma mboersma

@sbueringer sbueringer

@fabriziopandini fabriziopandini

@nunnatsa nunnatsa

Labels
area/devtools Issues or PRs related to devtools cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. tide/merge-method-squash Denotes a PR that should be squashed by tide when it merges.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@johananl @sbueringer @nunnatsa @k8s-ci-robot @mboersma @chrischdi @fabriziopandini