Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

🐛 remove disableMachineCreate annotation from new machinesets during rolling machine deployment reconciliation #11415

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

🐛 remove disableMachineCreate annotation from new machinesets during rolling machine deployment reconciliation #11415

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions internal/controllers/machinedeployment/machinedeployment_rolling.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ import (

clusterv1 "sigs.k8s.io/cluster-api/api/v1beta1"
"sigs.k8s.io/cluster-api/internal/controllers/machinedeployment/mdutil"
"sigs.k8s.io/cluster-api/util/patch"
)

// rolloutRolling implements the logic for rolling a new MachineSet.
Expand Down Expand Up @@ -72,6 +73,11 @@ func (r *Reconciler) rolloutRolling(ctx context.Context, md *clusterv1.MachineDe
}

func (r *Reconciler) reconcileNewMachineSet(ctx context.Context, allMSs []*clusterv1.MachineSet, newMS *clusterv1.MachineSet, deployment *clusterv1.MachineDeployment) error {
newMS, err := r.cleanupDisableMachineCreateAnnotation(ctx, newMS)
if err != nil {
return err
}

if deployment.Spec.Replicas == nil {
return errors.Errorf("spec.replicas for MachineDeployment %v is nil, this is unexpected", client.ObjectKeyFromObject(deployment))
}
Expand Down Expand Up @@ -293,3 +299,25 @@ func (r *Reconciler) scaleDownOldMachineSetsForRollingUpdate(ctx context.Context

return totalScaledDown, nil
}

// cleanupDisableMachineCreateAnnotation will remove the disable machine create annotation from new MachineSets that were created during reconcileOldMachineSetsOnDelete.
func (r *Reconciler) cleanupDisableMachineCreateAnnotation(ctx context.Context, newMS *clusterv1.MachineSet) (*clusterv1.MachineSet, error) {
log := ctrl.LoggerFrom(ctx)

if newMS.Annotations != nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this duplicates the removal code. Also it would run twice in the onDelete case so we should be able to remove it there.

Maybe also move it to the top of this function.

Caveat: with doing this it won't be possible (anymore) to manually set this annotation on a MS if it is the new machineset for the normal rollout type. But IMHO this should be okay, because the MD controller owns the MS.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It will never fully run twice since its checking the annotation on the MS if one run removes it the second one shouldn't run because the annotation isn't there. I can add a check before calling the function to check the annotation exists if that helps?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my point is: I think we don't need it at ondelete anymore / we only need this code once :-)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we do need it if a user updates the MD and it creates a new MS the old one gets the annotation added. If the user undoes his changes the controller will attempt to use the old MS with the annotation that will prevent machines from coming up. So I think we need logic for the removal of this annotation in both onDelete and Rolling.

Please correct me if my understanding is wrong.

if _, ok := newMS.Annotations[clusterv1.DisableMachineCreateAnnotation]; ok {
log.V(4).Info("removing annotation on latest MachineSet to enable machine creation")
patchHelper, err := patch.NewHelper(newMS, r.Client)
if err != nil {
return nil, err
}
delete(newMS.Annotations, clusterv1.DisableMachineCreateAnnotation)
err = patchHelper.Patch(ctx, newMS)
if err != nil {
return nil, err
}
}
}

return newMS, nil
}
38 changes: 38 additions & 0 deletions internal/controllers/machinedeployment/machinedeployment_rolling_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -217,6 +217,41 @@ func TestReconcileNewMachineSet(t *testing.T) {
},
error: nil,
},
{
name: "Rolling Updated Cleanup disable machine create annotation",
machineDeployment: &clusterv1.MachineDeployment{
ObjectMeta: metav1.ObjectMeta{
Namespace: "foo",
Name: "bar",
},
Spec: clusterv1.MachineDeploymentSpec{
Strategy: &clusterv1.MachineDeploymentStrategy{
Type: clusterv1.RollingUpdateMachineDeploymentStrategyType,
RollingUpdate: &clusterv1.MachineRollingUpdateDeployment{
MaxUnavailable: intOrStrPtr(0),
MaxSurge: intOrStrPtr(0),
},
},
Replicas: ptr.To[int32](2),
},
},
newMachineSet: &clusterv1.MachineSet{
ObjectMeta: metav1.ObjectMeta{
Namespace: "foo",
Name: "bar",
Annotations: map[string]string{
clusterv1.DisableMachineCreateAnnotation: "true",
clusterv1.DesiredReplicasAnnotation: "2",
clusterv1.MaxReplicasAnnotation: "2",
},
},
Spec: clusterv1.MachineSetSpec{
Replicas: ptr.To[int32](2),
},
},
expectedNewMachineSetReplicas: 2,
error: nil,
},
}

for _, tc := range testCases {
Expand Down Expand Up @@ -252,6 +287,9 @@ func TestReconcileNewMachineSet(t *testing.T) {

g.Expect(*freshNewMachineSet.Spec.Replicas).To(BeEquivalentTo(tc.expectedNewMachineSetReplicas))

_, ok := freshNewMachineSet.GetAnnotations()[clusterv1.DisableMachineCreateAnnotation]
g.Expect(ok).To(BeFalse())

desiredReplicasAnnotation, ok := freshNewMachineSet.GetAnnotations()[clusterv1.DesiredReplicasAnnotation]
g.Expect(ok).To(BeTrue())
g.Expect(strconv.Atoi(desiredReplicasAnnotation)).To(BeEquivalentTo(*tc.machineDeployment.Spec.Replicas))
Expand Down