Auto migrate datanodes to real chunks v2 #457
Open
+205
−105
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fix #396
The issue For successful master update to 24.2 it is required there are no active datanodes registered in cypress with
use_imaginary_chunk_locations=%true
, otherwise master couldn't be started.To achieve it on existing cluster, one must set
//sys/@config/node_tracker/enable_real_chunk_locations=%true
and force datanodes with imaginary chunks location to unregister at master (by restarting them or somehow else). After that datanodes will re-register and haveuse_imaginary_chunk_locations=%false
, but for successful master update only absence of active nodes withuse_imaginary_chunk_locations=%true
is required.Previous implementation only set
//sys/@config/node_tracker/enable_real_chunk_locations=%true
and doesn't restart datanodes. It is removed in this PR.New implementation adds new update flow status
WaitingForImaginaryChunksAbsence
, which is automatically added if master or data nodes are updated. For this status:enable_real_chunk_locations==%false
— it sets it to true, otherwise proceedstatus=online && use_imaginary_chunk_locations=%true
— it setsDataNodesNeedPodsRemoval
condition and wait until there are no nodes withstatus=online && use_imaginary_chunk_locations=%true
. Otherwise or after condition is satisfied it is proceed to master update (or datanodes update if master excluded from update by updatePlan/updateSelector).DataNodesNeedPodsRemoval
condition, it removes its pods (which ultimately leads to unregistering datanodes in master). The pods will be re-created (with possible image update if needed) atWaitingForPodsCreation
stage later as usual.