Fix: Import data not considering the initiated cutover during resumption #2369
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
priyanshi-yb
changed the title
…ether cutover detected by importer and use it
makalaaneesh
approved these changes
Mar 5, 2025
There was a problem hiding this comment.
LGTM! @sanyamsinghal can you also pls take a look. Since this is a critical part, want two sets of eyes on this!
sanyamsinghal
approved these changes
Mar 5, 2025
| CutoverToSourceProcessedBySourceImporter bool `json:"CutoverToSourceProcessedBySourceImporter"` | ||
|
|
||
| //All the cutover detectedc flags by importer when they processed the cutover event |
There was a problem hiding this comment.
what does detected mean here? does it mean that cutover is complete?
I think that is not the case. processed flags are those.
There was a problem hiding this comment.
please make the comments a bit more descriptive in that case.
Minor nit: order of variables here
- Requested
- Detected
- Processed
There was a problem hiding this comment.
what does detected mean here? does it mean that cutover is complete?
It means that the cutover is detected by the importer command by receiving the cutover event in the queue.
| return err | ||
| } | ||
| if ok { | ||
| return nil |
There was a problem hiding this comment.
add a log line before return
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Describe the changes in this pull request
Fix - During the resumption of import data, check if the cutover is initiated before continuing the streaming. If cutover is initiated, but all the events are not processed by importer yet, on resumption, need to check if cutover event is processed by importer or not (reading the last processed queue segment by that importer and checking the last Event in that segment)
Describe if there are any user-facing changes
In resumption case of import data, it will only continue in the case there are some events left to ingest and cutover is not initiated.
How was this pull request tested?
Manually tested all the three importers (to target, to source, and to source-replica)
Does your PR have changes that can cause upgrade issues?