Skip to content

File Reader | Add next_line_file_offset #8953

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

romayalon
Copy link
Contributor

@romayalon romayalon commented Apr 9, 2025

Describe the Problem

On #8923, we would like to preserve the next_line_file_offset in order to start reading in the next iteration from the middle of the file and not read the whole file again.
Moved the fileReader changes from 8923 to a new PR.

Explain the Changes

  1. Added next_line_file_offset to file Reader.
  2. Added unit tests.

Issues: Fixed #xxx / Gap #xxx

Testing Instructions:

  1. sudo jest --testRegex=jest_tests/test_newline_reader --t "next_line_file_offset"
  • Doc added/updated
  • Tests added

@romayalon romayalon force-pushed the romy-add-total-lines-bytes-offset-to-file-reader branch 2 times, most recently from 509414c to 7587425 Compare April 9, 2025 11:16
@pull-request-size pull-request-size bot added size/S and removed size/XS labels Apr 9, 2025
@romayalon romayalon changed the title File Reader | Add line_bytes_offset File Reader | Add next_line_file_offset Apr 9, 2025
@romayalon romayalon force-pushed the romy-add-total-lines-bytes-offset-to-file-reader branch from 7587425 to b223ddb Compare April 9, 2025 11:57
@romayalon romayalon requested a review from nadavMiz April 9, 2025 13:56
@romayalon romayalon force-pushed the romy-add-total-lines-bytes-offset-to-file-reader branch from b223ddb to 2b24711 Compare April 14, 2025 06:27
@romayalon romayalon requested a review from tangledbytes April 14, 2025 06:27
Copy link
Member

@tangledbytes tangledbytes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@romayalon romayalon force-pushed the romy-add-total-lines-bytes-offset-to-file-reader branch from 2b24711 to 2116887 Compare April 14, 2025 08:44
@romayalon romayalon merged commit 8f1c3b4 into noobaa:master Apr 14, 2025
11 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants