-
Notifications
You must be signed in to change notification settings - Fork 57
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Avoid reading and decompressing index blocks during query planning #1725
Avoid reading and decompressing index blocks during query planning #1725
Conversation
Signed-off-by: Johannes Kalmbach <[email protected]>
Signed-off-by: Johannes Kalmbach <[email protected]>
Signed-off-by: Johannes Kalmbach <[email protected]>
Conformance check passed ✅No test result changes. |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #1725 +/- ##
==========================================
+ Coverage 89.86% 89.93% +0.06%
==========================================
Files 389 393 +4
Lines 37308 37611 +303
Branches 4204 4231 +27
==========================================
+ Hits 33527 33824 +297
- Misses 2485 2490 +5
- Partials 1296 1297 +1 ☔ View full report in Codecov by Sentry. |
Signed-off-by: Johannes Kalmbach <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
1-1 with Johannes, looks great, some minor improvements in clarity requested
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Awesome!
Signed-off-by: Johannes Kalmbach <[email protected]>
Quality Gate passedIssues Measures |
Avoid reading and decompressing the first and last block of an index scan during query planning (which requires only a size estimate and not the exact size). This added significant performance overhead, when only few blocks were eventually read in the index scan. Instead, we now do the following: First check whether the first and last block fully match the scan specification (this is fast). Second, if this is not the case, assume that the number of elements is a fixed fraction of total number of elements in the block (which we know from the metadata). The fixed fraction is defined by the new runtime parameter
small-index-scan-size-estimate-divisor
.