Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hash-map based GROUP BY can now process lazy inputs #1651

Merged
merged 7 commits into from
Jan 4, 2025

Conversation

joka921
Copy link
Member

@joka921 joka921 commented Nov 29, 2024

Since #1229, QLever supports a hash-map based GROUP BY, which can be activated via the runtime parameter group-by-hash-map-enabled (which is false by default). So far, this required a fully materialized input. With this change, the input can be processed lazily. The output is still fully materialized and never lazy (because, when the input is not sorted, we cannot output any group until we have seen the last input row).

Signed-off-by: Johannes Kalmbach <[email protected]>
Signed-off-by: Johannes Kalmbach <[email protected]>
Signed-off-by: Johannes Kalmbach <[email protected]>
Signed-off-by: Johannes Kalmbach <[email protected]>
Copy link

codecov bot commented Dec 18, 2024

Codecov Report

Attention: Patch coverage is 92.75362% with 5 lines in your changes missing coverage. Please review.

Project coverage is 89.86%. Comparing base (97c195a) to head (bbed6e7).
Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
src/engine/GroupBy.cpp 92.75% 0 Missing and 5 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1651      +/-   ##
==========================================
+ Coverage   89.84%   89.86%   +0.02%     
==========================================
  Files         389      389              
  Lines       37268    37287      +19     
  Branches     4202     4204       +2     
==========================================
+ Hits        33482    33508      +26     
+ Misses       2485     2480       -5     
+ Partials     1301     1299       -2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Member

@hannahbast hannahbast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good and works fine in some test queries we tried. Minor change left.

Signed-off-by: Johannes Kalmbach <[email protected]>
Copy link
Member

@hannahbast hannahbast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

As a follow-up, we should find a way to enable this automatically, in particular, for queries like SELECT ?g (COUNT(?g) AS ?count) WHERE { GRAPH ?g { ?s ?p ?o } } GROUP BY ?g

@hannahbast hannahbast changed the title Make the hash map group by lazy. Make the hash map group by lazy Dec 20, 2024
@hannahbast hannahbast changed the title Make the hash map group by lazy Hash-map based GROUP BY can now process lazy inputs Jan 4, 2025
@sparql-conformance
Copy link

@hannahbast hannahbast merged commit 2953f16 into ad-freiburg:master Jan 4, 2025
22 checks passed
Copy link

sonarqubecloud bot commented Jan 4, 2025

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants