continuous benchmarking #263
Comments
yliang412
added a commit
that referenced
this issue
Dec 10, 2024
…ution (#262) Part of #263. ## Problem There is a lack of performance evaluation in optd. Without proper benchmarking, we will not know if some changes cause significant regression in performance. We should keep track of the two aspects of optimizer performance: - plan generation time - actual execution time ## Summary of changes - Add a [Criterion-rs](https://bheisler.github.io/criterion.rs/book/) based benchmark to measure optd performance. - Use the `sqlplannertest` yaml-based setup, bench if there is a `bench` task. - Bump `sqlplannertest` to version 0.4. - better individual test matching added in risinglightdb/sqlplannertest-rs#9 Planning: - Input: SQL statement (aka `datafusion_sql::parser::Statement`). - Measures the time it takes to generate a datafusion physical plan. Execution: - Input: datafusion physical plan outputted from optd (aka `Arc<dyn ExecutionPlan>`). - Measures the time it takes to execute the entire query. ## Next Steps https://github.com/benchmark-action/github-action-benchmark can directly take `cargo bench` results as input. So it will be easy to set up CI. --------- Signed-off-by: Yuchen Liang <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
We need to keep track of optd's performance.
Tasks
The text was updated successfully, but these errors were encountered: