Skip to content

feat: add perf walltime support #13

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

not-matthias
Copy link
Member

@not-matthias not-matthias commented Jun 25, 2025

Depends on CodSpeedHQ/runner#94

Currently blocked by: CodSpeedHQ/instrument-hooks#6

@GuillaumeLagrange
Copy link
Contributor

GuillaumeLagrange commented Jun 25, 2025

@not-matthias can you fix CI please before I review ? 👼

Bonus question: any idea why the diff is so huge in gh display?
image

@not-matthias
Copy link
Member Author

not-matthias commented Jun 25, 2025

@not-matthias can you fix CI please before I review ? 👼

Yeah, of course!

Bonus question: any idea why the diff is so huge in gh display?

I think it's due to the clang-format pre-commit-hook (mostly valgrind.h). Should I add it to the blacklist? It's not really created by us

This comment was marked as outdated.

Copy link

codspeed-hq bot commented Jun 25, 2025

CodSpeed WallTime Performance Report

Merging #13 will degrade performances by 33.33%

Comparing cod-918-perf-doesnt-work-for-cpp-codspeed (da6c95f) with main (719c41f)

Summary

⚡ 3 improvements
❌ 6 regressions
✅ 33 untouched benchmarks
🆕 16 new benchmarks

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

Benchmark BASE HEAD Change
BarTest[MyFixture] < 1 ns < 1 ns +25%
DoubleTest[MyTemplatedFixture, double] < 1 ns < 1 ns -33.33%
FooTest[MyFixture] < 1 ns < 1 ns +50%
IntTest[MyTemplatedFixture, int] < 1 ns < 1 ns -16.67%
TestA[MyTemplate1, int] < 1 ns < 1 ns -16.67%
TestB[MyTemplate2, int, double] < 1 ns < 1 ns -20%
BM_Capture[int_string_test] < 1 ns < 1 ns -33.33%
BM_rand_vector < 1 ns < 1 ns -20%
🆕 BM_sleep_100ms N/A 100.1 ms N/A
🆕 BM_sleep_100ns N/A 58.6 µs N/A
🆕 BM_sleep_100us N/A 158.7 µs N/A
🆕 BM_sleep_10ms N/A 10.1 ms N/A
🆕 BM_sleep_1ms N/A 1.1 ms N/A
🆕 BM_sleep_1ns N/A 58.6 µs N/A
🆕 BM_sleep_1us N/A 59.6 µs N/A
🆕 BM_sleep_50ms N/A 50.1 ms N/A
BM_Template1_Capture[two_type_test, int, double] < 1 ns < 1 ns +80%
🆕 BM_sleep_100ms N/A 100.1 ms N/A
🆕 BM_sleep_100ns N/A 58.6 µs N/A
🆕 BM_sleep_100us N/A 158.7 µs N/A
... ... ... ... ...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

@not-matthias not-matthias force-pushed the cod-918-perf-doesnt-work-for-cpp-codspeed branch 2 times, most recently from 7e397c3 to a105afb Compare June 25, 2025 17:49
@GuillaumeLagrange
Copy link
Contributor

GuillaumeLagrange commented Jun 26, 2025

@not-matthias can you fix CI please before I review ? 👼

Yeah, of course!

Bonus question: any idea why the diff is so huge in gh display?

I think it's due to the clang-format pre-commit-hook (mostly valgrind.h). Should I add it to the blacklist? It's not really created by us

Absolutely, rule of thumb should be that we never format external files like this, and if possible when we edit external files we make it so our formatter matches the initial style of the file to avoid unneccessary diffs, same for callgrind.h.

@GuillaumeLagrange
Copy link
Contributor

@not-matthias could you make sure to rebase this on top of the windows PR once it's merged ? And then I'll review for real to avoid rework

@adriencaccia adriencaccia requested a review from art049 June 26, 2025 09:35
@not-matthias not-matthias force-pushed the cod-918-perf-doesnt-work-for-cpp-codspeed branch from a105afb to 014fcee Compare June 26, 2025 14:03
Copy link
Member

@art049 art049 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good overall! I think we'll need to be extra careful to make sure this still builds on Windows after #15; especially since we didn't test the instrument hooks on windows yet. I'll let @GuillaumeLagrange do the more thorough review once rebased on the initial PR

@not-matthias not-matthias force-pushed the cod-918-perf-doesnt-work-for-cpp-codspeed branch 2 times, most recently from 190aec9 to c2aa8d6 Compare June 27, 2025 14:25
@not-matthias not-matthias force-pushed the cod-918-perf-doesnt-work-for-cpp-codspeed branch from c2aa8d6 to da25a57 Compare June 27, 2025 14:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants