Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test(sample): Add E2E tests for app start transactions #4619

Draft
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

krystofwoldrich
Copy link
Member

@krystofwoldrich krystofwoldrich commented Mar 4, 2025

📢 Type of change

  • Bugfix
  • New feature
  • Enhancement
  • Refactoring

📜 Description

💡 Motivation and Context

💚 How did you test it?

📝 Checklist

  • I added tests to verify changes
  • No new PII added or SDK only sends newly added PII if sendDefaultPII is enabled
  • I updated the docs if needed.
  • I updated the wizard if needed.
  • All tests passing
  • No breaking changes

🔮 Next steps

#skip-changelog

@krystofwoldrich krystofwoldrich changed the title Add detox test test(sample): Add E2E tests for app start transactions Mar 4, 2025
Copy link
Contributor

github-actions bot commented Mar 4, 2025

Android (new) Performance metrics 🚀

  Plain With Sentry Diff
Startup time 416.74 ms 409.22 ms -7.51 ms
Size 7.15 MiB 8.39 MiB 1.23 MiB

Baseline results on branch: main

Startup times

Revision Plain With Sentry Diff
4161236+dirty 429.89 ms 498.74 ms 68.85 ms
6e8584e+dirty 383.37 ms 400.84 ms 17.47 ms
b1e8712+dirty 322.55 ms 331.84 ms 9.29 ms
e99226d+dirty 413.20 ms 460.37 ms 47.17 ms
8bda0cc+dirty 383.08 ms 417.40 ms 34.32 ms
3853f43+dirty 278.12 ms 338.72 ms 60.60 ms
2534337+dirty 597.14 ms 665.04 ms 67.90 ms
80e955a+dirty 399.50 ms 453.16 ms 53.66 ms
8ba5377+dirty 361.68 ms 370.62 ms 8.94 ms
fe13591+dirty 539.51 ms 597.92 ms 58.40 ms

App size

Revision Plain With Sentry Diff
4161236+dirty 7.15 MiB 8.38 MiB 1.23 MiB
6e8584e+dirty 7.15 MiB 8.13 MiB 1002.18 KiB
b1e8712+dirty 7.15 MiB 8.04 MiB 912.27 KiB
e99226d+dirty 7.15 MiB 8.38 MiB 1.23 MiB
8bda0cc+dirty 7.15 MiB 8.38 MiB 1.23 MiB
3853f43+dirty 7.15 MiB 8.08 MiB 959.34 KiB
2534337+dirty 7.15 MiB 8.11 MiB 988.68 KiB
80e955a+dirty 7.15 MiB 8.38 MiB 1.23 MiB
8ba5377+dirty 7.15 MiB 8.39 MiB 1.23 MiB
fe13591+dirty 7.15 MiB 8.35 MiB 1.20 MiB

Previous results on branch: add-detox-test

Startup times

Revision Plain With Sentry Diff
922b103+dirty 387.53 ms 391.51 ms 3.98 ms

App size

Revision Plain With Sentry Diff
922b103+dirty 7.15 MiB 8.39 MiB 1.23 MiB

Copy link
Contributor

github-actions bot commented Mar 4, 2025

iOS (legacy) Performance metrics 🚀

  Plain With Sentry Diff
Startup time 1235.82 ms 1233.81 ms -2.00 ms
Size 2.63 MiB 3.75 MiB 1.12 MiB

Baseline results on branch: main

Startup times

Revision Plain With Sentry Diff
8bda0cc+dirty 1221.90 ms 1208.11 ms -13.79 ms
80b2ce3+dirty 1265.92 ms 1268.60 ms 2.69 ms
31fcca2+dirty 1209.17 ms 1216.21 ms 7.04 ms
db44eaf+dirty 1227.09 ms 1230.65 ms 3.56 ms
2ec71da+dirty 1225.85 ms 1231.57 ms 5.72 ms
e2b64fe+dirty 1232.22 ms 1255.20 ms 22.98 ms
8ab11b6+dirty 1203.23 ms 1206.02 ms 2.79 ms
1c65324+dirty 1235.17 ms 1235.08 ms -0.09 ms
a0b0298+dirty 1220.73 ms 1226.87 ms 6.14 ms
83f6f6c+dirty 1218.33 ms 1220.27 ms 1.93 ms

App size

Revision Plain With Sentry Diff
8bda0cc+dirty 2.63 MiB 3.70 MiB 1.06 MiB
80b2ce3+dirty 2.36 MiB 2.84 MiB 486.98 KiB
31fcca2+dirty 2.36 MiB 2.90 MiB 552.95 KiB
db44eaf+dirty 2.36 MiB 3.10 MiB 755.81 KiB
2ec71da+dirty 2.36 MiB 3.13 MiB 784.66 KiB
e2b64fe+dirty 2.36 MiB 2.85 MiB 495.80 KiB
8ab11b6+dirty 2.36 MiB 3.11 MiB 759.83 KiB
1c65324+dirty 2.36 MiB 3.04 MiB 698.64 KiB
a0b0298+dirty 2.63 MiB 3.75 MiB 1.12 MiB
83f6f6c+dirty 2.36 MiB 3.10 MiB 759.79 KiB

Previous results on branch: add-detox-test

Startup times

Revision Plain With Sentry Diff
922b103+dirty 1233.41 ms 1226.48 ms -6.93 ms

App size

Revision Plain With Sentry Diff
922b103+dirty 2.63 MiB 3.75 MiB 1.12 MiB

Copy link
Contributor

github-actions bot commented Mar 4, 2025

iOS (new) Performance metrics 🚀

  Plain With Sentry Diff
Startup time 1232.92 ms 1230.49 ms -2.43 ms
Size 3.19 MiB 4.32 MiB 1.13 MiB

Baseline results on branch: main

Startup times

Revision Plain With Sentry Diff
8bda0cc+dirty 1217.90 ms 1223.02 ms 5.12 ms
80b2ce3+dirty 1245.12 ms 1262.04 ms 16.92 ms
31fcca2+dirty 1222.04 ms 1226.51 ms 4.47 ms
db44eaf+dirty 1238.49 ms 1236.56 ms -1.93 ms
2ec71da+dirty 1230.29 ms 1239.50 ms 9.21 ms
e2b64fe+dirty 1285.78 ms 1297.56 ms 11.78 ms
8ab11b6+dirty 1222.91 ms 1216.08 ms -6.83 ms
1c65324+dirty 1239.71 ms 1239.86 ms 0.15 ms
a0b0298+dirty 1227.71 ms 1234.12 ms 6.41 ms
83f6f6c+dirty 1232.02 ms 1229.15 ms -2.87 ms

App size

Revision Plain With Sentry Diff
8bda0cc+dirty 3.19 MiB 4.26 MiB 1.08 MiB
80b2ce3+dirty 2.92 MiB 3.40 MiB 492.75 KiB
31fcca2+dirty 2.92 MiB 3.46 MiB 557.31 KiB
db44eaf+dirty 2.92 MiB 3.66 MiB 761.15 KiB
2ec71da+dirty 2.92 MiB 3.69 MiB 791.06 KiB
e2b64fe+dirty 2.92 MiB 3.41 MiB 499.97 KiB
8ab11b6+dirty 2.92 MiB 3.67 MiB 772.38 KiB
1c65324+dirty 2.92 MiB 3.61 MiB 705.56 KiB
a0b0298+dirty 3.19 MiB 4.32 MiB 1.13 MiB
83f6f6c+dirty 2.92 MiB 3.67 MiB 772.37 KiB

Previous results on branch: add-detox-test

Startup times

Revision Plain With Sentry Diff
922b103+dirty 1218.81 ms 1211.89 ms -6.93 ms

App size

Revision Plain With Sentry Diff
922b103+dirty 3.19 MiB 4.32 MiB 1.13 MiB

Base automatically changed from fix-attach-app-start-to-the-first-started-root-span to main March 4, 2025 17:57
Copy link
Contributor

github-actions bot commented Mar 5, 2025

Android (legacy) Performance metrics 🚀

  Plain With Sentry Diff
Startup time 384.94 ms 420.98 ms 36.04 ms
Size 17.75 MiB 20.12 MiB 2.37 MiB

Baseline results on branch: main

Startup times

Revision Plain With Sentry Diff
40c35c5 438.38 ms 437.52 ms -0.86 ms
76d1baf+dirty 335.72 ms 355.52 ms 19.80 ms
c2a4e9b 480.42 ms 485.60 ms 5.18 ms
5a22220 412.38 ms 447.35 ms 34.97 ms
397a69c 425.02 ms 423.40 ms -1.63 ms
8bda0cc 450.38 ms 433.89 ms -16.49 ms
ae7b03d 428.82 ms 412.33 ms -16.49 ms
484813b 434.55 ms 452.31 ms 17.75 ms
3853f43 329.68 ms 346.32 ms 16.64 ms
7d161c0 413.02 ms 435.29 ms 22.26 ms

App size

Revision Plain With Sentry Diff
40c35c5 17.75 MiB 20.11 MiB 2.37 MiB
76d1baf+dirty 17.73 MiB 20.04 MiB 2.31 MiB
c2a4e9b 17.73 MiB 20.06 MiB 2.33 MiB
5a22220 17.73 MiB 19.93 MiB 2.20 MiB
397a69c 17.75 MiB 20.12 MiB 2.37 MiB
8bda0cc 17.75 MiB 20.11 MiB 2.36 MiB
ae7b03d 17.75 MiB 20.11 MiB 2.37 MiB
484813b 17.73 MiB 20.07 MiB 2.33 MiB
3853f43 17.73 MiB 19.81 MiB 2.08 MiB
7d161c0 17.75 MiB 20.12 MiB 2.37 MiB

Previous results on branch: add-detox-test

Startup times

Revision Plain With Sentry Diff
922b103 441.90 ms 436.64 ms -5.25 ms

App size

Revision Plain With Sentry Diff
922b103 17.75 MiB 20.12 MiB 2.37 MiB

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant