Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test(e2e): set up display documents e2e #8828

Open
wants to merge 5 commits into
base: next
Choose a base branch
from
Open

Conversation

RitaDias
Copy link
Contributor

@RitaDias RitaDias commented Mar 4, 2025

Description

Added e2e tests to test document displayed from sapp-2388

What to review

Do the tests make sense?

Notes for release

N/A

Copy link

vercel bot commented Mar 4, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
page-building-studio ✅ Ready (Inspect) Visit Preview 💬 Add feedback Mar 11, 2025 2:24pm
performance-studio ✅ Ready (Inspect) Visit Preview 💬 Add feedback Mar 11, 2025 2:24pm
test-studio ✅ Ready (Inspect) Visit Preview 💬 Add feedback Mar 11, 2025 2:24pm
2 Skipped Deployments
Name Status Preview Comments Updated (UTC)
studio-workshop ⬜️ Ignored (Inspect) Visit Preview Mar 11, 2025 2:24pm
test-next-studio ⬜️ Ignored (Inspect) Mar 11, 2025 2:24pm

Copy link
Contributor

github-actions bot commented Mar 4, 2025

No changes to documentation

Copy link
Contributor

github-actions bot commented Mar 4, 2025

Coverage Report

Status Category Percentage Covered / Total
🔵 Lines 42.01% 54407 / 129505
🔵 Statements 42.01% 54407 / 129505
🔵 Functions 46.88% 2728 / 5819
🔵 Branches 79.35% 10185 / 12835
File CoverageNo changed files found.
Generated in workflow #32013 for commit d248c64 by the Vitest Coverage Report Action

Copy link
Contributor

github-actions bot commented Mar 4, 2025

Component Testing Report Updated Mar 11, 2025 7:44 AM (UTC)

❌ Failed Tests (3) -- expand for details
File Status Duration Passed Skipped Failed
comments/CommentInput.spec.tsx ✅ Passed (Inspect) 1m 3s 15 0 0
formBuilder/ArrayInput.spec.tsx ✅ Passed (Inspect) 12s 3 0 0
formBuilder/inputs/PortableText/Annotations.spec.tsx ❌ Failed (Inspect) 2m 33s 3 0 3
formBuilder/inputs/PortableText/copyPaste/CopyPaste.spec.tsx ✅ Passed (Inspect) 40s 11 7 0
formBuilder/inputs/PortableText/copyPaste/CopyPasteFields.spec.tsx ✅ Passed (Inspect) 0s 0 12 0
formBuilder/inputs/PortableText/Decorators.spec.tsx ✅ Passed (Inspect) 23s 6 0 0
formBuilder/inputs/PortableText/DisableFocusAndUnset.spec.tsx ✅ Passed (Inspect) 13s 3 0 0
formBuilder/inputs/PortableText/DragAndDrop.spec.tsx ✅ Passed (Inspect) 25s 6 0 0
formBuilder/inputs/PortableText/FocusTracking.spec.tsx ✅ Passed (Inspect) 1m 5s 15 0 0
formBuilder/inputs/PortableText/Input.spec.tsx ✅ Passed (Inspect) 1m 32s 21 0 0
formBuilder/inputs/PortableText/ObjectBlock.spec.tsx ✅ Passed (Inspect) 1m 42s 21 0 0
formBuilder/inputs/PortableText/PresenceCursors.spec.tsx ✅ Passed (Inspect) 13s 3 9 0
formBuilder/inputs/PortableText/Styles.spec.tsx ✅ Passed (Inspect) 26s 6 0 0
formBuilder/inputs/PortableText/Toolbar.spec.tsx ✅ Passed (Inspect) 1m 36s 21 0 0
formBuilder/tree-editing/TreeEditing.spec.tsx ✅ Passed (Inspect) 0s 0 3 0
formBuilder/tree-editing/TreeEditingNestedObjects.spec.tsx ✅ Passed (Inspect) 0s 0 3 0

Copy link
Contributor

github-actions bot commented Mar 4, 2025

⚡️ Editor Performance Report

Updated Tue, 11 Mar 2025 14:33:19 GMT

Benchmark reference
latency of sanity@latest
experiment
latency of this branch
Δ (%)
latency difference
article (title) 20.2 efps (50ms) 23.3 efps (43ms) -7ms (-13.1%)
article (body) 63.3 efps (16ms) 63.3 efps (16ms) +0ms (-/-%)
article (string inside object) 22.7 efps (44ms) 23.3 efps (43ms) -1ms (-2.3%)
article (string inside array) 20.4 efps (49ms) 20.8 efps (48ms) -1ms (-2.0%)
recipe (name) 30.3 efps (33ms) 45.5 efps (22ms) -11ms (-33.3%)
recipe (description) 34.5 efps (29ms) 50.0 efps (20ms) -9ms (-31.0%)
recipe (instructions) 99.9+ efps (6ms) 99.9+ efps (5ms) -1ms (-/-%)
synthetic (title) 16.4 efps (61ms) 19.0 efps (53ms) -9ms (-13.9%)
synthetic (string inside object) 16.3 efps (62ms) 19.6 efps (51ms) -11ms (-17.1%)

efps — editor "frames per second". The number of updates assumed to be possible within a second.

Derived from input latency. efps = 1000 / input_latency

Detailed information

🏠 Reference result

The performance result of sanity@latest

Benchmark latency p75 p90 p99 blocking time test duration
article (title) 50ms 77ms 100ms 369ms 850ms 13.3s
article (body) 16ms 18ms 21ms 145ms 190ms 5.5s
article (string inside object) 44ms 47ms 54ms 90ms 331ms 7.4s
article (string inside array) 49ms 51ms 56ms 61ms 366ms 7.6s
recipe (name) 33ms 35ms 36ms 37ms 80ms 8.9s
recipe (description) 29ms 30ms 31ms 32ms 3ms 6.2s
recipe (instructions) 6ms 7ms 8ms 17ms 0ms 3.2s
synthetic (title) 61ms 64ms 71ms 284ms 1974ms 13.6s
synthetic (string inside object) 62ms 63ms 72ms 97ms 1695ms 9.4s

🧪 Experiment result

The performance result of this branch

Benchmark latency p75 p90 p99 blocking time test duration
article (title) 43ms 49ms 55ms 252ms 494ms 11.9s
article (body) 16ms 18ms 21ms 175ms 236ms 5.9s
article (string inside object) 43ms 46ms 54ms 157ms 211ms 7.4s
article (string inside array) 48ms 55ms 57ms 79ms 352ms 7.7s
recipe (name) 22ms 24ms 27ms 45ms 27ms 7.9s
recipe (description) 20ms 20ms 22ms 24ms 0ms 4.7s
recipe (instructions) 5ms 7ms 7ms 8ms 0ms 3.1s
synthetic (title) 53ms 55ms 61ms 211ms 864ms 12.6s
synthetic (string inside object) 51ms 52ms 54ms 211ms 683ms 8.1s

📚 Glossary

column definitions

  • benchmark — the name of the test, e.g. "article", followed by the label of the field being measured, e.g. "(title)".
  • latency — the time between when a key was pressed and when it was rendered. derived from a set of samples. the median (p50) is shown to show the most common latency.
  • p75 — the 75th percentile of the input latency in the test run. 75% of the sampled inputs in this benchmark were processed faster than this value. this provides insight into the upper range of typical performance.
  • p90 — the 90th percentile of the input latency in the test run. 90% of the sampled inputs were faster than this. this metric helps identify slower interactions that occurred less frequently during the benchmark.
  • p99 — the 99th percentile of the input latency in the test run. only 1% of sampled inputs were slower than this. this represents the worst-case scenarios encountered during the benchmark, useful for identifying potential performance outliers.
  • blocking time — the total time during which the main thread was blocked, preventing user input and UI updates. this metric helps identify performance bottlenecks that may cause the interface to feel unresponsive.
  • test duration — how long the test run took to complete.

@RitaDias RitaDias force-pushed the sapp-2388-e2e-tests branch from e051582 to d9b99cc Compare March 4, 2025 09:25
@RitaDias RitaDias force-pushed the sapp-2388-e2e-tests branch from d53da6f to 878729b Compare March 4, 2025 09:36
@RitaDias RitaDias force-pushed the sapp-2388-e2e-tests branch from 095eb45 to 54d097f Compare March 4, 2025 09:57
@RitaDias RitaDias force-pushed the sapp-2388-e2e-tests branch from 9156e36 to b7a3fe0 Compare March 4, 2025 12:18
test(e2e): update e2e tests

test(e2e): first stab at e2e new tests

test(e2e): remove unneeded before each

test(e2e): add more tests

test(e2e): add more tests

test(e2e): add more tests

test(e2e): add more tests

test(e2e): add more tests

test(e2e): clean up

test(e2e): clean up

test(e2e): add more tests

refactor(structure): updated condition based on suggestion

e2e(structure): move e2e tests to the files

e2e(structure): add test
test.afterAll(async ({sanityClient}) => {
const dataset = sanityClient.config().dataset

await archiveRelease({sanityClient, dataset, releaseId: asapReleaseId})
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wondering if it were better if these were grouped by releaseId instead of operation. Now, if e.g archiving asapReleaseId fails for some reason, we won't get to archiving the undecidedReleaseId

test.afterAll(async ({sanityClient}) => {
const dataset = sanityClient.config().dataset

await sanityClient.delete(publishedDocument._id)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this fails, we'll leave the release around forever. Might not be a big deal as we're probably deleting the dataset anwyay

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I'm not mistaken the datasets are deleted per run (might be mistaken, is that not the case?) Or at least that was my assumption.
If it fails it does indeed stay forever, but I feel that this is for us to fix outside of the scope of the code, unless you feel there is something we can do to make it more certain that this succeeds?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants