Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mrc-4333 e2e tests #179

Merged
merged 58 commits into from
Mar 18, 2025
Merged
Changes from 1 commit
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
d9030c4
run backend with basicauth for smoke test
EmmaLRussell Feb 17, 2025
401f806
add playwright config
EmmaLRussell Feb 17, 2025
f4c8cee
add e2e to ci, first part of server script
EmmaLRussell Feb 18, 2025
44810f1
can run e2e tests with github auth
EmmaLRussell Feb 18, 2025
999dfc1
format
EmmaLRussell Feb 18, 2025
858c3b7
add filter test
EmmaLRussell Feb 19, 2025
1b70507
add packet page summary tests (local and generic)
EmmaLRussell Feb 20, 2025
47b2c73
packetPage section tests
EmmaLRussell Feb 20, 2025
9110d07
lint
EmmaLRussell Feb 20, 2025
24d37e7
create super user in ci
EmmaLRussell Feb 20, 2025
f96ff73
lint
EmmaLRussell Feb 20, 2025
65e5a64
install playwright
EmmaLRussell Feb 20, 2025
5fb99dc
add Platform test
EmmaLRussell Feb 20, 2025
d1fc962
Merge branch 'main' into mrc-4333-e2e
EmmaLRussell Feb 20, 2025
a433d1d
lint
EmmaLRussell Feb 20, 2025
78b79cd
export base url var in ci
EmmaLRussell Feb 20, 2025
51c7f14
add local parameter group row test
EmmaLRussell Feb 20, 2025
131f07e
packet group page tests
EmmaLRussell Feb 20, 2025
45ed78f
add parameter column test
EmmaLRussell Feb 20, 2025
b659293
update readme and run app in dev on ci
EmmaLRussell Feb 25, 2025
c33c16e
test fixes for ci
EmmaLRussell Feb 25, 2025
46c86a1
comments
EmmaLRussell Feb 25, 2025
72c2ae9
fixes for dev run
EmmaLRussell Feb 25, 2025
65470ce
format
EmmaLRussell Feb 25, 2025
f88ea33
docker run does not need basic auth
EmmaLRussell Feb 25, 2025
42fa7b9
revert previous change
EmmaLRussell Feb 25, 2025
a8452ed
refactor to get credentials in auth setup ts - cleanup ongoing
EmmaLRussell Mar 6, 2025
09c2362
working scripts to run against dev and prod
EmmaLRussell Mar 6, 2025
7f14adb
require basic creds to be in env
EmmaLRussell Mar 6, 2025
fe74641
throw error if any fetches from packit during auth fail
EmmaLRussell Mar 6, 2025
011e99b
delete old scripts
EmmaLRussell Mar 6, 2025
929c9d4
instance relative links and upload test results on CI
EmmaLRussell Mar 7, 2025
688b118
Update README.md
EmmaLRussell Mar 7, 2025
47a41f4
Update README.md
EmmaLRussell Mar 7, 2025
a0a15ca
Update .github/workflows/frontend-test-and-build.yml
EmmaLRussell Mar 7, 2025
73a855a
lint
EmmaLRussell Mar 7, 2025
2d6b370
more lint
EmmaLRussell Mar 7, 2025
373fa3c
numerous changes from review
EmmaLRussell Mar 7, 2025
831f778
update README
EmmaLRussell Mar 7, 2025
1652e88
formatting
EmmaLRussell Mar 7, 2025
19472a7
install playwright deps
EmmaLRussell Mar 7, 2025
c59b700
updates for prod
EmmaLRussell Mar 9, 2025
caf28a0
check that test files import "test" from the tag test fixture
EmmaLRussell Mar 17, 2025
66a606f
Update README.md
EmmaLRussell Mar 17, 2025
746c90c
Update app/e2e/packetPage.demo.spec.ts
EmmaLRussell Mar 17, 2025
cf9be0b
Update app/playwright.config.ts
EmmaLRussell Mar 17, 2025
d168b21
lint
EmmaLRussell Mar 17, 2025
005330f
update dev and prod urls
EmmaLRussell Mar 17, 2025
df25e94
CI timeout increase :(
EmmaLRussell Mar 17, 2025
15dee7e
try checkbox click
EmmaLRussell Mar 17, 2025
ef187e3
reset timeout
EmmaLRussell Mar 17, 2025
a2d8065
Update README.md
EmmaLRussell Mar 18, 2025
12dbc8f
set locale in config
EmmaLRussell Mar 18, 2025
63c64f1
Merge branch 'mrc-4333-e2e' of github.com:mrc-ide/packit into mrc-433…
EmmaLRussell Mar 18, 2025
da88cfd
update README
EmmaLRussell Mar 18, 2025
7cb2e42
update tagCheckFixture for dev subdomains
EmmaLRussell Mar 18, 2025
ed09eb4
use tagTestFixure from auth.setup and explicitly set base url in defa…
EmmaLRussell Mar 18, 2025
0eb3b65
update node versions
EmmaLRussell Mar 18, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
add playwright config
EmmaLRussell committed Feb 17, 2025
commit 401f80682b24c4c730c5d18fe0b19300c57d2191
2 changes: 1 addition & 1 deletion .github/workflows/frontend-test-and-build.yml
Original file line number Diff line number Diff line change
@@ -59,7 +59,7 @@ jobs:
- name: Test app
run: npm test --prefix=app -- --coverage
- name: Run dependencies
run: ./scripts/run-dependencies && ./api/scripts/run basicauth && ./api/scripts/smoke-test
run: ./scripts/run-dependencies && ./api/scripts/run basicauth && ./scripts/basic-create-super-user && ./api/scripts/smoke-test
- name: Upload coverage to codecov
uses: codecov/codecov-action@v3
with:
4 changes: 3 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
.idea
.vscode
packit.iml
packit.iml
app/test-results
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would these be useful to upload as artefacts of the CI?

eg.

- name: Upload test reports
uses: actions/upload-artifact@v4
if: ${{ failure() }}
with:
name: reports
path: |
api/app/build/reports/
api/app/build/test-results/
${{ runner.temp }}/docker-logs
${{ runner.temp }}/runner-logs

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that auth.json is written to the test-results dir, so if that is erroneously not deleted (e.g. if playwright dies mid-run), we'd be uploading the github access token for all to see. So auth.json might want to be written somewhere else instead.

Copy link
Contributor Author

@EmmaLRussell EmmaLRussell Mar 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It wouldn't be the github access token, it would just be the temporary packit token, which would only apply to the localhost instance we're running on CI, so I think we should be ok unless we start running tests against prod on CI. I'll try to omit the auth.json though. (Playwright doesn't let you write outside the parent folder, but I'll try deleting it before upload.

app/playwright-report
6 changes: 6 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -60,3 +60,9 @@ See [app/README.md](https://github.com/mrc-ide/packit/blob/main/app/README.md)
## Authentication in Packit

See [docs/auth.md](docs/auth.md)

## e2e Tests

Playwright tests are in `./app/e2e`. They can be configured to use `PLAYWRIGHT_BASE_URL` env var, but will fall back to
`http://localhost:3000`. They expect basic auth super user to be available, so you can run `./scripts/dev-start --super-user`
to start the api and app, then from `./app` run `npm run test:e2e` to run the tests.
14 changes: 14 additions & 0 deletions app/e2e/auth.setup.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
import { test as setup, expect } from "@playwright/test";

// Define "setup" as a dependency for any test project which requires prior authentication
setup("authenticate", async({ page }, testInfo) => {
await page.goto("/");
// Dev superuser credentials
await page.getByLabel("Email").fill("resideUser@resideAdmin.ic.ac.uk");
await page.getByLabel("Password").fill("password");
await page.getByRole("button", { name: /Log in/i }).click();
await expect(page.locator("body")).toHaveText(/Manage Access/);
// write out context to tmp location to be picked up by dependent tests
const authFile = testInfo.outputPath("auth.json");
await page.context().storageState({path: authFile});
});
6 changes: 6 additions & 0 deletions app/e2e/index.spec.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
import { test, expect } from "@playwright/test";

test("packet group list", async({ page }) => {
await page.goto("/");
await expect(page.locator("body")).toHaveText(/Manage Access/);
});
60 changes: 60 additions & 0 deletions app/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions app/package.json
Original file line number Diff line number Diff line change
@@ -13,6 +13,7 @@
},
"dependencies": {
"@hookform/resolvers": "^3.3.4",
"@playwright/test": "^1.50.1",
"@radix-ui/react-accordion": "^1.2.2",
"@radix-ui/react-alert-dialog": "^1.0.5",
"@radix-ui/react-avatar": "^1.0.4",
@@ -69,6 +70,7 @@
"start": "react-scripts start",
"build": "react-scripts build",
"test": "react-scripts test",
"test:e2e": "playwright test",
"lint": "npx eslint ./src ",
"lint:fix": "npx eslint --fix ./src",
"eject": "react-scripts eject",
79 changes: 79 additions & 0 deletions app/playwright.config.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
import process from "node:process";
import { defineConfig, devices } from "@playwright/test";

const setupProject = { name: "setup", testMatch: /.*\.setup\.ts/ };

const outputDir = "./test-results";
export default defineConfig({
testDir: "./e2e",
outputDir,
fullyParallel: true,
/* Maximum time one test can run for. */
timeout: process.env.CI ? 30 * 1000 : 15 * 1000,
expect: {
/**
* Maximum time expect() should wait for the condition to be met.
* For example in `await expect(locator).toHaveText();`
*/
timeout: 10000
},
/* Fail the build on CI if you accidentally left test.only in the source code. */
forbidOnly: !!process.env.CI,
/* Retry on CI only */
retries: process.env.CI ? 2 : 0,
/* Allow parallel tests */
workers: undefined,
/* Reporter to use. See https://playwright.dev/docs/test-reporters */
reporter: "html",
/* Shared settings for all the projects below. See https://playwright.dev/docs/api/class-testoptions. */
use: {
/* Maximum time each action such as `click()` can take. Defaults to 0 (no limit). */
actionTimeout: 0,
/* Base URL to use in actions like `await page.goto('/')`. */
baseURL: process.env.PLAYWRIGHT_BASE_URL || "http://localhost:3000",

/* Collect trace when retrying the failed test. See https://playwright.dev/docs/trace-viewer */
trace: "on-first-retry",

headless: true,
screenshot: "only-on-failure"
},

/* Configure projects for major browsers */
projects:
process.env.CI
? [
setupProject,
{
name: "chromium",
use: {
...devices["Desktop Chrome"],
// Use prepared auth state.
storageState: `${outputDir}/auth.json`
},
dependencies: ['setup']
},
{
name: "firefox",
use: {
...devices["Desktop Firefox"],
// Use prepared auth state.
storageState: `${outputDir}/auth.json`
},
dependencies: ['setup']
}
]
: /* Just test on Chrome when running on local dev */
[
setupProject,
{
name: "chromium",
use: {
...devices["Desktop Chrome"],
// Use prepared auth state.
storageState: `${outputDir}/auth.json`
},
dependencies: ['setup']
}
]
});
6 changes: 5 additions & 1 deletion scripts/dev-start
Original file line number Diff line number Diff line change
@@ -54,7 +54,11 @@ if [ $API_BUILD_SKIP -eq 0 ]; then
"$HERE"/../api/scripts/build
fi

"$HERE"/../api/scripts/run
if [ $ADD_SUPER_USER -eq 1 ]; then
# assume we want to run with basic auth if we're creating a basic super user
BASIC_AUTH="basicauth"
fi
"$HERE"/../api/scripts/run $BASIC_AUTH

if [ $ADD_SUPER_USER -eq 1 ]; then
echo "Waiting for api to initialise"