Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Issue #347]: Add pa11y-ci checks #350

Draft
wants to merge 8 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 63 additions & 0 deletions .github/workflows/ci-a11y.yml
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: type out the full name ("accessibility" rather than "a11y")

Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
name: Accessibility tests
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

follow the naming convention of other CI workflows and prefix with "CI" and use "Title Case"

image


on:
pull_request:
paths:
- app/**
- .github/workflows/ci-a11y.yml

jobs:
build:
name: Pa11y-ci tests
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

name the job based on what it's doing not the tool it's using e.g. "Accessibility" not "Pa11y"

runs-on: ubuntu-latest
defaults:
run:
working-directory: ./app

env:
NODE_VERSION: 20
LOCKFILE_PATH: ./app/package-lock.json
PACKAGE_MANAGER: npm

steps:
- name: Checkout source
uses: actions/checkout@v4

- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: ${{ env.NODE_VERSION }}
cache-dependency-path: ${{ env.LOCKFILE_PATH }}
cache: ${{ env.PACKAGE_MANAGER }}

- name: Install dependencies
run: npm ci

- name: Create screenshots directory
run: mkdir -p screenshots-output

- name: Build project
run: npm run build

- name: Start server and log output
run: npm run start &

- name: Wait for frontend to be ready
run: |
# Ensure the server wait script is executable
chmod +x ./bin/wait-for-frontend.sh
./bin/wait-for-frontend.sh

- name: Run pa11y-ci
run: |
set -e # Ensure the script fails if any command fails
npm run test:pa11y-desktop
npm run test:pa11y-mobile
echo "pa11y-ci tests finished."

- name: Upload screenshots to artifacts
if: always()
uses: actions/upload-artifact@v3
with:
name: screenshots
path: ./app/screenshots-output
16 changes: 16 additions & 0 deletions app/.pa11yci-desktop.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
{
"defaults": {
"timeout": 240000,
"runners": ["axe"],
"ignore": ["color-contrast"],
"concurrency": 1,
"chromeLaunchConfig": {
"ignoreHTTPSErrors": true,
"args": ["--disable-dev-shm-usage", "--no-sandbox"]
},
"actions": [
"wait for element #main-content to be visible",
"screen capture screenshots-output/desktop-main-view.png"
]
}
}
23 changes: 23 additions & 0 deletions app/.pa11yci-mobile.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
{
"defaults": {
"timeout": 240000,
"runners": ["axe"],
"ignore": ["color-contrast"],
"concurrency": 1,
"chromeLaunchConfig": {
"ignoreHTTPSErrors": true,
"args": ["--disable-dev-shm-usage", "--no-sandbox"]
},
"viewport": {
"width": 390,
"height": 844,
"mobile": true
},
"actions": [
"wait for element #main-content to be visible",
"screen capture screenshots-output/mobile-main-view.png",
"click element .usa-navbar button",
"screen capture screenshots-output/mobile-expand-menu.png"
]
}
}
31 changes: 31 additions & 0 deletions app/bin/wait-for-frontend.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# wait-for-frontend.sh

#!/bin/bash
set -e

# Color formatting for readability
GREEN='\033[0;32m'
RED='\033[0;31m'
NO_COLOR='\033[0m'

MAX_WAIT_TIME=800 # seconds, adjust as necessary
WAIT_TIME=0


echo "Waiting for server to become ready..."

# Use curl to check the server health endpoint
until curl --output /dev/null --silent --head --fail http://localhost:3000;
do
printf '.'
sleep 5

WAIT_TIME=$(($WAIT_TIME + 5))
if [ $WAIT_TIME -gt $MAX_WAIT_TIME ]
then
echo -e "${RED}ERROR: Server did not become ready within ${MAX_WAIT_TIME} seconds.${NO_COLOR}"
exit 1
fi
done

echo -e "${GREEN}Server is ready after ~${WAIT_TIME} seconds.${NO_COLOR}"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CI script to wait for localhost:3000 - this kind of thing would be useful if you also needed to also wait for the API to load (if you have an e2e feature that you want to pa11y to test)

Loading
Loading