Charlie Freestone Blog Post - Some Best Practices for Accessible Automation Testing #317
Conversation
There was a problem hiding this comment.
Overall, plenty of good advice.
I was surprised that an article about testing with "Accessible" in the title only lightly touched on accessibility concerns - using roles to locate elements. I feel this is somewhat misleading, as the rest of the article concerns your thoughts on naming and tagging tests to make them more informative, plus the part which I think is about not worrying too much about specificity, not testing in too much detail the things which are not all that important, none of which is about accessibility.
I would advise changing the title to reflect this.
_posts/2025-05-14-some-best-practices-for-accessible-automation.md
Outdated
Show resolved
Hide resolved
| While on my most recent project I had the unique experience of working closely with many testers and test minded individuals. This allowed me to learn some much-needed lessons about how to best implement automation testing with accessibility in mind, a sometimes-overlooked area of test automation. | ||
|
|
||
| ## Introduction | ||
| What I hope to share with you is how simple it can be to both think about and implement the approach to make sure your tests are accessible. This can be because you have strict reasons on the project to ensure compliance with certain metrics, because this can lead to improved readability of your code and its longevity, or because you want to read some wonderful thoughts from certain Testers perspective. |
There was a problem hiding this comment.
certain Testers perspective
Not sure what you are trying to say... Do you mean "from a certain tester's perspective", i.e. you? Or did you mean plural "testers"?
I think this introductory paragraph highlights my main question about your article: what does it mean for a test to be accessible, and what do metrics or code readability have to do with that? Are you concerned about making your test code accessible to testers who need to use a screen reader? If so, I think you need to make that clear, and give more examples of how you can achieve that.
There was a problem hiding this comment.
Certain testers perspective was meaning myself.
What I wanted to share was skills I learnt which anyone can write in their tests which is in line which accessibility testing. Its not aimed at testers who need a screen reader. Its lessons I have learnt from doing Accessibility testing, which can make your automation tests more accessible.
There was a problem hiding this comment.
Certain testers perspective was meaning myself.
Then you want "from a certain tester's perspective" (need apostrophe)
Accessibility testing is about testing that a UI is accessible to all users, whether they are keyboard-only users, have vision or motor impairments, dyslexia, epilepsy, etc. The only part of this article which is about accessibility is locating elements by role. You are not really making your automation tests more accessible, you are making them more readable.
There was a problem hiding this comment.
By the way, how did you do Accessibility testing, what tool(s) did you use? That would also make an interesting blog post.
There was a problem hiding this comment.
Made the adjustment to add the apostrophes.
The application of these methods was what I picked up while learning about accessibility testing on the project. Readability does allign itself into accessibility. If something is more readable it is inherantly more accessable. The lessons I learnt about accessibility testing I have led to creating tests which are more accessible to others, readability is one of those areas.
There was a problem hiding this comment.
By the way, how did you do Accessibility testing, what tool(s) did you use? That would also make an interesting blog post.
I shall keep this in mind for a future blog
There was a problem hiding this comment.
Made the adjustment to add the apostrophes.
Ah no, you've added quotation marks around the phrase. I mean this:
from a certain tester's perspective
I still take issue with your use of the word accessible here. You need to be really careful with that word, because accessibility testing (how accessible is your UI) is a totally different ballgame to writing tests in a way that makes them easier for other people to read, modify and maintain - that is not accessibility testing.
Readability or understandability is one very small part of making test code accessible, and has little to do with the UI itself being accessible. Did you use the axe plugin for playwright? https://playwright.dev/docs/accessibility-testing
There was a problem hiding this comment.
Ok if you still think it needs an overhaul then just tell me what you would like me to change so its no longer an issue please.
There was a problem hiding this comment.
The legitmate reason for myself making this blog post was by myself doing some Accessibility Testing and then through that I learnt some best practicies to implement into my automation tests.
_posts/2025-05-14-some-best-practices-for-accessible-automation.md
Outdated
Show resolved
Hide resolved
_posts/2025-05-14-some-best-practices-for-accessible-automation.md
Outdated
Show resolved
Hide resolved
_posts/2025-05-14-some-best-practices-for-accessible-automation.md
Outdated
Show resolved
Hide resolved
_posts/2025-05-14-some-best-practices-for-accessible-automation.md
Outdated
Show resolved
Hide resolved
_posts/2025-05-14-some-best-practices-for-accessible-automation.md
Outdated
Show resolved
Hide resolved
_posts/2025-05-14-some-best-practices-for-accessible-automation.md
Outdated
Show resolved
Hide resolved
_posts/2025-05-14-some-best-practices-for-accessible-automation.md
Outdated
Show resolved
Hide resolved
_posts/2025-05-14-some-best-practices-for-accessible-automation.md
Outdated
Show resolved
Hide resolved
_posts/2025-05-14-some-best-practices-for-accessible-automation.md
Outdated
Show resolved
Hide resolved
|
|
||
| Things like filters and drop-downs can be difficult to locate but they can be found. A good tip for finding this is by entering the webpage and open up DevTools, then clicking near the item you wish to find the selector of, but not on it. After that press tab and open up the console within DevTools. If you then type the command: `document.activeElement` you will find the info you need. | ||
|
|
||
| By the same accessible reasoning it is good practice to not use `.getbyTestID`.This is due to the fact that names of those same ID’s will not be useful to someone with an accessibility issue, as it will sound like Jargon when navigated to. |
There was a problem hiding this comment.
I don't believe screen readers will announce data-testid attributes, so I don't know what you mean here.
|
|
||
| This is because there is an interface that runs behind the scenes of the UI and Playwright can pick selectors which will not be seen by someone who needs to use the keyboard to navigate around the system. For these users, the elements they have access to are either tabbed to or navigated to via arrow keys. If using a screen reader these selectors need to have a consistent and readable naming convention. Otherwise, any software that reads it aloud would read out jargon which will only confuse the User. | ||
|
|
||
| Things like filters and drop-downs can be difficult to locate but they can be found. A good tip for finding this is by entering the webpage and open up DevTools, then clicking near the item you wish to find the selector of, but not on it. After that press tab and open up the console within DevTools. If you then type the command: `document.activeElement` you will find the info you need. |
There was a problem hiding this comment.
You will not find the information you need if the page has not been implemented using semantic html or aria attributes.
| ## Don’t just getBy, Role with it | ||
| After correctly measuring our expectations, we want to roll into selecting all the necessary parameters we can get by with. Therefore, it comes as no surprise we use the `.getByRole function`, but in order for us to make sense of it then it's all about those pesky selectors. | ||
|
|
||
| This is because there is an interface that runs behind the scenes of the UI and Playwright can pick selectors which will not be seen by someone who needs to use the keyboard to navigate around the system. For these users, the elements they have access to are either tabbed to or navigated to via arrow keys. If using a screen reader these selectors need to have a consistent and readable naming convention. Otherwise, any software that reads it aloud would read out jargon which will only confuse the User. |
There was a problem hiding this comment.
What is this consistent and readable naming convention that you mention here? You do not elaborate on it. Are you thinking about semantic html? Ensuring elements have an accessible name?
Element locators are only tangentially related to what a screen reader will announce, and both are fully dependent on the underlying HTML.
|
|
||
| This is because there is an interface that runs behind the scenes of the UI and Playwright can pick selectors which will not be seen by someone who needs to use the keyboard to navigate around the system. For these users, the elements they have access to are either tabbed to or navigated to via arrow keys. If using a screen reader these selectors need to have a consistent and readable naming convention. Otherwise, any software that reads it aloud would read out jargon which will only confuse the User. | ||
|
|
||
| Things like filters and drop-downs can be difficult to locate but they can be found. A good tip for finding this is by entering the webpage and open up DevTools, then clicking near the item you wish to find the selector of, but not on it. After that press tab and open up the console within DevTools. If you then type the command: `document.activeElement` you will find the info you need. |
There was a problem hiding this comment.
Why are filters and dropdowns difficult to locate?
| If you want to go the extra mile it's even better to use annotations for ticket numbers. That way tags can be used exclusively to filter what tests are run, filtering test results. This is what is documented in the Playwright documentatio. | ||
|
|
||
| ## Conclusion | ||
| Well, there have been some odd analogies and puns along the way but hopefully this peek into a tester’s mind has helped you understand the importance of, and some ways to implement some good accessible skills in your automation tests while also keeping it simple to implement. Maybe there are even more simple techniques out there so go out and find them, or better yet bring them up with others and let's get everything even more accessible! |
There was a problem hiding this comment.
As I keep trying to point out, the only "accessible skills" you elaborate on in this post is locating elements by role, which is one tiny part of accessibility. None of the remaining tips are related to accessibility testing, which is my primary point of contention in this post.
| While on my most recent project I had the unique experience of working closely with many testers and test minded individuals. This allowed me to learn some much-needed lessons about how to best implement automation testing with accessibility in mind, a sometimes-overlooked area of test automation. | ||
|
|
||
| ## Introduction | ||
| What I hope to share with you is how simple it can be to both think about and implement the approach to make sure your tests are accessible. This can be because you have strict reasons on the project to ensure compliance with certain metrics, because this can lead to improved readability of your code and its longevity, or because you want to read some wonderful thoughts from certain testers perspective. |
There was a problem hiding this comment.
Typo
from a certain tester's perspective.
I am not sure what you mean by tests being accessible, as that is not the same as testing the UI is accessible.
|
|
||
| This neat little function is the bread-and-butter basics of automating in playwright, it's what we want to see when we place our test into a certain configuration. This doesn’t mean it has to do handstands or backflips, but just like any gymnastics routine we do wait on it with bated breath, hoping for results we will cheer for. This anticipation makes it easy for us to write something which makes us wait for that big finale in our tests, even if the tests took the literal definition of breaking a leg. | ||
|
|
||
| And therein lies the issue; during a test we usually assert for confirmation to ensure we are where we expect. That means we can `expect(response.status()).toBe(200)` which is a wonderful successful request that has found the requested resource of a page. Instead we get a `201 response`, which is just as equally a successful request but it led to the creation of a resource. Test results can’t always be perfect and that means that this stopping point during the test, before we check the larger critical parameters later on, will skip the test. Instead, we have to accept that not every response will be exciting and just `expect(response.status()).toBe(OK)` which allows us to still capture data from a test even if the status code is different. |
There was a problem hiding this comment.
This fix appears to have slipped through the net.
expect(response.status()).toBeOK()
Please add a direct link to your post here:
https://cfreestone-scottlogic.github.io/blog/2025/05/14/some-best-practices-for-accessible-automation.html
Have you (please tick each box to show completion):
npm installfollowed bynpx mdspell "**/{FILE_NAME}.md" --en-gb -a -n -x -tif that's your thing)Posts are reviewed / approved by your Regional Tech Lead.