You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
To detect regressions and to make sure NVDA features work in specific GUI-based scenarios, NVDA source code defines system tests via Robot framework. Currently system tests are run to test:
NVDA startup and shutdown
Starting from the installer
Chrome tests
Notepad and symbol announcement tests
As discussed in #14104, drag and drop target announcement pull request could have been enhanced with system tests. Although it might be possible to skip system tests under specific conditions, NVDA tests are not usig it well at this time. Making matters complicated is different user interfaces between various Windows releases, along with making sure tests cover whatever Windows release Appveyor is using (currently Windows Server 2019, equivalent to Windows 10 October 2018 Update (10.0.17763) which is on long-term support, so unlikely to be tested by consumers and businesses anymore).
Proposal:
Let NVDA tests define a minimal test case for Windows features and create a list of useful Windows feature and app interaction tests. The base specification should define and include:
Ability to detect specific Windows releases, either using winVersion module from NVDA or sys.getwindowsversion() structure.
Run or skip tests based on Windows version the test runner is using.
Possible Windows feature and ap interaction tests:
Based on past discussions, the following should be tested (and testable):
Drag and drop target effect announcement
Emoji panel support
Status bar tests in places such as File Explorer
Search suggestions announcement, including suggestion count
Additional app testing can include (optional):
More Notepad tests
Windows Terminal, PowerShell, Command Prompt input and output
Frequently asked questions:
Q. Why create a discussion instead of a feature request issue?
This is so that a more general discussion can take place before creating issues to deal with specific cases, including creating foundations to support Windows feature and app interacion tests.
Q. Why care about Windows feature tests?
NVDA is a screen reader for Microsoft Windows, therefore it should be able to function and give test results in basic Windows feature usage and interaction scenarios.
Q. Can Windows feature tests cover all Windows releases to date?
No. Even though NVDA can run on Windows 7 (and Server 2008 R2) Service Pack 1 or later, as of September 2022, Windows 8.1 (Server 2012/2012 R2) and later are under mainstream and/or extended support.
Q. Can tests cover Windows interface languages other than English?
As much as we need to think about localization, as system tests are run mostly on Appveyor (local testing is possible) with American (United States) English as default interface language, tests will primarily deal with English text.
Q. Can tests be defined for specific architectures?
Yes and no. Yes in that architecture (x86, x64, ARM/ARM64) can be fetched via environment variables and could be a way to determine tests to be performed. At the same time, the primary testing architecture (from Appveyor side of things) is AMD64 (x64), so tests will primarily target interaction with 64-bit Windows for Intel and AMD processors (ARM64 might be a possibility if virtual machines decide to move to AArch64).
Q. Any other questions?
Feel free to add them as discussion progresses.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
Stemming from comments in #14101 and #14104:
Background and rationale:
To detect regressions and to make sure NVDA features work in specific GUI-based scenarios, NVDA source code defines system tests via Robot framework. Currently system tests are run to test:
As discussed in #14104, drag and drop target announcement pull request could have been enhanced with system tests. Although it might be possible to skip system tests under specific conditions, NVDA tests are not usig it well at this time. Making matters complicated is different user interfaces between various Windows releases, along with making sure tests cover whatever Windows release Appveyor is using (currently Windows Server 2019, equivalent to Windows 10 October 2018 Update (10.0.17763) which is on long-term support, so unlikely to be tested by consumers and businesses anymore).
Proposal:
Let NVDA tests define a minimal test case for Windows features and create a list of useful Windows feature and app interaction tests. The base specification should define and include:
Possible Windows feature and ap interaction tests:
Based on past discussions, the following should be tested (and testable):
Additional app testing can include (optional):
Frequently asked questions:
Q. Why create a discussion instead of a feature request issue?
This is so that a more general discussion can take place before creating issues to deal with specific cases, including creating foundations to support Windows feature and app interacion tests.
Q. Why care about Windows feature tests?
NVDA is a screen reader for Microsoft Windows, therefore it should be able to function and give test results in basic Windows feature usage and interaction scenarios.
Q. Can Windows feature tests cover all Windows releases to date?
No. Even though NVDA can run on Windows 7 (and Server 2008 R2) Service Pack 1 or later, as of September 2022, Windows 8.1 (Server 2012/2012 R2) and later are under mainstream and/or extended support.
Q. Can tests cover Windows interface languages other than English?
As much as we need to think about localization, as system tests are run mostly on Appveyor (local testing is possible) with American (United States) English as default interface language, tests will primarily deal with English text.
Q. Can tests be defined for specific architectures?
Yes and no. Yes in that architecture (x86, x64, ARM/ARM64) can be fetched via environment variables and could be a way to determine tests to be performed. At the same time, the primary testing architecture (from Appveyor side of things) is AMD64 (x64), so tests will primarily target interaction with 64-bit Windows for Intel and AMD processors (ARM64 might be a possibility if virtual machines decide to move to AArch64).
Q. Any other questions?
Feel free to add them as discussion progresses.
Thanks.
Beta Was this translation helpful? Give feedback.
All reactions