Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mobile: Accessibility: Improve focus handling in the note actions menu and modal dialogs #11929

Open
wants to merge 15 commits into
base: dev
Choose a base branch
from

Conversation

personalizedrefrigerator
Copy link
Collaborator

@personalizedrefrigerator personalizedrefrigerator commented Mar 4, 2025

Summary

This pull request adds custom focus-handling logic to the mobile app. Specifically, it:

  • When a modal/menu is visible: Marks the main application content as inert (web) and not accessibility focusable (iOS and Android). This should prevent focus from leaving open modals and focusing other parts of the app. (WCAG 2.2 SC 2.4.3. Focus order, SC 2.4.11: Focus not obscured).
    • Whether a modal or menu is visible is determined by whether at least one FocusControl.ModalContent component has visible={true}.
    • This fixes, for example, the confusing focus behavior on iOS while navigating the note actions menu.
  • When a modal/menu is dismissed: Autofocuses the last AccessibleView that was auto-focused while the menu was open. (WCAG 2.2 SC 2.4.3. Focus order)
    • Previously, if selecting an action from a menu immediately showed and tried to focus a component, the component would not receive focus.
    • This fixes an issue where the "Voice recorder" heading would not auto-focus when opened. (Instead, focus jumped to the default location).
  • Fixes an issue where plugin dialogs did not automatically receive focus when opened (plugin panels were okay) (WCAG 2.2 SC 2.4.11: Focus not obscured).

This pull request adds a document to readme/dev/spec with more information about how the updated focus management works.

To-do

This pull request, if feasible:

  • Testing: Add automated tests for auto-focus behavior. This may require mocking AccessibilityInfo.setAccessibilityFocus.

This pull request or a follow-up pull request:

  • Currently, the "edit" button in the note screen is shown within a <Portal>. This causes React Native Paper to render it outside of the inert/importantForAccessibility={false} region. As a result, it's still possible to click the "edit" button while the note actions menu is open. Currently, this switches to edit mode, but does not dismiss the note actions menu. Either:
    • The "edit" button should not be pressable while the note actions menu is open.
    • Clicking "edit" should dismiss the note actions menu.
  • This pull request does not change the library used for the note actions menu. As a result, these two issues persist:
    • The close button for the note actions menu is unlabelled.
    • The buttons in the note actions menu are not read as buttons by the screen reader (though they're still clickable).

Testing plan

This pull request has been tested manually (iOS 18.1 simulator & Android 13) by:

  1. On iOS,
    1. Enabling VoiceOver (MacOS).
    2. Moving VoiceOver focus into the iOS simulator.
  2. On Android, enabling TalkBack
  3. Opening a note.
  4. Opening the note actions menu.
  5. Navigating through the options in the menu, verifying that focus does not visit the note title or other parts of the app.
    • An exception to this is the "edit" button, which is shown using a React Native <Portal>, which displays content outside of the region with disabled focus.
  6. Moving focus back to the "Attach" button.
  7. Pressing the button.
  8. Selecting "Record audio" in the attach menu.
  9. Verifying that the "Voice recorder" heading is auto-focused.

Additionally, on Android 13,

  1. Open the plugin configuration screen.
  2. Open the about dialog for a plugin.
  3. Click the "close" button.
  4. Verify that focus is restored to the plugin button that opened the dialog.

Note: This commit has been manually tested by:
1. Enabling VoiceOver (MacOS).
2. Moving VoiceOver focus into the iOS simulator.
3. Opening a note.
4. Opening the note actions menu.
5. Navigating through the options in the menu, verifying that focus does not visit the note title or other parts of the app.
   - An exception to this is the "edit" button, which is shown using a React Native <Portal>, which displays content outside of the region with disabled focus.
6. Moving focus back to the "Attach" button.
7. Pressing the button.
8. Selecting "Record audio" in the attach menu.
9. Verifying that the "Voice recorder" heading is auto-focused.
@personalizedrefrigerator personalizedrefrigerator added mobile All mobile platforms accessibility Related to accessibility labels Mar 4, 2025
>
<AccessibleView
// Auto-focuses an empty view at the beginning of the sidemenu -- if we instead
// focus the container view, VoiceOver fails to focus to any components within
// the sidebar.
refocusCounter={!open ? 1 : undefined}
testID='sidemenu-menu-focus-region'
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently, testID, if provided, is included in the focus log message. It may make sense to use a different prop for this (e.g. debugTag) and make it required if refocusCounter is specified.

When closed, Android modals re-focus whatever had focus before the modal was open. This was
conflicting with the logic that disables focusability for the main app content until
all modals are closed.
Adding onAccessibilityEscape to the note actions menu should be done separately.
Comment on lines +5 to +11
const FocusControl = {
Provider: FocusControlProvider,
ModalWrapper,
MainAppContent,
};

export default FocusControl;
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The goal of wrapping these components in a FocusControl object was to make it clear to a reader that MainAppContent (and similar components) are specific to the FocusControl logic. An alternative could be to rename ModalWrapper to FocusControlModalWrapper and MainAppContent to FocusControlMainAppContent (or similar).

@personalizedrefrigerator
Copy link
Collaborator Author

The most recent CI failure seems related to desktop end-to-end tests (and thus unrelated to this pull request):

Logs
 ➤ YN0000: [@joplin/app-desktop]:   1) noteList.spec.ts:78:6 › noteList › arrow keys should navigate the note list ───────────────────
➤ YN0000: [@joplin/app-desktop]: 
➤ YN0000: [@joplin/app-desktop]:     Error: Timed out 5000ms waiting for expect(locator).toHaveAttribute(expected)
➤ YN0000: [@joplin/app-desktop]: 
➤ YN0000: [@joplin/app-desktop]:     Locator: locator('.rli-noteList').getByRole('option', { name: 'note_3' })
➤ YN0000: [@joplin/app-desktop]:     Expected string: "true"
➤ YN0000: [@joplin/app-desktop]:     Received string: "false"
➤ YN0000: [@joplin/app-desktop]:     Call log:
➤ YN0000: [@joplin/app-desktop]:       - expect.toHaveAttribute with timeout 5000ms
➤ YN0000: [@joplin/app-desktop]:       - waiting for locator('.rli-noteList').getByRole('option', { name: 'note_3' })
➤ YN0000: [@joplin/app-desktop]:       -   locator resolved to <div tabindex="-1" role="option" draggable="true" aria-setsize="4" aria-posinset="4" aria-selected="false" class="note-list-item-wrapper even" data-id="2c397688bf924756860ddab325baa3de" id="list-note-2c397688bf924756860ddab325baa3de">…</div>
➤ YN0000: [@joplin/app-desktop]:       -   unexpected value "false"
➤ YN0000: [@joplin/app-desktop]:       -   locator resolved to <div tabindex="-1" role="option" draggable="true" aria-setsize="4" aria-posinset="4" aria-selected="false" class="note-list-item-wrapper even" data-id="2c397688bf924756860ddab325baa3de" id="list-note-2c397688bf924756860ddab325baa3de">…</div>
➤ YN0000: [@joplin/app-desktop]:       -   unexpected value "false"
➤ YN0000: [@joplin/app-desktop]:       -   locator resolved to <div tabindex="-1" role="option" draggable="true" aria-setsize="4" aria-posinset="3" aria-selected="false" class="note-list-item-wrapper odd" data-id="2c397688bf924756860ddab325baa3de" id="list-note-2c397688bf924756860ddab325baa3de">…</div>
➤ YN0000: [@joplin/app-desktop]:       -   unexpected value "false"
➤ YN0000: [@joplin/app-desktop]:       -   locator resolved to <div tabindex="-1" role="option" draggable="true" aria-setsize="4" aria-posinset="3" aria-selected="false" class="note-list-item-wrapper odd" data-id="2c397688bf924756860ddab325baa3de" id="list-note-2c397688bf924756860ddab325baa3de">…</div>
➤ YN0000: [@joplin/app-desktop]:       -   unexpected value "false"
➤ YN0000: [@joplin/app-desktop]:       -   locator resolved to <div tabindex="-1" role="option" draggable="true" aria-setsize="4" aria-posinset="3" aria-selected="false" class="note-list-item-wrapper odd" data-id="2c397688bf924756860ddab325baa3de" id="list-note-2c397688bf924756860ddab325baa3de">…</div>
➤ YN0000: [@joplin/app-desktop]:       -   unexpected value "false"
➤ YN0000: [@joplin/app-desktop]:       -   locator resolved to <div tabindex="-1" role="option" draggable="true" aria-setsize="4" aria-posinset="3" aria-selected="false" class="note-list-item-wrapper odd" data-id="2c397688bf924756860ddab325baa3de" id="list-note-2c397688bf924756860ddab325baa3de">…</div>
➤ YN0000: [@joplin/app-desktop]:       -   unexpected value "false"
➤ YN0000: [@joplin/app-desktop]:       -   locator resolved to <div tabindex="-1" role="option" draggable="true" aria-setsize="4" aria-posinset="3" aria-selected="false" class="note-list-item-wrapper odd" data-id="2c397688bf924756860ddab325baa3de" id="list-note-2c397688bf924756860ddab325baa3de">…</div>
➤ YN0000: [@joplin/app-desktop]:       -   unexpected value "false"
➤ YN0000: [@joplin/app-desktop]:       -   locator resolved to <div tabindex="-1" role="option" draggable="true" aria-setsize="4" aria-posinset="3" aria-selected="false" class="note-list-item-wrapper odd" data-id="2c397688bf924756860ddab325baa3de" id="list-note-2c397688bf924756860ddab325baa3de">…</div>
➤ YN0000: [@joplin/app-desktop]:       -   unexpected value "false"
➤ YN0000: [@joplin/app-desktop]:       -   locator resolved to <div tabindex="-1" role="option" draggable="true" aria-setsize="4" aria-posinset="3" aria-selected="false" class="note-list-item-wrapper odd" data-id="2c397688bf924756860ddab325baa3de" id="list-note-2c397688bf924756860ddab325baa3de">…</div>
➤ YN0000: [@joplin/app-desktop]:       -   unexpected value "false"
➤ YN0000: [@joplin/app-desktop]: 
➤ YN0000: [@joplin/app-desktop]: 
➤ YN0000: [@joplin/app-desktop]:        at models/NoteList.ts:36
➤ YN0000: [@joplin/app-desktop]: 
➤ YN0000: [@joplin/app-desktop]:       34 |
➤ YN0000: [@joplin/app-desktop]:       35 | 	public async expectNoteToBeSelected(title: string|RegExp) {
➤ YN0000: [@joplin/app-desktop]:     > 36 | 		await expect(this.getNoteItemByTitle(title)).toHaveAttribute('aria-selected', 'true');
➤ YN0000: [@joplin/app-desktop]:          | 		                                             ^
➤ YN0000: [@joplin/app-desktop]:       37 | 	}
➤ YN0000: [@joplin/app-desktop]:       38 | }
➤ YN0000: [@joplin/app-desktop]:       39 |
➤ YN0000: [@joplin/app-desktop]: 
➤ YN0000: [@joplin/app-desktop]:         at NoteList.expectNoteToBeSelected (/Users/runner/work/joplin/joplin/packages/app-desktop/integration-tests/models/NoteList.ts:36:48)
➤ YN0000: [@joplin/app-desktop]:         at /Users/runner/work/joplin/joplin/packages/app-desktop/integration-tests/noteList.spec.ts:106:18

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
accessibility Related to accessibility mobile All mobile platforms
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant