Buggy behavior/Observations using mobile VoiceOver

[Nancy-Salpepi]

Test device
iPad 9th generation

Operating System
iPadOS 18.3.1

Browser
Safari + VO

Problem description
For phetsims/qa#1214, I thought it would be easiest to put this all in one issue, but feel free to separate out. Here are some of the issues I saw when testing with mobile VoiceOver:

• When closing the Transitions dialog I hear: "For closer observations, play sim on slower speed."
• In dev version: Closing a dialog sometimes takes me out of the sim--info dialog, oops dialog. Focus is on the Show Sidebar button or on a tab, but this seems to be fixed on main.
• I can’t move the transitions dialog, but mobile VO reads what is behind it. Is that OK?
• When toggling between experiment and model, I sometimes hear the word “if” before "experiment, switch to model" or "model, switch to experiment. " -- see first video
• It isn’t possible to be precise with the slider thumb. I have to use the buttons in the transitions dialog to set specific wavelengths. Is that OK? --see second video

IF.mov

largeJumps.mov

[arouinfar]

I can’t move the transitions dialog, but mobile VO reads what is behind it. Is that OK?

@Nancy-Salpepi are the obscured controls still interactive? Can you switch the atomic model or pause the sim, for example? If so, this behavior seems acceptable. I think the devs should still investigate if/how the dialog can be movable with mobile VO, though.

It isn’t possible to be precise with the slider thumb. I have to use the buttons in the transitions dialog to set specific wavelengths. Is that OK? --see second video

Precise wavelengths are unimportant except those listed in the transitions dialog with buttons, so this behavior is reasonable.

[Nancy-Salpepi]

@Nancy-Salpepi are the obscured controls still interactive? Can you switch the atomic model or pause the sim, for example? If so, this behavior seems acceptable. I think the devs should still investigate if/how the dialog can be movable with mobile VO, though.

Yes. I can still interact with the obscured controls.

[jessegreenberg]

I did some testing using iOS 17.5.1.

When closing the Transitions dialog I hear: "For closer observations, play sim on slower speed."

I am seeing this too about half the times I try. I don't know why VO puts the virtual cursor there when the dialog closes. A potential fix is to put focus back on the checkbox after closing the non-modal dialog.

In dev version: Closing a dialog sometimes takes me out of the sim--info dialog, oops dialog. Focus is on the Show Sidebar button or on a tab, but this seems to be fixed on main.

I was not able to reproduce this with the info dialog. It may be happening for the oops dialog for the same reason as the transitions dialog - might be fixed by manually putting focus somewhere after the dialog closes.

I can’t move the transitions dialog, but mobile VO reads what is behind it. Is that OK?

I am not sure what we should do about this. Lets discuss during a design meeting. We have one scheduled for tomorrow.

When toggling between experiment and model, I sometimes hear the word “if” before "experiment, switch to model" or "model, switch to experiment.

I hear something too, but I don't think it is "if" - I think it is the previous accessible name. ABSwitch changes the accessible name of the component to read the new value. My theory is that VO starts to read the old name, then we change the accessibleName, and then VO is interrupted and reads the new one. I don't know if we can or should do anything about this.

It isn’t possible to be precise with the slider thumb.

True, iOS VO does not support the shift keyboard step sizes.

[pixelzoom]

I'm going to unassigned myself because there is nothing sim-specific about this problem, and I have no idea how to address it in common code. We can either hold up MOTHA until it's fixed in common code, or we can publish MOTHA with this as a "known problem" and do an MR later.

[pixelzoom]

@jessegreenberg For problems related to the Transitions "dialog"... Reminder that we are faking a non-modal dialog using a Panel, in. case that's the source of the problem. And it seems like we should address phetsims/sun#916 (create a non-modal dialog) instead of continuing to add workaround to TransitionsDialog.ts.

[terracoda]

I the "if" might be a copy and paste error from the original explanation of the AB Switch. The names should follow this pattern:

<button>A, Switch to B</button> <button>B, Switch to A</button>

If VoiceOver is providing the "if" all on its own, then it is another VO bug.