[Dialog] Focus gets lost under certain conditions with nested dialogs

[atomiks]

Bug report

Current behavior

1. Nested dialogs: open nested dialog and press outside with a pointer. Focus is ignored and not returned to its trigger, so subsequent tabs aren't trapped unless pressing inside the dialog with the pointer
2. Close confirmation: type a char in the textarea and press Cancel. Press the esc key 3 times quickly. Focus gets returned to the root trigger, not the Cancel trigger

Expected behavior

1. Focus should return to the trigger with preventScroll: true if it's available for outside presses instead of being ignored
2. Focus shouldn't return to the root and should stay trapped

Reproducible example

Link to a CodeSandbox, StackBlitz, or other IDE.

Base UI version

1.0.0-alpha.6

Which browser are you using?

Chrome/Safari/Firefox/Safari iOS etc.

Which OS are you using?

Mac OS/Windows/Other etc.

Which assistive tech are you using (if applicable)?

Voiceover/JAWS etc.

Additional context

Provide any additional context that might help us identify the problem and find a solution.