Completion problems

[rgoldberg]

There are numerous completion problems: some only for certain shells, some for specific CompletionKinds for certain shells, and some that are only for arguments but not for option values.

bash

• Offers candidates for all positional arguments regardless of current argument position
• Repeatedly offers singular options instead of just once
• Offers flags & options after --
• directory not implemented for arguments
• file not implemented for arguments
• directory & file(…) for option values moves to next parameter after selecting the first item, even if you've submitted a directory, instead of allowing you to request candidates from within the selected directories. If you move back to the parameter & append a /, tabbing will then offer properly filtered candidates.

There might not be a way in bash 3.2.57, or even in later bashes, to improve the following:

• outputs to stdout, then opens a new prompt rather than interactive output below command line

fish

• Offers candidates for all positional arguments regardless of current argument position
• Doesn't offer flags or options unless you type a leading -
• Repeatedly offers singular options instead of just once
• Repeats parameter help text for each candidate
• file() offers no candidates when no extensions were specified
• file(extensions:) doesn't offer directories

zsh

• shellCommand does not offer any candidates

There aren't necessarily bugs, they are just potentially suboptimal setups / aren't properly documented in the docs or code:

• If the user hasn't already typed a leading -, doesn't offer flags or options if there are unsupplied positional arguments
• file(extension:) if existing part of the completing parameter is a path to a leaf directory, and if no non-hidden files in that directory match any of the given extensions, then all non-hidden files in that directory are offered as candidates, regardless of their respective extensions. Is that intended? If so, it should be documented.

all

The following @Option doesn't offer true & false. Using a @Flag would probably be better, but if a developer wants to use a Bool @Option, the completion scripts should probably offer true & false without additional setup being required.

@Option(help: "Case Sensitive")
var caseSensitive: Bool

Documentation

The CompletionKind / CompletionKind.Kind documentation is also sometimes wrong or confusing, so I will update that along with the associated code.

PR

I will work on a PR to fix these problems. If I find additional problems, I'll add them to this issue.

Related Issues

• CompletionKind should support multiple options #566: My PR will fix the problems mentioned, but won't implement the proposed solution.
• Add completion for default subcommand to supercommand #488: My PR won't add support for this.
• Custom fish completions do not work #376: It looks like this has already been fixed, but if it somehow resurfaces, I'll fix it.

Versions

ArgumentParser version:
main branch

Swift version:

swift-driver version: 1.62.15 Apple Swift version 5.7.2 (swiftlang-5.7.2.135.5 clang-1400.0.29.51)
Target: x86_64-apple-macosx12.0

Checklist

• If possible, I've reproduced the issue using the main branch of this package
• I've searched for existing GitHub issues

Steps to Reproduce

Generate completion scripts using the aforementioned CompletionKinds.

Try to use them.

Expected behavior

Completion works.

Actual behavior

Some completion either does nothing, or outputs an error.

See per-shell sections above for details.

[rgoldberg]

I am partway through a PR for this, with many improvements for all 3 shells.

What is the oldest version of each shell with which the respective script must maintain compatibility? Here are some guesses:

• bash 3.2.57 (included in macOS 10.13)
• zsh 5.3 (included in macOS 10.13)
• fish 3.7.1 (latest fish for almost a year; fish is not included with any macOS, so we can expect users to know about upgrading it)

I know that SAP can be used on OSes other than macOS (sorry to be Mac-centric, but I don't know shell versions distributed with / popular for other OSes).

Supporting bash 3.2.57 might be annoying because it doesn't support associative arrays. Being able to require some bash 4.x or (better yet) 5.x might simplify things.

[natecook1000]

If we can make things work with the default-installed Bash version, I think we should. I know it's a bit of a pain, but it would feel like a mismatch (and probably generate more issues) to require Bash v4 or v5. The other two shell versions both seem reasonable!

[rgoldberg]

@natecook1000 thanks. Will do.

2 questions:

Overloaded Swift custom completion functions

I think there should be 2 extra arguments to Swift custom completion functions:

1. 0-based index of the word for which completion candidates are being requested. This is necessary because 2 (all 3 after my fixes, as long as it's fish 4+) of the shells include any command line words that are after the word for which completions are being requested; without this index, there's no way of knowing the word for which completions are being requested.

2. 0-based index of the location of the cursor within the word for which completion candidates are being requested. This is useful because a user could request completions from the before any character in the word, or at the end of the word. Maybe a custom Swift completion function would like to know the exact cursor location.

I would leave the existing single-argument function as is but deprecate it, and add & recommend the new three-argument overload.

Completion inconsistencies between shells

There are inconsistencies amongst how the completions behave in the 3 shells.

Is it OK if I make all 3 behave as similarly as possible? I'll try to document all inconsistencies that I couldn't overcome within a reasonable amount of effort.

I will standardize around what I perceive to be the best completion behavior. In most cases, this will be closest to the current zsh behavior. I'll attempt to note all behavior changes.

Most inconsistencies seem to be contained within the shell completion behavior itself, so fixing them should be backwards compatible with existing SAP uses.

The 3 shells, however, pass the command-line words to Swift custom completion functions in varied ways. Making that info consistent, and fixing bugs for which people might already workarounds, could break existing Swift custom completion functions. See below for more info.

[rgoldberg]

@natecook1000 Thanks for merging my 2 other PRs.

Here some of the problems I'm fixing. There are many more issues, but the following are ones about which I have open questions.

Should I include anyone else in this discussion?

Backwards compatibility

The way that Strings are passed from Swift to shell completion scripts is broken in numerous ways in SAP 1.5.0.

Frequently, they aren't properly escaped (so the whole completion script can break; Swift either must not include certain values as completions, or must escape things properly in some places, but not escape them in others, differently for the different shells, etc.), but sometimes they are, so devs never know what they must do.

Some Strings are split into different words on spaces, so values cannot contain spaces, but some other places (or for different shells), spaces are acceptable.

Etc.

Devs might have worked around all these bugs, but many workarounds won't work properly after my PR fixes all these problems.

It will be easiest to not try to be compatible with the old, incredibly broken way of handling Strings. My PR will be binary compatible (with one deprecation that is easy to fix; users can just add 2 arguments to existing custom completion functions, and they can ignore the arguments and let their old code run without deprecation warnings), but String`s will be consistently & correctly handled (barring oversights in my PR). It's probably enough of a change that SAP should be bumped up a major version, just to indicate how existing workarounds could break.

Custom completion command line contents

Simple question

Should the shells pass the full command line to Swift custom completion functions? That would include other commands (separated by ;) before and/or after the commend for which completions are being requested, pipes into and/or of that command, etc. That would simplify things for SAP implementation & provide the user with the most data, but it might be harder for them to parse stuff (& might give tools that SAP shell completion more data than they should get). Or should the shells only send the section of the command line for the command for which completions are being requested? (that's the current behavior).

If Swift shouldn't receive the full command line, what should mark things to excise? I assume ;, |, &, &&, ||, …?

Details

I have found multiple inconsistencies & problems with how the contents of the current command line are supplied to Swift custom completion functions by the 3 supported shells.

Undiscovered inconsistencies & problems might exist.

None of the shells supplies all words from the command line that we should want shells to pass to Swift custom completion functions.

We will unfortunately need to supply verbatim (instead of unquoted) words to Swift; otherwise, e.g., $x & '$x', etc. would incorrectly look the same.

My upcoming PR will support 2 types of custom functions:

1. the existing single-argument SAP 1.5.0 function, which will be deprecated
2. a new three-argument function (with 2 index arguments), which will replace the deprecated function

We could try to keep the broken existing behavior in the deprecated function, but there are so many other problems with completions that will require breaking fixes that we probably want to bump major version, and fix the problems in the deprecated function, too.

In examples:

• ^ is the location of the cursor when TAB is pressed; it is not a literal ^.
• … is a placeholder for the relevant word(s); it is not a literal ….
• Command lines can extend before or after the relevant example, except they cannot extend before a leading com.
• "a ➡ b" indicates that a from the command line becomes b as a Swift argument

Table detailing how the command line is communicated from shells to Swift custom completion functions:

feature	example	bash	fish	zsh
accepts args before current subcommand	com … sub ^	🎉1	✅	✅
includes words before current subcommand	… sub ^	✅	✅	🎉
includes non-empty current word	current^	✅	🎉	✅
includes empty current word not followed by other words	^	✅	🎉	🎉
includes empty current word followed by other words	^ later	🎉	🎉	🎉
includes words after current word	^ …	✅	4+	✅
preserves quoting (i.e. supplies verbatim words)	'$x' ➡ '$x'	✅	4+	✅
does not expand params, etc.	$x ➡ $x	✅	✅	✅
includes redirects (separate symbols & source/target)	<…, >…, >>…, …	✅	✅2	-
includes process substitution (as one word)	<(…)	✅34	✅	✅
includes command substitution as one word	$(…)	✅	✅	✅
omits pipes in	…|	✅	✅	✅
omits pipes out	|…	✅	✅	✅
omits prior commands	…;	✅	✅	✅
omits later commands	;…	✅	✅	✅

• ✅: Already works in SAP 1.5.0.
• 🎉: Fixed in my upcoming PR.
• X+: Fixed in my upcoming PR for shell version X+, but not for earlier versions of the shell.
• -: Does not work in SAP 1.5.0. Haven't yet investigated a solution. Might not be able to fix.
• ?: Must investigate if it already works or is even supported by the shell. Might not be able to fix possible problems.

Footnotes

1. Fixed in my upcoming PR. In SAP 1.5.0, if arguments exist before the current subcommand, Swift custom completion is never called (I haven't yet investigated exactly what arguments cause this, if not all do). ↩

2. redirect symbols (<, >, >>, …) omitted ↩

3. <( & each space-delimited token in …) separate ↩

4. For bash 3.2.57, if a process substitution is on the command line, Swift custom completion is never called; for bash 5.2.37, however, completion is called. Not sure about other versions. Will try 4.0 later, then possibly others even later. ↩

[rgoldberg]

Testing platforms for custom completion command line contents

The above results were obtained on macOS for the following shell versions:

• bash 3.2.57
• bash 5.2.37
• fish 3.7.1
• fish 4.0b1
• zsh 5.8.1
• zsh 5.9

All versions of a shell behaved the same, unless explicitly noted.

Will try to test zsh 5.3, bash 4.0, and any shell versions released from now until when the PR is merged.

Should I test any other shell versions?

I do not have any OSes other than macOS handy. If testing must be performed on other OSes (or on multiple versions of macOS), can you find someone else to handle them?

[rgoldberg]

Questions about custom completion command line contents

1. Do we want input and/or output redirects to be included? What if there is no way to get all the shells to either include or exclude them in identifiable ways?
2. I'd want to make process substitutions, redirects, etc. look the same regardless of the shell. So I'd either concatenate or split them in a standard manner. In the examples below, each separate code fragment is its own element in the [String] argument to a Swift custom completion function:
   1. Constructs with start & end delimiters, e.g.:
      1. Prefer <(proc-sub a b)
      2. Possibly < & (proc-sub a b)
      3. Possibly <( & proc-sub a b & )
      4. Definitely not <( & proc-sub a b)
      5. Definitely not <( & proc-sub & a & b)
      6. Definitely not <( & proc-sub & a & b & )
   2. Constructs with only a start marker, e.g.:
      1. Prefer > & out
3. Do we want parameters, command substitutions, and/or anything else that can be evaluated on the command line passed verbatim as in the command line, or evaluated (i.e. expanded)?
   1. I'd say no to everything besides parameters (because of side effects and/or runtime), but possibly OK to parameters.

[rgoldberg]

Swift completion output

After my upcoming PR, all completion candidates supplied from Swift must be output as the verbatim text for the equivalent of a zsh single-quoted string (i.e. no escape values are supported), except that single quotes may be used within the output values.

For shells (like bash & zsh) that support verbatim single-quoted strings (no escapes, single quotes not allowed as direct content), SAP will properly replace each single quote in the content by splitting the output into multiple single-quoted strings concatenated with escaped single quotes between them.

For shells (like fish) that do not support verbatim single-quoted strings, SAP will properly escape all necessary characters (e.g., for fish shell, \ ➡ \\ & ' ➡ \')

For shellCommand, the commands should be output expecting to be inserted into a string to be passed to eval. From the Swift perspective, the strings will be handled the same way as per completion values (the actual escaping by SAP might be different, but Swift users won't see any difference).

There are many problems with how SAP 1.5.0 parses completions from Swift. Some (if not all) types of completions in the different shells split completions on spaces (so a b becomes 2 separate completions: a & b.)

It's so broken, I think it makes sense to only follow the consistent, fixed rules that will be followed by my PR (and will be considered bugs if not obeyed).

This might technically require SAP to bump by a major version.

[rgoldberg]

Index issues:

fish 3-: words are unquoted, but "cursor index in current word" is based on the verbatim characters, so it could be greater than it should be. Fixed in fish 4+. If someone else wants to improve this for fish 3-, they can.

zsh: "cursor index in current word" is one greater than correct if the cursor is immediately after a backslash used as an escape (i.e. zsh reports the cursor as after the character that is escaped by the backslash, instead of after the backslash itself).

[rgoldberg]

@natecook1000

In what format do you want text to be passed between shells & Swift in the various places where they can communicate?

The new SAP_SHELL & SAP_SHELL_VERSION environment variables will allow Swift code to know the shell & version with which Swift is interacting, so Swift code can properly parse/format verbatim text for the specific shell. It might be annoying/complex to deal with such parsing/formatting, so verbatim text should be avoided as much as possible, but it might be unavoidable in certain circumstances.

The following ignores environment variables as a communication method between shells & Swift. Anyone using them can do whatever they want with them.

It's also what I've gathered going through this all. If anything is incorrect, please let me know.

From shell to Swift

Text is sent from a shell to Swift only for custom completions, which pass the contents of part of the command line from the shell to Swift.

In this case, we must send verbatim text, otherwise there would be no way to distinguish between:

• a variable reference & a string literal that, when unquoted, looks like a variable reference ($x vs. '$x')
• a command substitution & a lookalike string literal (<(…) vs. '<(…)')
• …

Are there any bash, fish, or zsh parsing libraries in Swift that we can recommend to users?

The various shells send command lines differently in SAP 1.5.0, some in broken ways; I am fixing that.

From Swift to shell

Text sent from Swift to a shell should probably be expected to be in different formats depending on the use case.

The only places about which I know where Swift sends text to a shell are in the completion functions, and in the command, subcommand, flag & option names.

Completion functions:

• case list([String])
  • Contents of each element of the Swift [String] are a literal value of a single completion value.
  • Neither escapes nor references are supported.
    • No escapes: \\ will behave as a literal string containing two backslashes (instead of a single escaped backslash).
    • No references: $x will behave as a literal string with a $ followed by an x; it will not reference the value of a variable named x.
  • Characters that must be escaped within the shell to behave properly in the shell will be escaped by SAP (in or by the script).
  • SAP could use one of the following characters as a delimiter between elements from the [String]:
    • NUL
      • harder to implement
      • allows completions to contain newlines
    • newline
      • easier to implement
      • prevents completions from containing newlines
      • easier to debug
      • if completions shouldn't contain newlines, easier to do this
• case custom(@Sendable ([String], Int, Int) -> [String])
  • Same as case list([String])
• case default
  • Same as case list([String])
• case file(extensions: [String])
  • Contents of each element of the returned [String] are a glob, which will require correct escaping for each shell:
    • e.g., in zsh, \1*$x will match any extension that begins with a literal 1 & ends with the value of the variable x; \\1*\$x will match any extension that begins with a literal \1 & ends with a literal $x.
• case shellCommand(String)
  • Contents of Swift String are a verbatim shell script that will be run via the equivalent of zsh eval (so a bad script in the String will not break the completion script)
    • e.g., $x will reference the value of the variable named x

Command, subcommand, flag & option names should behave like case list([String]), but implementation would be much simpler if the allowable characters could be constrained. escaping whitespace, quotes, other shell symbols (e.g., $, ;, …), etc. would be exceedingly cumbersome. Can the supported characters be limited those that do not require escaping or quoting in the shell? Does the following regex specify all the characters I should test (I won't specifically limit supported characters to the following regex, but I won't test Chinese characters or emoji, even if they probably will work, unless you tell me I shoul)?

[-+._:0-9A-Za-z]

I have seen /bin/[ & /usr/local/bin/g[. [ would require escaping everywhere, but I'd really prefer to not have to deal with that possibility everywhere.

[rgoldberg]

In what order should arrays of completions be shown? The two main options would be

1. sorted (presumably lexicographically, but taking into account full numbers instead of sorting by successive digits; e.g., 10 would be sorted after 9)
2. in the original order supplied from Swift

I think the former is mainly (if not exclusively) what is done in 1.5.0, but I prefer the latter. If you want your completions to be sorted a certain way, the latter provides flexibility, while the former is restrictive.

[natecook1000]

@rgoldberg Can't thank you enough for your investigation and work on these issues! I've gone section by section through the above -- please let me know if you're blocked on my responses and we can discuss further.

Overloaded Swift custom completion functions

Both those points look reasonable to me, as long as the overload and deprecation aren't source-breaking. Let's make sure that we include tests that validate that the single-parameter function is still usable. Providing two Ints seems like the only viable option, but it will be easy to confuse them - what would you think about using a labeled tuple or a struct to give them names?

Completion inconsistencies between shells

I will standardize around what I perceive to be the best completion behavior. In most cases, this will be closest to the current zsh behavior. I'll attempt to note all behavior changes.

Absolutely – this is a great goal across the board.

Backwards compatibility

Devs might have worked around all these bugs, but many workarounds won't work properly after my PR fixes all these problems.
...
It's probably enough of a change that SAP should be bumped up a major version, just to indicate how existing workarounds could break.

Let's take a look at how breaking these changes are as we land them. My expectation is that you will be fixing far more latent bugs in tools that rely on ArgumentParser than you will be breaking workarounds.

Custom completion command line contents

It makes the most sense to me that command-line tools only get the section of the command line that is relevant to that tool, if that is possible -- tools shouldn't get data that isn't intended for them. Your list of operator delimiters looks good for what marks the section -- does that work for you?

Testing platforms

My main concern is about having a way to test these in an automated fashion – even that matrix is more than I want anyone (including/especially you) to have to check and validate manually. Have you used existing tools (like expect?) for testing completion scripts? Have you given thought to what one might look like that we could include in the libary?

Questions about custom completion command line contents

Do we want input and/or output redirects to be included?

Redirects are outside the scope of a command's execution, so I'd rather they not be included. Are there contexts where information about redirects necessarily needs to be relevant when generating completions?

Swift completion output

I think I'll need to see the change in behavior to fully understand what you're describing in this section.

Index issues

Re: fish 3 and zsh "cursor index in current word": Let's make sure we document the inconsistencies where we find them, if there isn't a reasonable fix that we can apply.

From shell to Swift

Are there any bash, fish, or zsh parsing libraries in Swift that we can recommend to users?

I don't know of any - we could look at providing Swift wrappers for parsing/expansion/etc via each of the shells as a future enhancement.

From Swift to shell

Overall, I'd rather look at adding overloads that allow a more structured return type than finding delimiters to use in strings. Would it work to let people return something shell-agnostic that ArgumentParser could then format correctly for the destination shell?

Re: limiting the character set – is this in completions, or in option/flag/argument names more broadly? The character set you've described looks good, but I think we'll need to be careful if we're imposing a limit that existing tools haven't had to deal with.

Order of arrays of completions

As far as I know, ArgumentParser isn't doing sorting of any kind – is that a behavior the shells are providing? I'm happy for it to be in the manually specified order if that's possible.

[rgoldberg]

@natecook1000 Thanks for the replies. I'll look over them soon.

Can I start breaking out sets of fixes to submit as PRs? Should I make a separate issue for each PR?

So I can coordinate PR sizes with the reviewer, will you or will someone else review my PRs?

[rgoldberg]

@natecook1000 Thanks again for feedback. I'll get back to you if I run into anything else for which you haven't already supplied answers.

Overloaded Swift custom completion functions

Both those points look reasonable to me, as long as the overload and deprecation aren't source-breaking. Let's make sure that we include tests that validate that the single-parameter function is still usable. Providing two Ints seems like the only viable option, but it will be easy to confuse them - what would you think about using a labeled tuple or a struct to give them names?

I'm happy to do whatever you want. Would the tuple or struct also include the [String] argument or just the Ints?

Custom completion command line contents

It makes the most sense to me that command-line tools only get the section of the command line that is relevant to that tool, if that is possible -- tools shouldn't get data that isn't intended for them. Your list of operator delimiters looks good for what marks the section -- does that work for you?

See my answer below about redirects for a discussion about why including redirects & potentially pipes (at least incoming ones) might be useful.

If we do include pipes (& potentially other command-line segments), we might want to include 2 more indices: the index of first word of current command, and (one more than?) the index of last word of current command (or just the word count for the current command).

Even if I can't think of reasons to include other command-line segments right now, someone else might have a reason at some point. The more info we provide, the less likely we are to leave something out that someone needs. If someone writes a script by hand, they have access to the full command line, so why not provide the whole thing to Swift custom completion functions, too? (with indices so users can get commonly used subsets)

Testing platforms

My main concern is about having a way to test these in an automated fashion – even that matrix is more than I want anyone (including/especially you) to have to check and validate manually. Have you used existing tools (like expect?) for testing completion scripts? Have you given thought to what one might look like that we could include in the libary?

I haven't looked into automating this. I just did my tests manually as I needed to investigate how scripts interacted with their respective shell.

I'd imagine that the best way to automate testing for all this would be to test all builtin shells from macOS on each macOS version for which there is a GitHub runner image, the latest stable version of each of the shells (obtained via Homebrew), and designated versions of shells (I don't currently know how those designated versions would be installed).

I imagine that there'd be tests that start a command line, enter text, enter a TAB, then scrape the stderr (or stdout, as appropriate) for the completions.

It's all a bit hand wavy, as I imagine that any such automated testing wouldn't be implemented until after all the fixes are in.

Questions about custom completion command line contents

Do we want input and/or output redirects to be included?

Redirects are outside the scope of a command's execution, so I'd rather they not be included. Are there contexts where information about redirects necessarily needs to be relevant when generating completions?

I would imagine that an in redirect (or an incoming pipe) might be useful to know because some commands require input either from stdin or from a file specified by a path. If you already have the redirect (or pipe), completions for an input file path shouldn't be offered. I don't think there's an equivalent situation for out redirects (or outgoing pipes), however, because in the absence of an out redirect (or pipe), the output will go to the console. (Am I missing anything?)

If the above is correct, there would be an inconsistency if only including in redirects but not incoming pipes, and an asymmetry if only including in redirects & pipes but not out redirects & pipes.

The various shell scripts currently all omit pipes from the sources that we use for command lines, but they handle redirects differently. I think we can get the full command line from each of the shells, so we can always manually select whatever we want, but that will take more time, be more verbose, more error-prone, and likely more brittle for forward compatibility (e.g., a shell could introduce a new symbol that we must parse out).

Swift completion output

I think I'll need to see the change in behavior to fully understand what you're describing in this section.

Sorry, please ignore that section; my proposals therein were inadvisable. I went over the same issues elsewhere with better solutions.

My general principle is for Swift to receive & to output text without having to know shell syntax as much as possible, but to use shell syntax where necessary. This can now be done thanks to the 2 singletons I recently introduced.

My forthcoming PRs will additionally ensure that malformed shell text from users will not break scripts outside of the individual completion for which they were specified (e.g., for some completions for some shells, a quote in a completion will completely break the generated script rather than just result in that one completion not working).

From Swift to shell

Overall, I'd rather look at adding overloads that allow a more structured return type than finding delimiters to use in strings. Would it work to let people return something shell-agnostic that ArgumentParser could then format correctly for the destination shell?

For list completions that are written into the script, we could create an array instead of a delimited string (which I've already done in my local branch for at least one shell).

IIRC, shellCommand & custom both provide completions by writing to stdout, for which you must have a delimiter.

We could avoid delimiters by using environment variables, but that would get much more complex than just using either NUL (i.e. Unicode code point 0; not the 3 letters NUL) or newline as a delimiter, and would break existing shellCommand completions.

I can't imagine either NUL or newline being valid characters within a completion; newline would be simpler to use than NUL. If we use one of those characters as a delimiter, it shouldn't be supported by any other completion, so it should be safe to use for list completions in shells where that is simpler than an array.

Do you agree that we should use some delimiter? If so, can we use newline & NUL as appropriate (NUL might be simpler/better than newline in some places, and vice versa), or must I use NUL? (I can't imagine that newline would be able to be used but not NUL, so, unless you disagree, I expect to use just NUL, or to choose between NUL & newline as convenient)

Re: limiting the character set – is this in completions, or in option/flag/argument names more broadly? The character set you've described looks good, but I think we'll need to be careful if we're imposing a limit that existing tools haven't had to deal with.

I think that, in the long run, the completion scripts should support all valid flags, options, and (sub)commands. Currently, I'm sure that various characters will break the completion scripts (probably spaces, newlines, special shell characters (e.g., $)). Are any characters currently not allowed for flags, options, and/or (sub)commands? If so, are they listed anywhere, are there any programmatic checks, and should any additional characters be made unsupported? If not, can certain characters be designated as unsupported (& which ones)? Are there any tests to ensure that SAP supports unconventional characters in various locations?

Order of arrays of completions

As far as I know, ArgumentParser isn't doing sorting of any kind – is that a behavior the shells are providing? I'm happy for it to be in the manually specified order if that's possible.

IIRC, at least some of the shell scripts sort some of the completions. I think that file & directory completions should be sorted by the scripts, but the other completions should remain in the order output by the Swift code, since the Swift code can order the completions if it so desires, but it can't affect the order of file or directory completions.