Very slow on remote filesystems

[fwsmit]

When cd'd into a remote filesystem, the syntax highlighter can be very slow, impacting how quickly text appears in the prompt. This wouldn't be an issue if the highlighting was turned off in this case.

Might be solved with #361

[danielshahaf]

See https://github.com/zsh-users/zsh-syntax-highlighting/blob/master/docs/highlighters/main.md#parameters.

Please either close the issue or identify a use-case that's not addressed by the existing functionality.

[fwsmit]

Sure, it can be solved with the ZSH_HIGHLIGHT_DIRS_BLACKLIST, but ideally the plugin can figure this out for itself. I've had issues with this for a long time and wasn't sure what caused it. So if this isn't done automatically, or the syntax highlighting is done in parallel, I consider it a bug (though you might otherwise). Feel free to close this issue though.

[danielshahaf]

Sure, it can be solved with the ZSH_HIGHLIGHT_DIRS_BLACKLIST, but ideally the plugin can figure this out for itself.

How?

I guess this is going to be OS-dependent? We can do OS-dependent code (branching on $OSTYPE and $(uname -s; uname -r)), but it might be better to leave such work to library plugins that we use in a blackbox fashion.

So if this isn't done automatically, or the syntax highlighting is done in parallel,

What's "in parallel" in this context?

Feel free to close this issue though.

Not yet; we haven't yet finished evaluating the feature request (now that we know there is one). As to why I wrote what I did, see https://producingoss.com/en/you-are-what-you-write.html#writing-tone.

[fwsmit]

Sure, it can be solved with the ZSH_HIGHLIGHT_DIRS_BLACKLIST, but ideally the plugin can figure this out for itself.

How?

For example by implementing #361. This could be combined with blacklisting a directory when it times out multiple times in a row.

I guess this is going to be OS-dependent? We can do OS-dependent code (branching on $OSTYPE and $(uname -s; uname -r)), but it might be better to leave such work to library plugins that we use in a blackbox fashion.

I don't think it's needed to explicitly detect remote file systems as some might be fast enough. I prefer the above method.

So if this isn't done automatically, or the syntax highlighting is done in parallel,

What's "in parallel" in this context?

That the typing of text isn't inhibited by a slow file system. When you are cd'd in a slow directory, all typing seems to be slowed down, making it almost if not entirely impossible to run a command (even if the command doesn't need the file system).

Feel free to close this issue though.

Not yet; we haven't yet finished evaluating the feature request (now that we know there is one). As to why I wrote what I did, see https://producingoss.com/en/you-are-what-you-write.html#writing-tone.

Thanks for taking the time to evaluate the options :)

[danielshahaf]

That the typing of text isn't inhibited by a slow file system. When you are cd'd in a slow directory, all typing seems to be slowed down, making it almost if not entirely impossible to run a command (even if the command doesn't need the file system).

The issue here is that the command-line is highlighted from square one on every keypress. You can probably workaround this right now by using the stty(1) flow control keys (default Ctrl+S and Ctrl+Q) while typing a command-line that involves a slow-to-highlight word: cp /networkfilesys<Ctrl+S>tem/foo /tmp/bar<Ctrl+Q><Enter> would do only O(1) remote stat()s, rather than one per character in the command-line.

---

So, that being said, let's first enumerate all the different ideas here:

• Cap highlighter execution time #361
• (existing) ZSH_HIGHLIGHT_DIRS_BLACKLIST
• automatically detect "networky" mountpoints and default highlighting of /networkmountpoint/foo to off
• blacklist[] a directory when it times out multiple times in a row
• let typing the command line continue whilst stat()s are happening (may need some kind of "not yet determined" style?)
• don't reparse from square one, at least in simple cases. Reuse the previous invocation's parse and $region_highlight in the current invocation.

I'm sure more ideas can be thought of. Let's see what others have to say :)

[fwsmit]

That the typing of text isn't inhibited by a slow file system. When you are cd'd in a slow directory, all typing seems to be slowed down, making it almost if not entirely impossible to run a command (even if the command doesn't need the file system).

The issue here is that the command-line is highlighted from square one on every keypress. You can probably workaround this right now by using the stty(1) flow control keys (default Ctrl+S and Ctrl+Q) while typing a command-line that involves a slow-to-highlight word: cp /networkfilesys<Ctrl+S>tem/foo /tmp/bar<Ctrl+Q><Enter> would do only O(1) remote stat()s, rather than one per character in the command-line.

So that would involve the user pressing those hotkeys, right? I don't think that's a good solution. I'm striving for zsh-syntax-highlighting to work out of the box.

So, that being said, let's first enumerate all the different ideas here:

* [Cap highlighter execution time #361](https://github.com/zsh-users/zsh-syntax-highlighting/issues/361)

I think the timeout should be something like 30ms, otherwise typing might still feel sluggish.

* blacklist[] a directory when it times out multiple times in a row

This would be good in combination with the timeout. Although, the timeout should still not be big enough to disrupt the user, since it will happen at least once for every directory they are in (unless you immediately blacklist all subdirectories, which would be an option).

* (existing) `ZSH_HIGHLIGHT_DIRS_BLACKLIST`

* automatically detect "networky" mountpoints and default highlighting of /networkmountpoint/foo to off


* let typing the command line continue whilst stat()s are happening (may need some kind of "not yet determined" style?)

* don't reparse from square one, at least in simple cases. Reuse the previous invocation's parse and $region_highlight in the current invocation.

I'm sure more ideas can be thought of. Let's see what others have to say :)

Another option would be to disable syntax highlighting for if a directory exists when it times out multiple times, since that's the part where it times out.

[fwsmit]

It seems like fish is using a background thread: https://github.com/fish-shell/fish-shell/blob/master/src/highlight.cpp. This might be the cleanest solution after all.