Bug: Highlighting synchronization with shown text

[AntonPieper]

If I remove N characters and add N characters again without changing the token type within the update delay, the highlighting was removed due to how spans work (but does not reappear). This happens, because highlighting does not happen immediately but only after a period of inactivity and the diff algorithm is based on current tokens and last highlighted tokens. A naive/simple fix to this issue is to track the first and last edited character and then always replace that range with the new highlight tokens. Another way of fixing it would be to immediately apply the highlighting as soon as the tokenization finishes.

[markusfisch]

Hm, I somehow can't really reproduce this 🤔

Could you please give an example like removing and adding "oor" from "gl_FragCoord" or something like that?

[AntonPieper]

Of course, I can give an example:

It should be:

• vec4 -> v -> vec4

But what happens:

• vec4 -> v -> v ec4

This happens at the boundaries of tokens, because they are fixed to words/characters and not rows/columns.

Another extreme example:

(Works best with highest update delay)

Select All -> Copy -> Delete all -> Paste