Stream: t-compiler/rust-analyzer

Topic: Semantic Token delay


Domenic Quirl (Apr 14 2021 at 21:50, on Zulip):

Hi everyone. I hope it's fine to pop in here without working on r-a (I asked Borger about it and he said it'd be fine, feel free to tell me if it's not). For those who don't know me, I briefly appeared previously in https://rust-lang.zulipchat.com/#narrow/stream/185405-t-compiler.2Frust-analyzer/topic/PSA.3A.20rowan.20might.20be.20a.20severe.20case.20of.20over.20engineering as someone who made @matklad think about rowan even more than he already did.

Recently, I started writing a language server for the thing I also use cstree for, and the other day I ran into https://github.com/microsoft/vscode/issues/89640 : because of using semantic highlighting, and the LSP request for the token taking some small amount of time, there is a noticeable delay until highlighting updates, for example when you comment out a line. This has since started to annoy me tremendously (and probably more than it really should), because

- changes like commenting out a single line are so small and "simple" that it just _feels_ like the update should be instant, and
- the entire file is _syntactically_ highlighted basically on the spot upon opening it from the textMate language grammar.

The delay scales with file length, but I can see it even in small files (r-a does slightly better than I do currently for these sizes, though I'm unsure why exactly). So I'm looking for a way to have some visual update happen more quickly, through either the language server or the extension. The latter side I am criminally unfamiliar with and have just cobbled together enough to make the whole thing work.

Have you guys implemented anything specifically to help with this for r-a, or looked at this previously, or does r-a do something more general to speed up turnaround times on "important" updates? I do realize this is a minor issue in the grand scheme of things and I am kinda obsessing over this a bit, but such obsessing is a strong suite of mine and I'm taking a longer shot to try to convert that into something useful :sweat_smile:

Domenic Quirl (Apr 14 2021 at 21:51, on Zulip):

Again, I don't want to disrupt your work-related Zulipping ('s that a word?), so do tell me if I do

Lukas Wirth (Apr 14 2021 at 22:01, on Zulip):

Asking this on zulip is completely fine.
The same delayed highlighting behavior appears in RA when commenting out a line, at least for me and I don't think that came ever up as an issue somewhere.
The only thing I can think of that might reduce the time a bit is handling delta highlighting via textDocument/semanticTokens/full/delta but as RA also already does that I don't believe much can be gained from that.

Lukas Wirth (Apr 14 2021 at 22:02, on Zulip):

Though I don't have extensive knowledge about the spec or implementations either

Domenic Quirl (Apr 14 2021 at 22:14, on Zulip):

Yeah, I did implement token deltas, but in a way which is close to the VSCode typescript implementation (if I'm not mistaken RA's implementation is also similar). So the delta request will recompute all file tokens anyways, and only save on the tokens sent to the editor.

Domenic Quirl (Apr 14 2021 at 22:17, on Zulip):

So far I've noticed RA to be a bit faster for the smaller files, while I catch up as files sizes increase. But that comparison is in no way fair, since for one RA does a lot more other work, and also I believe you use a sync base + thread pool (?) while I set up with lspower for now, which is async

matklad (Apr 15 2021 at 06:49, on Zulip):

We didn't do anything specifically to improve hightlightting latency, but let me dump general facts I know

matklad (Apr 15 2021 at 06:49, on Zulip):

First, LSP spec is not really suitable for fast and precise coloring.

matklad (Apr 15 2021 at 06:50, on Zulip):

For precise coloring, you need to use type-inference results, and, in general, type inference can be slow, so this part needs to be async in terms of user experience (ie, when opening a file, the users will see more correct syntax highlighting kicking in milliseconds later)

matklad (Apr 15 2021 at 06:52, on Zulip):

What you can real-time guarantees for is lexer-based hightlighting. Lexing is done using finite state machines, and, unlike parsing or type-checking, it's easy to re-start the lexer from a specific point, so you can re-lex incrementally in bounded time.

matklad (Apr 15 2021 at 06:54, on Zulip):

So, the proper API for syntax hightlghting (like the one in IntelliJ obviously) would have two methods, one for synchroneous lexer-based highlighting (so that you can compute the correct colors before the next display flip) and one for asynchronouse type-inference based hightlighting.

matklad (Apr 15 2021 at 06:56, on Zulip):

The other thing which is important for latency is proper management of read/write requests.

matklad (Apr 15 2021 at 06:57, on Zulip):

Let's say user typed foobar. What your server might see in the protocol is

matklad (Apr 15 2021 at 07:05, on Zulip):

in this situation, it's important to make sure that you can react to insert bar immediately, without waiting for hightlighting to finish.

The approach we use for this in rust-analyzer is that we have a manually-coded event loop which accepts the messages. With this loop, we can schedule latency-critical operations (all edits, but some of the read operations as well) onto the loop's thread directly, and offload async operations to the background ("should this OP be synchroneous/block the main loop?" is a useful framing question). See https://github.com/rust-analyzer/rust-analyzer/blob/e131bfc747df1b21ae6ea04eb9c55001e06b7bf0/crates/rust-analyzer/src/main_loop.rs#L494-L515 on_sync and on are the two flavors.

Separately from scheduling, you need "async op can't block a sync one" property. I think there are two solutions here: either you make the state to be a purely functional data structure, so that async and sync can execute on two different versions of state at the same time, or you add cancellation for async operations, so that you can cancel pending jobs.

matklad (Apr 15 2021 at 07:07, on Zulip):

you use a sync base + thread pool (?) while I set up with lspower for now, which is async

I am in general async-sceptic, so I might be biased, but I don't think per se async buys anything for LSP server use-case :)

Domenic Quirl (Apr 15 2021 at 09:18, on Zulip):

Thank you! I very much appreciate the dump!

Domenic Quirl (Apr 15 2021 at 09:39, on Zulip):

matklad said:

So, the proper API for syntax hightlghting (like the one in IntelliJ obviously) would have two methods, one for synchroneous lexer-based highlighting (so that you can compute the correct colors before the next display flip) and one for asynchronouse type-inference based hightlighting.

I am inclined to agree, hence me trying to find a way to react to such changes more quickly with a coarser approximation, while full results are pending. Based on what all of you said here so far, it seems to me like if there is a way to make this work, that way is likely it would involve some custom code in the editor extension. Is there an (or a combination of) API access point(s) that one could use to trigger re-highlighting a part of a document with the textMate grammar, or maybe just remove the semantic highlighting scopes? If I understand correctly they just get put "on top" of the textMate ones, so if they are gone highlighting would fall back to syntactic?

matklad (Apr 15 2021 at 09:41, on Zulip):

https://code.visualstudio.com/api/references/vscode-api#DocumentRangeSemanticTokensProvider

Domenic Quirl (Apr 15 2021 at 09:41, on Zulip):

As I said I'm still getting into the TS side of things. But it appears to be possible to register for additional notifications in the extension for things like DocumentChangeEvents. So you could know those and check if they represent a change for which you want "intermediate" highlighting. But you'd have to be able to trigger that then happening

matklad (Apr 15 2021 at 09:41, on Zulip):

it has didChangeSemanticTokens, so it should be possible to hack something working here

Domenic Quirl (Apr 15 2021 at 09:43, on Zulip):

(your other points are also very helpful, but are something I will deal with later)

matklad (Apr 15 2021 at 09:45, on Zulip):

But, in terms of system engineering, I'd personally would:

Domenic Quirl (Apr 15 2021 at 09:45, on Zulip):

matklad said:

I am in general async-sceptic, so I might be biased, but I don't think per se async buys anything for LSP server use-case :)

Though I will say I am also unsure about this, since a lot of the LSP operations do in some way rely on each other, and you are not that much bound by awaiting something longer like IO. I primarily felt this was very easy to get going

Domenic Quirl (Apr 15 2021 at 09:53, on Zulip):

matklad said:

it has didChangeSemanticTokens, so it should be possible to hack something working here

Hm, so you're thinking something like

Domenic Quirl (Apr 15 2021 at 09:56, on Zulip):

Would the editor not issue a request for a token update anyways because the file is updated? Also does the TS method correspond to semanticTokens/refresh in terms of the LSP spec? Cause then one would probably need to provide a full set of tokens :thinking:

Domenic Quirl (Apr 15 2021 at 09:56, on Zulip):

:shrug: at the issue and shake the fist in the general direction of broken software

this does seem to be at least part of the way to go ^^

Domenic Quirl (Apr 15 2021 at 10:07, on Zulip):

I was hoping there was some https://code.visualstudio.com/api/references/commands that allows to interact with VSC on the topic of highlighting, but it seems you can only ask it to ask the LS about something for you

Jeremy Kolb (Apr 15 2021 at 15:16, on Zulip):

Domenic Quirl said:

Would the editor not issue a request for a token update anyways because the file is updated? Also does the TS method correspond to semanticTokens/refresh in terms of the LSP spec? Cause then one would probably need to provide a full set of tokens :thinking:

semanticTokens/refresh is sent from the server to the client. We do that on a workspace reload if I remember correctly.

Jeremy Kolb (Apr 15 2021 at 15:18, on Zulip):

In terms of the delta we compute the full set of tokens and just send the delta back but it's a pretty naive algorithm that I took from clangd

Last update: Jul 28 2021 at 04:15UTC