Stream: t-compiler/wg-rls-2.0

Topic: rust-analyzer#1213


matklad (Apr 28 2019 at 13:47, on Zulip):

@Edwin Cheng I've looked more at 1213 and I must say I don't entirely like the bifurcation behind compound and atomic tokens

matklad (Apr 28 2019 at 13:47, on Zulip):

I wonder if we, with a more thorough refactoring, can make a better API?

matklad (Apr 28 2019 at 13:48, on Zulip):

let me scketch what I have in mind...

Edwin Cheng (Apr 28 2019 at 13:48, on Zulip):

What we do it is in the Parser level

matklad (Apr 28 2019 at 13:50, on Zulip):
    #[inline(always)] // to make sure compund case is compiled away when `kind` is compile time constant
    pub(crate) fn at(&self, kind: SyntaxKind) -> bool {
        match kind {
            DOTDOT => self.at2(DOT, DOT),
            DOTDOTDOT => ...
            _ => self.current() == kind
        }
    }

    // Private impl details
    fn at2(&self, kind1: SyntaxKind, kind2: SyntaxKind) -> bool {
        self.current2() == Some((kind1, kind2))
    }
matklad (Apr 28 2019 at 13:51, on Zulip):

Could we do something like this?

matklad (Apr 28 2019 at 13:51, on Zulip):

That is, the grammar impl will just ask if at(DOTDOT), without thinking too much if it is a compound token

matklad (Apr 28 2019 at 13:52, on Zulip):

And the same trick could be applied to at_ts: we include multi-char tokens in tokensets, and internally we run special logic for multibyte bits of ts

Edwin Cheng (Apr 28 2019 at 13:53, on Zulip):

Yes of course, but it is not so efficient if we concern it, as we have to check it every time. If it is not an concern, we can do it too. But we have to do it in all at, nth, eat methods

matklad (Apr 28 2019 at 13:53, on Zulip):

I think it could be just as efficient

matklad (Apr 28 2019 at 13:53, on Zulip):

the trick is that kind is almost always a compile-time constant

matklad (Apr 28 2019 at 13:54, on Zulip):

So, if llvm inlines stuff, it will compile away the match kind branch

matklad (Apr 28 2019 at 13:55, on Zulip):

The only place where abstraction will leak are match p.current() branches in parser

matklad (Apr 28 2019 at 13:56, on Zulip):

instead of

match p.current() {
    DOTDOT =>
}

You'll have to write

match p.current() {
    DOT if p.at(DOTDOT) =>
}

But that presumably is ok?

Edwin Cheng (Apr 28 2019 at 13:57, on Zulip):

That's okay.

Edwin Cheng (Apr 28 2019 at 13:58, on Zulip):

Let me try your method first, and let see if there is any edge case :)

matklad (Apr 28 2019 at 13:59, on Zulip):

Yeah, there could be edge cases especially around lookahead, but maybe lookahead can be restricted to atomic tokens?

matklad (Apr 28 2019 at 14:00, on Zulip):

To check performance, one can use cargo run --release -p ra_cli -- parse --no-dump < some-big-file.rs. A good candidate for some-big.file.rs is te parser from rust-lang/rust repo

Edwin Cheng (Apr 28 2019 at 16:02, on Zulip):

but maybe lookahead can be restricted to atomic tokens

It seem to be not work, but i did manage to handle it in Parser::nth , I found that we already have the dollar handling code in it, so ... maybe we should not be too worry about performance right now :rolling_on_the_floor_laughing:

Last update: Nov 19 2019 at 17:40UTC