Stream: t-compiler/rust-analyzer

Topic: Question About Architecture


Vikash Balasubramanian (Feb 16 2021 at 03:03, on Zulip):

Hello everyone, I have been recently getting into parsers and I was going through the architecture and documentation of rust analyzer.

It seems like a lot of the moving parts of rust-analyzer are implemented from scratch. May I know what motivated this? Why not use a parser generator framework like LALRPOP (https://crates.io/crates/lalrpop) ? I have been thinking about implementing a language server for Python in Rust, and any tips would be helpful.

matklad (Feb 16 2021 at 07:38, on Zulip):

See https://rust-analyzer.github.io/blog/2020/09/16/challeging-LR-parsing.html

bjorn3 (Feb 16 2021 at 08:39, on Zulip):

Python has a PEG grammar, which is similar to a CFG grammar except that in case of ambiguity the first matching rule applies. This means that it may not be possible to parse Python using the LALR algorithm. Instead you have to use something like Packrat or Pika parsing.

Vikash Balasubramanian (Feb 17 2021 at 01:25, on Zulip):

@bjorn3 That's actually pretty surprising to me. It seems like RustPython (https://github.com/RustPython/RustPython/tree/master/parser) is using LALRPOP. I will ask the author if they had any issues. Obviously RustPython didn't really focus on error resilience, so it may not apply anyways.

Vikash Balasubramanian (Feb 17 2021 at 01:28, on Zulip):

@matklad Thanks that was a really well written blog post. I also did some more reading and the Rowan framework for RA seems to be pretty good. Although I wonder how general it really is. It has been designed with RA in mind. Do you think it might have any limitations when it comes to Python parsing ?

bjorn3 (Feb 17 2021 at 06:34, on Zulip):

According to https://www.python.org/dev/peps/pep-0617/ python used a grammar that is almost LL(1), but not completely. That PEP describes the steps to migrate to a new PEG based grammar.

bjorn3 (Feb 17 2021 at 06:35, on Zulip):

Guido van Rossum has written a series about PEG grammars: https://medium.com/@gvanrossum_83706/peg-parsing-series-de5d41b2ed60

Chetan Khilosiya (Feb 20 2021 at 14:34, on Zulip):

Hello everyone. Good morning. I am new to the rust analyzer project. I would like to contribute in this project. I am looking at the issue https://github.com/rust-analyzer/rust-analyzer/issues/7454. Where can I discuss beginner level stuff about task (which topic)? I can't find the relevant topic.

Florian Diebold (Feb 20 2021 at 14:35, on Zulip):

you can just create a new topic for your question (by entering a new topic name)

Chetan Khilosiya (Feb 20 2021 at 14:38, on Zulip):

ok. Thank you @Florian Diebold

Last update: Jul 24 2021 at 21:15UTC