Stream: t-compiler/wg-rls-2.0

Topic: Structural Search Replace

matklad (Nov 15 2019 at 20:46, on Zulip):

I've almost nerdsnipped myself with this, but decided to write up an issue with mentoring instructions in the end: :)

csmoe (Nov 16 2019 at 07:55, on Zulip):

@matklad may help here, I'm transforming generated code with it these days.

matklad (Nov 16 2019 at 09:07, on Zulip):

@csmoe the idea is exactly to build rerast-like thin, but using rust-analyzer's types, which I believe I fundamentally more suitable for this kind of things in the limit (ie, when we finishing implementing all the things we yet have to add...)

Felix Kohlgrüber (Nov 20 2019 at 19:28, on Zulip):

How can I parse a String into an expr syntax tree node? And also, should ssr be performed on the concrete or abstract syntax tree?

matklad (Nov 20 2019 at 19:30, on Zulip):

@Felix Kohlgrüber SourceFile::parse in ra_syntax defines parsing of files

matklad (Nov 20 2019 at 19:31, on Zulip):

I think a similar method could be defined for ast::Expr, but you'll need parse_fragment function as an entry point

matklad (Nov 20 2019 at 19:31, on Zulip):

We don't have traditional AST (what is called ast is, in fact, a typed concrete syntax tree)

matklad (Nov 20 2019 at 19:32, on Zulip):

So I think it makes sense to run ssr on concrete syntax tree, with maybe some hacks for more abstract comparison (like, not taking the order of fields into account, etc)

matklad (Nov 20 2019 at 19:32, on Zulip):

One more interesting bit is that, because replace bit has to happen at the CST level, you sort-of need to run the search on the CST as well

Felix Kohlgrüber (Nov 20 2019 at 19:34, on Zulip):

Ok thx, got it!

matklad (Nov 20 2019 at 19:37, on Zulip):

@Felix Kohlgrüber actually, I thik you can also use make::expr_from_string -- it's hacky, but works

matklad (Nov 20 2019 at 19:38, on Zulip):

expr_from_text that is

Felix Kohlgrüber (Nov 20 2019 at 19:40, on Zulip):

ah ok, that should work. Thanks!

Felix Kohlgrüber (Nov 20 2019 at 19:44, on Zulip):

@matklad In the issue description, you wrote that writing a custom parser for the patterns might be better than using regex. Could you explain why you think so?

Felix Kohlgrüber (Nov 20 2019 at 19:44, on Zulip):

@matklad Ok so this is how mentions work. Sry, I'm new to zulip.

matklad (Nov 20 2019 at 19:46, on Zulip):

@Felix Kohlgrüber mainly because regex is a big dependency. We have it in the crate graph already (which is regrettable, I believe at least two of the three uses are completely unnecessary) , but core crates like ra_ide_api do not have this dependency

Felix Kohlgrüber (Nov 20 2019 at 19:48, on Zulip):

@matklad ok, sounds reasonable. I'll hand-roll my own ;-)

mikhail-m1 (Jan 20 2020 at 11:21, on Zulip):

@matklad I have several questions:

  1. You started fromSource { file_id: $file_id:expr, ast: $ast:expr } but later used

    replace $ident:fragment with __search_pattern_ident is file_id: $file_id:expr, should it be $<something>:expr?

  2. Is any way to sanitize temporary names except of using __search_pattern_<ident1>, __search_pattern_<ident2> and etc?

  3. I tried convert it to macros declaration and parse, it works but I get ast::TokenTree, may be there is a way to convert ast::TokenTree or use it during search?
  4. If I need to write code to replace $file_id:expr should it be done just by string functions or there is a better way,?
  5. Do you have any example how to compare trees?
mikhail-m1 (Jan 22 2020 at 11:59, on Zulip):

little bit more details for 3:

  1. convert a pattern to macro_rules! { ({}) => () }
  2. parse
  3. replace $file_id:expr by placeholders
  4. convert back to string
  5. parse again as expt

@matklad what do you think?

matklad (Jan 22 2020 at 12:23, on Zulip):

Don’t have time to really dig into this right now, but I don’t think we
need to parse anything as Macro here. I think just a text-based search for
patterns should be enough for the initial impl, and should be easier to

Last update: Jan 28 2020 at 07:20UTC