Stream: t-compiler/wg-rls-2.0

Topic: Design meeting: syntax trees


matklad (Feb 18 2019 at 11:37, on Zulip):

The meeting will happen at Wednesday, 20th, 14:00 -- 15:00 UTC, in this Zulip stream.

The goal of the meeting is to nail down the API of syntax trees for rls 2.0/compiler.

Required reading:

rowan/SyntaxNode is pretty close to Swift libsyntax, the two main differences are:

matklad (Feb 20 2019 at 13:05, on Zulip):

@WG-rls2.0 reminder , meeting starts in one hour!

matklad (Feb 20 2019 at 14:02, on Zulip):

@WG-rls2.0 meeting time!

matklad (Feb 20 2019 at 14:02, on Zulip):

Let's see who is here

matklad (Feb 20 2019 at 14:02, on Zulip):

/me

pnkfelix (Feb 20 2019 at 14:02, on Zulip):

/me admits they have not done the homework. :embarrassed:

matklad (Feb 20 2019 at 14:02, on Zulip):

o/

Florian Diebold (Feb 20 2019 at 14:03, on Zulip):

I'm here, might not be 100% present though and I don't know how much I can contribute on this topic ;)

Igor Matuszewski (Feb 20 2019 at 14:03, on Zulip):

:wave:

matklad (Feb 20 2019 at 14:04, on Zulip):

Ok, so let's get started!

mw (Feb 20 2019 at 14:05, on Zulip):

We should also invite @eddyb in case they don't know about this meeting yet

matklad (Feb 20 2019 at 14:06, on Zulip):

Pinged the whole of WG-grammar on discord just in case

pnkfelix (Feb 20 2019 at 14:06, on Zulip):

@matklad , based on my cursory review of Swift lib/Syntax vs rowan, my first question is: Rowan seems to be abstracted over what concrete syntax the client might want

matklad (Feb 20 2019 at 14:07, on Zulip):

Yep!

pnkfelix (Feb 20 2019 at 14:07, on Zulip):

is lib/Syntax similar? Or does it hard-code knowledge of Swift syntax itself in that library?

matklad (Feb 20 2019 at 14:07, on Zulip):

It's even slightly more subtle than that

matklad (Feb 20 2019 at 14:07, on Zulip):

both rowan and swift libsyntax use a single type as the underlying storage for all nodes

matklad (Feb 20 2019 at 14:08, on Zulip):

Specifically, swift uses this "DST" type to store any node: https://github.com/apple/swift/blob/e066d452e8128d9ac5d457d6c6a81d67e850541d/include/swift/Syntax/RawSyntax.h#L221-L223

matklad (Feb 20 2019 at 14:09, on Zulip):

It looks like this in memory:

kind|n_children|child1|child2|child3
matklad (Feb 20 2019 at 14:09, on Zulip):

where kind is an u16 tag for the specific node kind (Class Declaration, sum expression etc)

matklad (Feb 20 2019 at 14:11, on Zulip):

The concrete syntax the client might want is then a new-type wrapper around this node, which adds static info like "the first child is expected to be an identifier".

matklad (Feb 20 2019 at 14:11, on Zulip):

does this make sense?

pnkfelix (Feb 20 2019 at 14:12, on Zulip):

i think so

matklad (Feb 20 2019 at 14:13, on Zulip):

Great!

So, the primary question is "would a similar implementation work everywhere in the compiler?".

I defenitelly like it for IDE, for the following reasons:

matklad (Feb 20 2019 at 14:14, on Zulip):

Note that this impl is not really customizable: if you want to associated additional info with syntax nodes, you'll have to use some sort of a side-table.

Jeremy Kolb (Feb 20 2019 at 14:15, on Zulip):

Does Swift use this approach in their compiler as well or just IDE? I guess why wouldn't it work?

mw (Feb 20 2019 at 14:15, on Zulip):

from the point of incremental compilation you usually want to restrict the amount of data that can be accessed without tracking

matklad (Feb 20 2019 at 14:15, on Zulip):

IIRC, libsytax is a new thing, and they are in the process of transitioning compiler to it.

mw (Feb 20 2019 at 14:16, on Zulip):

i.e. if you get the parent of a node, it should go through a query

nikomatsakis (Feb 20 2019 at 14:16, on Zulip):

One note about comments/whitespace: it feels to me like such things should be...recoverable, but not necessarily something we want 99% of the time.

nikomatsakis (Feb 20 2019 at 14:16, on Zulip):

Note that this impl is not really customizable: if you want to associated additional info with syntax nodes, you'll have to use some sort of a side-table.

But this part feels ok to me

Florian Diebold (Feb 20 2019 at 14:17, on Zulip):

most queries probably don't directly access the syntax, but an intermediate representation generated from it by another query?

matklad (Feb 20 2019 at 14:17, on Zulip):

@mw true, yeah. I expect that perhaps you might not want to use syntax trees directly at all. syntax node remembers absolute offset , and that's bad for incremental

matklad (Feb 20 2019 at 14:17, on Zulip):

so yeah, what @Florian Diebold , there could be an intermediate lowred representation (AST)

mw (Feb 20 2019 at 14:18, on Zulip):

yes, some sort of reduced representation for specific purposes would solve that problem

nikomatsakis (Feb 20 2019 at 14:18, on Zulip):

I'm not entirely convinced. It depends on what the minimum amount of work you want to do is

matklad (Feb 20 2019 at 14:18, on Zulip):

re "recoverability", the biggest drawback of such trees is memory footprint. like, every token like + is a separate allocation.

nikomatsakis (Feb 20 2019 at 14:19, on Zulip):

in particular, if you have your base representation with absolute offsets etc, you will have to recompute the entire tree for each parse

nikomatsakis (Feb 20 2019 at 14:19, on Zulip):

maybe that's ok

pnkfelix (Feb 20 2019 at 14:19, on Zulip):

@nikomatsakis when you say comments+whitespace should be recoverable, are you saying you object to having them always their representation embedded in parse tree? even if e.g. we did not guarantee their presence?

nikomatsakis (Feb 20 2019 at 14:19, on Zulip):

also, memory allocation

mw (Feb 20 2019 at 14:19, on Zulip):

could we have two parsers: one that immediately produces the reduced form and another one that is invoked only when needed?

mw (Feb 20 2019 at 14:20, on Zulip):

the fast one wouldn't need to be error tolerant even? not sure

nikomatsakis (Feb 20 2019 at 14:20, on Zulip):

@pnkfelix I'm not saying I object, but I'm not sure it's a good idea. I think that maybe an initial parse should be something that is relatively incremental friendly and contains the information we want most often, and perhaps there is a follow-on query that gives a more detailed tree when needed (as @mw sort of suggested).

nikomatsakis (Feb 20 2019 at 14:20, on Zulip):

In other words, the right setup might not be to have a "maximal" tree at the "base"

Igor Matuszewski (Feb 20 2019 at 14:20, on Zulip):

^ I like the approach for fast parsing and then lazily geting more details when necessary

Florian Diebold (Feb 20 2019 at 14:21, on Zulip):

the fast one wouldn't need to be error tolerant even? not sure

it does still need to be error tolerant -- we want to e.g. still run type inference on incomplete code

matklad (Feb 20 2019 at 14:21, on Zulip):

I guess that would be possible, but with non-trivial accidental complexity.

mw (Feb 20 2019 at 14:21, on Zulip):

@Florian Diebold yes, that's why I was hesitant about that assumption

matklad (Feb 20 2019 at 14:21, on Zulip):

I wonder how much to we need to optimize syntax trees?

matklad (Feb 20 2019 at 14:22, on Zulip):

I mean, if we through them away before typechecking, they won't affect high water mark memory usage

mw (Feb 20 2019 at 14:22, on Zulip):

@matklad you said that the full syntax tree uses a lot of memory. is it also expensive to generate?

mw (Feb 20 2019 at 14:23, on Zulip):

I agree that it would make things more complicated

matklad (Feb 20 2019 at 14:23, on Zulip):

The last time I've done measurements, I think ra_syntax parser was about 2x slower than current libsyntax, and most slowness was constructing the data structure .

mw (Feb 20 2019 at 14:24, on Zulip):

we could try to generate both parsers from a single grammar definition

mw (Feb 20 2019 at 14:24, on Zulip):

not sure how much manual intervention error correction needs

matklad (Feb 20 2019 at 14:24, on Zulip):

well, I think we still want a single data type for the syntax tree?

matklad (Feb 20 2019 at 14:25, on Zulip):

Or do we? Could we directly produced a lowered representation?

matklad (Feb 20 2019 at 14:26, on Zulip):

For "lowered represenation", I was thinking about something more ECS-style, where there's no real tree, but things like FunctionId, StructId, etc,

mw (Feb 20 2019 at 14:26, on Zulip):

it should be possible to directly produce something as abstract as the compiler's current libsyntax

matklad (Feb 20 2019 at 14:26, on Zulip):

In the ID setup, lazyly lowering function body should be doable (In fact, rust-analzyer does something like this now)

mw (Feb 20 2019 at 14:26, on Zulip):

that should be possible too, I think?

mw (Feb 20 2019 at 14:27, on Zulip):

for the "batch" compiler case we only need the "lowered" version unless there's an error, right?

Igor Matuszewski (Feb 20 2019 at 14:27, on Zulip):

@matklad the "lowered representation" you mention is the equivalent of our current DefId setup that we end up with post-name resolution, right?

matklad (Feb 20 2019 at 14:27, on Zulip):

it should be possible to directly produce something as abstract as the compiler's current libsyntax

Yeah, but than I think you'll end up in situation where you have to duplicate code between "full" and "reduced" syntax trees, if they are different type.

matklad (Feb 20 2019 at 14:28, on Zulip):

that we end up with post-name resolution, right?

I'd say post macro-expansion

Igor Matuszewski (Feb 20 2019 at 14:28, on Zulip):

also about tokens for macros, the lossless format would allow us to easily create those from the CST, right?

Igor Matuszewski (Feb 20 2019 at 14:29, on Zulip):

I'd love to be able to end up with incrementally-stable entity ids after just parsing but not sure how much macro expansion will throw wrench into that

matklad (Feb 20 2019 at 14:29, on Zulip):

@Igor Matuszewski it should be easy to recover tokens themselves. Recovering hygiene info would be less direct, because there wouldn't be hygiene directly in the tree.

Igor Matuszewski (Feb 20 2019 at 14:29, on Zulip):

right

Igor Matuszewski (Feb 20 2019 at 14:30, on Zulip):

(hope I didn't mix multiple concerns there)

nikomatsakis (Feb 20 2019 at 14:30, on Zulip):

Can I step back a second and ask a meta question? I'm feeling a touch confused about the goals for this meeting but also the overall place that these trees fit into the plan. I'm wondering if a good goal for the meeting -- rather than settling on a fixed design -- would be to generate one or two basic options and have people try to elaborate those into more detailed proposals, so that we could then meet again to discuss them in more depth?

nikomatsakis (Feb 20 2019 at 14:31, on Zulip):

Or maybe we'll centralize on one approach, not sure.

nikomatsakis (Feb 20 2019 at 14:31, on Zulip):

I'm feeling a bit confused about all the considerations

pnkfelix (Feb 20 2019 at 14:31, on Zulip):

do we have our constraints listed somewhere?

matklad (Feb 20 2019 at 14:31, on Zulip):

Good question!

mw (Feb 20 2019 at 14:31, on Zulip):

a list of constraints might be a good intermediate goal

nagisa (Feb 20 2019 at 14:32, on Zulip):

/me is not even sure what the exhaustive list of requirements is.

nagisa (Feb 20 2019 at 14:32, on Zulip):

(but that might have been elaborated at some all hands meeting that I’ve forgot about/missed)

matklad (Feb 20 2019 at 14:33, on Zulip):

I would love to get the general idea about design sooner rather than later: if we are sure how trees should look like, we can do fun things like port rustft, etc...

matklad (Feb 20 2019 at 14:33, on Zulip):

for constrains, the list on swift libsyntax looks good

nikomatsakis (Feb 20 2019 at 14:34, on Zulip):

can you reproduce it here or put a direct link?

mw (Feb 20 2019 at 14:34, on Zulip):

this? https://github.com/apple/swift/tree/master/lib/Syntax#design-and-implementation-guidelines

matklad (Feb 20 2019 at 14:34, on Zulip):

https://github.com/apple/swift/tree/master/lib/Syntax#design-and-implementation-guidelines

matklad (Feb 20 2019 at 14:34, on Zulip):

yep

matklad (Feb 20 2019 at 14:35, on Zulip):

the "full-fidelity" bit is defenitlly a must-have for an IDE

pnkfelix (Feb 20 2019 at 14:35, on Zulip):

but what does that mean inthe presence of macros ?

pnkfelix (Feb 20 2019 at 14:35, on Zulip):

is the assumption there you're only talking about printing pre-expansion? We want the parse tree to be able to represent both pre- and post-expanded state

pnkfelix (Feb 20 2019 at 14:36, on Zulip):

(i'm looking at: ""full fidelity" - parsing a source file and printing the syntax tree should result in the same file.")

matklad (Feb 20 2019 at 14:36, on Zulip):

Yeah, that defenitlly should hold for code typed-in by the user

pnkfelix (Feb 20 2019 at 14:36, on Zulip):

but if the main point is that one should always be able to recover something identical to the original, then fine

mw (Feb 20 2019 at 14:36, on Zulip):

in a similar vain: do we need to do macro expansion on the full fidelity tree?

matklad (Feb 20 2019 at 14:37, on Zulip):

That probably means that macros should not modify tree in-place, but rather record that a specific macro expansion expands to a new tree

pnkfelix (Feb 20 2019 at 14:37, on Zulip):

(But the implication would then be that there should be enough information post-expansion to recover the pre-expanded form)

pnkfelix (Feb 20 2019 at 14:37, on Zulip):

or that: that we wouldn't modify tree in place

matklad (Feb 20 2019 at 14:37, on Zulip):

That is, IDE should be able to get both the syntax tree represneting the token tree of macro invocation, and the syntax tree representing the result of invocation

matklad (Feb 20 2019 at 14:39, on Zulip):

Back to requirements list:

I do want a homogeneous traversal API, with parent, first_child, next_sibling methods.

matklad (Feb 20 2019 at 14:40, on Zulip):

In IDE contexts, such API are useful because you typically start at the token at the cursor and need to traverse the tree bottom up to do useful stuff. This is in contrast to usual compiler functionality, where you traverse the tree top down.

nikomatsakis (Feb 20 2019 at 14:41, on Zulip):

I think being able to walk a tree of syntax without having to "know" the types deeply is a good constraint

nikomatsakis (Feb 20 2019 at 14:41, on Zulip):

It's not entirely clear to me whether the tree should be a data structure you get back from a single query, or whether it should be something you explore via queries

Igor Matuszewski (Feb 20 2019 at 14:41, on Zulip):

So basically a parent pointer, which in this case has to be of a universal dynamic node type?

nikomatsakis (Feb 20 2019 at 14:41, on Zulip):

I think this @matklad is sort of what you meant by "ECS-like" earlier?

matklad (Feb 20 2019 at 14:42, on Zulip):

@nikomatsakis yeah

nikomatsakis (Feb 20 2019 at 14:42, on Zulip):

e.g., if you had some kind of "syntax node id", you might be able to ask for things like "give me the parent of this id"

nikomatsakis (Feb 20 2019 at 14:43, on Zulip):

@matklad one thing I personally am not very clear on is what rust-analysis is currently using and what problems you see with it

nikomatsakis (Feb 20 2019 at 14:43, on Zulip):

part of this is that I've been too busy to do my homework

Igor Matuszewski (Feb 20 2019 at 14:43, on Zulip):

This assumes salsa-style incremental parsing, right?

Igor Matuszewski (Feb 20 2019 at 14:43, on Zulip):

If we use absolute offsets for each node/id, how do we guarantee stability across multiple reparses (so that we don't thrash the entire incremental cache if ids change)?

nikomatsakis (Feb 20 2019 at 14:44, on Zulip):

not necessarily, I mean the way I would probably handle this in salsa is to have some initial function that does the parse, producing a result, but expect most users to interact with it through queries that pull out bits of it

matklad (Feb 20 2019 at 14:44, on Zulip):

@nikomatsakis ra does the same thing as Swift's new syntax library (which is the same thing as Roslyn's desing)

mw (Feb 20 2019 at 14:44, on Zulip):

if we want things to be incremental it kind of has to go through queries, right?

matklad (Feb 20 2019 at 14:44, on Zulip):

well, the implementation detail differ, but the general structure is the same.

matklad (Feb 20 2019 at 14:44, on Zulip):

That is, I do not use queries for parsing.

nikomatsakis (Feb 20 2019 at 14:44, on Zulip):

that general structure being:

nikomatsakis (Feb 20 2019 at 14:45, on Zulip):

?

matklad (Feb 20 2019 at 14:45, on Zulip):

Yeah

Igor Matuszewski (Feb 20 2019 at 14:45, on Zulip):

so a separate pass that can be incremental on its own and produces a stable-referencable(?) IDs/nodes?

nikomatsakis (Feb 20 2019 at 14:45, on Zulip):

It seems like the macro expansion is also more at the AST level, right?

nikomatsakis (Feb 20 2019 at 14:46, on Zulip):

By this I mean: you want to be able to get an AST with the results of macro expansion eventually. But you don't necessarily expect to get a concrete syntax tree with said results.

nikomatsakis (Feb 20 2019 at 14:46, on Zulip):

Of course, you'd like to be able to serialize it back out perhaps for debugging, refactoring, etc (I can imagine a "expand macro" feature in an IDE, for example)

matklad (Feb 20 2019 at 14:46, on Zulip):

@nikomatsakis sort-of: macro expansion works on tokens, which are close to CST than to AST

Igor Matuszewski (Feb 20 2019 at 14:46, on Zulip):

The output of macro expansion needs to be parsable, tho

nagisa (Feb 20 2019 at 14:46, on Zulip):

It would make sense for macro expansion to occur as part of lowering into the AST

matklad (Feb 20 2019 at 14:47, on Zulip):

that is, I imagine AST does not really cares about trailing commas, but macros do

Igor Matuszewski (Feb 20 2019 at 14:47, on Zulip):

I know in some cases today the output is not, so that's something to have in mind

nagisa (Feb 20 2019 at 14:47, on Zulip):

so that you have the whatever libsyntax provides initially as a holistic representation of source and AST as the macro-expanded form.

nikomatsakis (Feb 20 2019 at 14:48, on Zulip):

Well, @matklad, I imagine that AST notes may be "unexpanded macro", which contains tokens, but once they are expanded, they are parsed. And moreover in order to even do macro expansion, we need to do name resolution, which presumably wants to be based on the AST, right?

mw (Feb 20 2019 at 14:49, on Zulip):

I'm still not clear on the interaction between macro expansion an name resolution

nagisa (Feb 20 2019 at 14:50, on Zulip):

@mw macro expansion can introduce new names into the scope which then influence expansion of the macros elsewhere(?)

mw (Feb 20 2019 at 14:50, on Zulip):

does it need the full name resolution logic or some rather reduced subset?

nikomatsakis (Feb 20 2019 at 14:50, on Zulip):

@mw I'm not sure exactly the sense of your question, but basic idea is that:

to expand foo::bar!, you have to resolve the path foo::bar

nikomatsakis (Feb 20 2019 at 14:50, on Zulip):

And, of course ,that may generate new names that affect later paths.

mw (Feb 20 2019 at 14:50, on Zulip):

are macros hygienic?

matklad (Feb 20 2019 at 14:50, on Zulip):

right, but I don't think we should directly produce AST from the result of macro expansion. I imagine the flow where macro expands to a token-tree, which is then passed to a parser that produces a "full-fidelity" tree (with dummy whitespace) whihc is then feeded back into AST-lowering procedure

nagisa (Feb 20 2019 at 14:51, on Zulip):

from the technical standpoint name resolution can be done as part of the same lowering/expansion procedure. We would need multiple passes, but that is something we need today anyway.

nikomatsakis (Feb 20 2019 at 14:51, on Zulip):

are macros hygienic?

Today, the answer is "sort of" -- not at the item level, in particular. But maybe worth asking why you're asking =)

nikomatsakis (Feb 20 2019 at 14:52, on Zulip):

Maybe the way to think of name/lowering/expansion is as an atomic unit for now -- it's hard enough for us to wrap our head around things

pnkfelix (Feb 20 2019 at 14:52, on Zulip):

really, atomic unit?

nikomatsakis (Feb 20 2019 at 14:52, on Zulip):

and once we have some idea of that, think about how to make it incrementally re-execute?

pnkfelix (Feb 20 2019 at 14:52, on Zulip):

I was about to say

pnkfelix (Feb 20 2019 at 14:52, on Zulip):

that in terms of IDE design

pnkfelix (Feb 20 2019 at 14:52, on Zulip):

we may want to build in the assumption that the UX is going to interact with a tree that is not yet fully expanded

mw (Feb 20 2019 at 14:52, on Zulip):

are macros hygienic?

Today, the answer is "sort of" -- not at the item level, in particular. But maybe worth asking why you're asking =)

does name resolution of macro names have to take hygiene into account or can it be something really simple

mw (Feb 20 2019 at 14:52, on Zulip):

but no need to go into this in detail now..

nikomatsakis (Feb 20 2019 at 14:53, on Zulip):

yeah, it seems like we're getting a bit astray somehow.

nikomatsakis (Feb 20 2019 at 14:53, on Zulip):

I guess my feeling @matklad is that if Rosyln / Swift are taking an approach, we'd do well to follow suit -- or at least have strong reasons to deviate.

nikomatsakis (Feb 20 2019 at 14:53, on Zulip):

but I'm not 100% sure what that means :)

matklad (Feb 20 2019 at 14:54, on Zulip):

Yeah, I feel the same, that's why I kinda started the meeting in the "explore a particular solution" mood :)

nikomatsakis (Feb 20 2019 at 14:54, on Zulip):

heh :)

Igor Matuszewski (Feb 20 2019 at 14:54, on Zulip):

Does Swift have token-based macros as well?

mw (Feb 20 2019 at 14:54, on Zulip):

I remember @Jonathan Turner mentioning that for Typescript (and maybe C#) macros were not supported in order to make IDE support doable

mw (Feb 20 2019 at 14:54, on Zulip):

so doing somthing becuase roslyn does it might not be a good idea because they have different constraints

pnkfelix (Feb 20 2019 at 14:55, on Zulip):

(I suppose an ongoing macro expansion could perhaps be lumped in with user edits ... as in, whatever the code IDE+parser-library does to deal with user edits, it might also handle the changes injected by macro expansion in the same way ... and therefore we should not need to design for macros up front....)

matklad (Feb 20 2019 at 14:55, on Zulip):

Yeah, "can we macro expand to roslyn tree" is a core question whihc might kill the design

Igor Matuszewski (Feb 20 2019 at 14:55, on Zulip):

Ideally I'd like us to support everything that rustc does - I know that it was very annoying not being able to accurately find definitions introduced by C-style macros in C++

nikomatsakis (Feb 20 2019 at 14:55, on Zulip):

So, stepping back, it feels to me like we should be doing more reviewing of what you've built in rust-analyzer @matklad. I know you producd that video and other content. I wonder if it would be useful to schedule more "interactive" discussion of this kind?

nikomatsakis (Feb 20 2019 at 14:56, on Zulip):

Basically I think we don't share quite enough context at the moment

matklad (Feb 20 2019 at 14:56, on Zulip):

Doing the "homework" above would be prerequisite :D

nikomatsakis (Feb 20 2019 at 14:56, on Zulip):

Or maybe it's just me :)

nikomatsakis (Feb 20 2019 at 14:56, on Zulip):

Fair :)

nikomatsakis (Feb 20 2019 at 14:57, on Zulip):

Maybe we need to form some homework study groups...

nikomatsakis (Feb 20 2019 at 14:57, on Zulip):

...also, I can just step back, as I'm pretty busy right now

matklad (Feb 20 2019 at 14:57, on Zulip):

you can pair up with @pnkfelix :)

matklad (Feb 20 2019 at 14:58, on Zulip):

So looks like it's time to wrap-up the meeting!

nikomatsakis (Feb 20 2019 at 14:58, on Zulip):

What do you think are good next steps?

matklad (Feb 20 2019 at 14:59, on Zulip):

1) to write down a more specific list of constaints for design

matklad (Feb 20 2019 at 15:00, on Zulip):

2) study rosly approach in more detail to figure out if it is a good fit. Specifically, macro & hygiene might be tricky

matklad (Feb 20 2019 at 15:00, on Zulip):

3) maaaby explore alternative set ups when the tree itself is salsa-based? I kinda belive in rosly trees though

nikomatsakis (Feb 20 2019 at 15:00, on Zulip):

So I really like the idea of "pairing up" in some sense

nikomatsakis (Feb 20 2019 at 15:01, on Zulip):

I feel like I would like to delve more deeply but I'm trying to think about what will make it actually happen

nikomatsakis (Feb 20 2019 at 15:01, on Zulip):

certainly scheduling some time helps :)

nikomatsakis (Feb 20 2019 at 15:01, on Zulip):

I guess I'd be game to e.g. schedule an hour to sit down with someone (or a few people) and try to walk through what happens when (say) something is parsed in rust-analyzer?

nikomatsakis (Feb 20 2019 at 15:02, on Zulip):

I'm mostly thinking about the "homework" phenomenon and ways to help it get done

mw (Feb 20 2019 at 15:02, on Zulip):

fwiw, I have very little time to actually spend on this. at least in the first half of the year :(

nikomatsakis (Feb 20 2019 at 15:02, on Zulip):

3) maaaby explore alternative set ups when the tree itself is salsa-based? I kinda belive in rosly trees though

I feel like this is maybe not important -- in particular, I think that we'll need some "root parse" action

matklad (Feb 20 2019 at 15:02, on Zulip):

I can totally do this! But it's better be "a few people", I do want to improve bus-factor here

pnkfelix (Feb 20 2019 at 15:03, on Zulip):

I plan to spend time on this

davidtwco (Feb 20 2019 at 15:03, on Zulip):

I've got time I could sink into this wg if there's a desire for some more regular hands.

davidtwco (Feb 20 2019 at 15:03, on Zulip):

I've not followed discussions closely so far but I can catch up.

matklad (Feb 20 2019 at 15:05, on Zulip):

So I schedule a sync video meeting for me, @nikomatsakis, @pnkfelix and @davidtwco with the goal of diving in into the current impl

Igor Matuszewski (Feb 20 2019 at 15:05, on Zulip):

I'd also love to participate, if you don't mind

nikomatsakis (Feb 20 2019 at 15:05, on Zulip):

Seems good -- I don't think I'll have much time for writing code just now so I'd put myself as lower priority, but I'm definitely game to listen in and kibbitz

matklad (Feb 20 2019 at 15:05, on Zulip):

anyone else is welcome to join of course!

Igor Matuszewski (Feb 20 2019 at 15:05, on Zulip):

Wasn't sure what we were voting for :)

Igor Matuszewski (Feb 20 2019 at 15:05, on Zulip):

Great!

matklad (Feb 20 2019 at 15:05, on Zulip):

I am ready to write the code! I just need to know that to write :D

nikomatsakis (Feb 20 2019 at 15:06, on Zulip):

I'll also try to schedule time to actually watch after the fact otherwise -- I know @matklad you also made that other video, which I just realized I should prob just put time on my calendar for (I found it hard to find a slot otherwise)

Igor Matuszewski (Feb 20 2019 at 15:06, on Zulip):

Yeah, I think fleshing out the design and current problems with our approach is the trickiest part of it all :(

matklad (Feb 20 2019 at 15:07, on Zulip):

Ok, I think the meeting is officially over than!

Igor Matuszewski (Feb 20 2019 at 15:07, on Zulip):

Gotta run, off to do more homework on the topic

Igor Matuszewski (Feb 20 2019 at 15:07, on Zulip):

Thanks!

pnkfelix (Feb 20 2019 at 15:07, on Zulip):

thanks for setting this up @matklad and putting up with those of us who are playing catch up

mw (Feb 20 2019 at 15:07, on Zulip):

:wave: Thanks!

matklad (Feb 20 2019 at 16:33, on Zulip):

@WG-rls2.0 stream/poll for rust-analyzer syntax/parsing code walkthrough: https://rust-lang.zulipchat.com/#narrow/stream/185405-t-compiler.2Fwg-rls-2.2E0/topic/rust-analyzer.20syntax.20trees.20deep.20dive

matklad (Feb 20 2019 at 17:07, on Zulip):

Notes: https://github.com/rust-lang/compiler-team/pull/6

matklad (Feb 20 2019 at 17:44, on Zulip):

Filed rust-analyzer#862 with list of the requirements. It's called "Finalize" design of syntax trees and I hope to close it "soon-ish".

With syntax trees in place, we can do some bold experiments, like porting rustfmt to them (which would fix an annoyng "can't format if there are syntax errors" issue) or producing rustc AST out of them.

Last update: Nov 19 2019 at 17:35UTC