Stream: t-compiler/wg-rls-2.0

Topic: Structured Code Generation

theduke (Aug 11 2020 at 10:15, on Zulip):

I've been dissatisfied with the current state of code generation for a while.

The prevailing options are either using syn + quote, or - as RA is doing - basic string concatentation.

Both allow creating malformed output which can only be detected at runtime, and limit generation to a simple single pass, unless you either have your own tailored AST, or deal with manipulating the syn AST, which is very awkward, since it really isn't built for manipulation.

A solution would be a library tailored for code generation. It could either use a custom AST for just that purpose, or provide a convenient wrapper around existing ones, like ra_syntax.

Some prior art: Kode for C++, php-generator.

I'm curious if there is interest in something like this from the rust-analyzer side.

Florian Diebold (Aug 11 2020 at 11:27, on Zulip):

I think we have the beginnings of this in and

Florian Diebold (Aug 11 2020 at 11:31, on Zulip):

there's obviously a lot to be improved there, but I think the general shape (creating and manipulating ast nodes) is what we want

Florian Diebold (Aug 11 2020 at 11:32, on Zulip):

note that we need to be able to incorporate user-written code as-is

Florian Diebold (Aug 11 2020 at 11:33, on Zulip):

I'm pretty sure matklad has a lot more to say about it though, but he's on vacation currently as far as I know

matklad (Aug 12 2020 at 09:40, on Zulip):

Yeah, syntax tree manipulation is a thorny problem, API-wise.

For IDE use-case, I am pretty sure that "creating malformed output" is a feature -- sometimes you need to produce bad output (if parts of it are malformed sytnax nodes from input), and I don't think I've faced a log of bugs which could have been prevented by stronger types in the AST.

For creating & manipulating AST nodes, I indeed like the general shape of and, I think the API is optimal for immutable syntax trees. The impl is suboptimal, in two ways:

matklad (Aug 12 2020 at 09:43, on Zulip):

However, and that is really tantalizing, the most usable API for editing syntax trees is a mutable one. Ie, when you can have reference to several nodes in the same tree, and modifications via one reference are reflected in another reference. This gives a much more intuitive API, as you can do things like:

for child in node.children() {

I still have hope that we can reconcile rust-analyzer's immutable trees (immutability is the right choice for everything else) and mutable editing API, but, so far, I haven't found a way to do this.

matklad (Aug 12 2020 at 09:45, on Zulip):

Separately, for the task of pragramtic code generation, I think quote! + shelling out to rustfmt (what we do to generate sytnax trees themselves) is a near-optimal choice. You could add all kinds of conveniences & type safety there, but you shouldn't as for such tasks "stupid & fast to compile" is a much more important property than "a perfectly cut gem made of types".

Last update: Sep 27 2020 at 14:15UTC