Stream: t-compiler/wg-rls-2.0

Topic: expanding procedural macros


matklad (May 23 2019 at 21:34, on Zulip):

I've just read a master's thesis on implementing procedural macros in IntelliJ Rust, and I think we could reuse this approach (and even code!)

The key idea is to isolate bits that use unstable compiler's API into a separate process, and write a stable IPC shim on top. This not only doges stability problems, but also greatly increases reliability: run away proc macro won't be able to crash the IDE process.

The shim is here: https://github.com/fedochet/rust-proc-macro-expander I believe we can basically just use it?

matklad (May 23 2019 at 21:35, on Zulip):

Also, kudos to https://github.com/fedochet, masters student at https://en.itmo.ru/ who did all the work!

matklad (May 23 2019 at 21:36, on Zulip):

cc @Edwin Cheng

matklad (May 23 2019 at 21:37, on Zulip):

also cc @Igor Matuszewski, the overall architecture pattern, when rustc_private bits are moved into a separate process looks like something we want for both RLSes

Edwin Cheng (May 24 2019 at 02:01, on Zulip):

That's neat !

Edwin Cheng (May 24 2019 at 04:14, on Zulip):

I did dig this problem a litte bit recently, the real hard part of the problem is , there is no way to call a proc-macro directly.
For example, imagine following proc-macro:

#[proc_macro_attribute]
pub fn foo(attr: TokenStream, item: TokenStream) -> TokenStream {
    // ...
}

Instead of the normal usage :

#[mycrate::foo]
fn wrapped_function() {}

We want to call it like this:

fn some_func() {
    let token_stream = mycrate::foo(...);
    // ...
}

However, in the current stable rustc there is no way to call a proc-macro directly.
There are some solutions and idea about it:
1. (UNSTABLE) Same as the link you provided, loading the proc-macro crate dylib manually, and use unstable api (i.e. proc_macro::bridge) to bypass it.
2. Add a compiler flag in rustc to bypass the proc_macro_attribute and change the crate type to rlib.
3. Change rustc proc-macro to use IPC and deal with that IPC manually.
4. (STABLE but complicated) Write a "copy-on-write" virtual file system (FUSE?) and overwrite the given proc-crate Cargo.toml and its source code, to mask all proc-macro related stuffs and compile it as a normal rlib crate.

Edwin Cheng (May 24 2019 at 04:14, on Zulip):

Personally, I want to try to implement 4, and happy to use (but not implement) 2. But for fast prototyping, 1 is okay.

Edwin Cheng (May 24 2019 at 04:26, on Zulip):

It is because seem like proc_macro::bridge are far from stabilized. Although this part of "proc-macro" slim is out of the scope of the main rust-analyzer project and it just use the nightly compiler indirectly, depends on a nightly toolchain is not ideal for me.

matklad (May 24 2019 at 08:40, on Zulip):

Approach 4 sounds interesting! I am afraid though, that it'll have a lot of overhead. For example, it would be hard to batch expansinos of several macros. In a sense, approach 4 already exists as a cargo expand subcommand.

Approach 1 seems fine to me:

Last update: Nov 12 2019 at 15:30UTC