Stream: t-compiler/wg-rls-2.0

Topic: mbe punct unit


Edwin Cheng (Mar 04 2020 at 03:45, on Zulip):

I just want to confirm what is correct punct token (single / composited) should be used in mbe parsing. Imagine the following program:

macro_rules! m {
    ( -> ) => { "->" };
    ( - ) => { "-" };

    ( $first:tt $($tail:tt)+ ) => {
        m!($first)
    }
}

fn main() {
    let result = m!(-> a b c);
    println!("{}", result);
}

I tested it in the rust-playground , the output is ->. I am quite surprised that it use composited token as tt. How about proc-macro token stream ? Does its use composited punct token as well ?

matklad (Mar 04 2020 at 09:57, on Zulip):

@Edwin Cheng yeah, this is one of the warts of the language. mbe matchers do observe composed tokens.

matklad (Mar 04 2020 at 09:57, on Zulip):

Proc macros operate on tokens which carry jointness information with them, so they can do whatever

Edwin Cheng (Mar 04 2020 at 10:23, on Zulip):

@matklad Do you think we could just handle this case by solely treat $var:tt meta-variable as composited token, without changing the whole mbe parsing infrastructure ?

matklad (Mar 04 2020 at 10:28, on Zulip):

sort of. When we match $var:tt, we should capture several tokens

matklad (Mar 04 2020 at 10:29, on Zulip):

To be honest, I don't remmeber of the top of my hand how our mbe parsing infra works, but i think it should be fixable with relatively little effort

Last update: May 29 2020 at 16:25UTC