Stream: t-lang/pub-macro-rules

Topic: "Private" macros


view this post on Zulip rylev (Feb 18 2021 at 10:15):

I'm working on listing out more "complex" uses of a macros and how they are translated in the the new path based scoping. For the use case of "private" macros (i.e., macros defined inside of other macros), I'm not sure if there will be a name resolution problem. I believe name resolution for items inside of macros happens after expansion, right? So would the following code lead to name resolution errors or not:

macro_rules! foo {
    ($o:expr) => {{
        macro_rules! __helper {
            ($e:expr) => {{
                println!("Expression: {}", stringify!($e));
                $e
            }};
        }

        __helper!($o)
    }};
}
foo!(1);
foo!(2);

At some point during expansion there will be two definitions of __helper. This isn't a problem currently as shadowing is fine in the current textual scope rules, but path based scoping this could lead to name clashes.

view this post on Zulip nikomatsakis (Feb 18 2021 at 10:28):

Yeah, this is the tricky thing. I think we should experiment to see if we can figure out the answer here.

view this post on Zulip nikomatsakis (Feb 18 2021 at 10:28):

@simulacrum raised this point in the meeting, and I think he's right -- unlike the other migrations, this may impact the design of the feature itself

view this post on Zulip nikomatsakis (Feb 18 2021 at 10:29):

I'd be satisfied so long as we have a workaround, personally

view this post on Zulip rylev (Feb 18 2021 at 10:34):

@nikomatsakis it sounds like you're saying there would be a name resolution conflict right?

view this post on Zulip nikomatsakis (Feb 18 2021 at 10:34):

I would expect so, since for ordinary items you cannot shadow like this

view this post on Zulip nikomatsakis (Feb 18 2021 at 10:35):

I have a theory though that we could do a workaround like so:

view this post on Zulip nikomatsakis (Feb 18 2021 at 10:35):

well, your example doesn't seem problematic

view this post on Zulip nikomatsakis (Feb 18 2021 at 10:35):

I think the example requires that foo invokes itself

view this post on Zulip nikomatsakis (Feb 18 2021 at 10:36):

so that there are multiple cases of __helper in scope

view this post on Zulip simulacrum (Feb 18 2021 at 10:36):

Oh, the other point I wanted to raise - I don't actually know if this works today - is if you can "export" lexical macros today from a function scope in some way. If so, then those also need a migration strategy (manual or not).

view this post on Zulip simulacrum (Feb 18 2021 at 10:37):

Name resolution is a fixed point algorithm, by the way, and runs alongside expansion, AFAIK.

view this post on Zulip nikomatsakis (Feb 18 2021 at 10:48):

I'm not sure if that works or not. I'd be surprised if it did.

view this post on Zulip nikomatsakis (Feb 18 2021 at 10:48):

Well, #[macro_export] might work

view this post on Zulip nikomatsakis (Feb 18 2021 at 10:48):

But that doesn't bother me so much, the workaround seems pretty clear--move the macro out from the fn

view this post on Zulip nikomatsakis (Feb 18 2021 at 10:49):

It's a good corner case to identify, though

view this post on Zulip simulacrum (Feb 18 2021 at 10:57):

Yeah, I mostly - as with the point I brought up in meeting - want to make sure our design doesn't have holes that could affect the design. For example, I think it's common for proc macros to use macro rules for hygiene or so? I think this is proc macro hack.

view this post on Zulip simulacrum (Feb 18 2021 at 10:57):

FWIW I'd definitely not be opposed to feature gating a simple implementation if that's possible so we can experiment rather than sort of "theorize"

view this post on Zulip rylev (Feb 18 2021 at 12:00):

nikomatsakis said:

I think the example requires that foo invokes itself

That's a good point - my example invokes foo twice so theoretically there are two versions of __helper created. Does the first call to foo get fully expanded before the second call? Forgive my ignorance of how expansion works but if each macro invocation expands only one layer at a time then there would be a point in my example where there are two __helper macros.

view this post on Zulip rylev (Feb 18 2021 at 12:01):

simulacrum said:

FWIW I'd definitely not be opposed to feature gating a simple implementation if that's possible so we can experiment rather than sort of "theorize"

Yes perhaps that's a good idea. I'll try to prioritize getting something in nightly so we can start playing with it.

view this post on Zulip Daniel Henry-Mantilla (Feb 18 2021 at 13:32):

If you want to experiment with the "future" system, know that you can currently pub(crate) use macro_in_scope; to give a macro "future" namespacing rules.
In the example from @rylev above, there is no issue since the macro expands to a braced block, which acts as a macro boundary (=> {{ … }} rhs of the foo macro; I personally prefer using => ( … ) to delimit macro outputs so that when these are braced it is more visible).

But if we tweak a bit @rylev's foo! not to use => ({ … }) but to use => ( … ) (which would be necessary if we wanted foo to expand to items accessible from outside the macro, such as a new type definition), then, if we did pub(crate) use __helper; after both calls to foo!, there would be an error about ambiguity (due to shadowing).

view this post on Zulip Daniel Henry-Mantilla (Feb 18 2021 at 13:38):

I am realizing that with a proc-macro one can hack a workaround, so I may publish a prototype this weekend. The idea would be to be able to define a scope within which all items would be re-exported.
Basically, if we wanted to have a macro expand to, for instance:

pub(super) struct Foo { some_field: SomeType }

with the proc-macro (let's call it fake_scoped!{ pub(super) struct Foo … }), we would instead get:

mod __some_unique_identifier__ {
    use super::*;

    pub(super::super) struct Foo { pub(super) some_field: SomeType }
}
use __some_unique__identifier::*;

This would allow us to circumvent this problem by doing:

macro_rules! foo {(  ) => (fake_scoped! {
    macro_rules! __helper__ {  }
    
})}

view this post on Zulip Daniel Henry-Mantilla (Feb 18 2021 at 13:40):

But I personally find that an option to opt out of the new system would be the simplest solution:

macro_rules! foo {(  ) => (
    #[legacy_scoping]
    macro_rules! __helper__ {  }
    
)}

view this post on Zulip nikomatsakis (Feb 19 2021 at 16:30):

I'm not opposed to a legacy scoping option

view this post on Zulip nikomatsakis (Feb 19 2021 at 16:30):

I'm also not in love with it :)

view this post on Zulip nikomatsakis (Feb 19 2021 at 16:30):

the "module with use" approach is kind of what I was thinking, but I'm not sure if it really works

view this post on Zulip Daniel Henry-Mantilla (Feb 19 2021 at 17:46):

With a way to ask to be private to that "module with use" it would work: with the proc-macro I have in mind, every privacy annotation would be "upped" one level, at which point #[legacy_scoping] would just be a hint to that proc-macro to ensure __helper__ does not get that treatment and remains pub(self) so as to dodge the glob import.


That being said, we are back to having to resort to __some_unique_identifier__, which is a far more direct solution to the initial problem:

macro_rules! foo {(  ) => (
    with_some_unique_identifier! {( $__name__:identifier ) => (
        macro_rules! $__name__ {  }
        $__name__! {  }
    )}
)}

This looks like a simpler solution to the problem.

But at this point I'd even dare say there is something better to have here: "immediately-called anonymous macro_rules".

Using a hypothetical match! ( <call-site-args> ) { <anon macro rules> } syntax, for instance:

match! ( 42 ) {
    ( $T:ty ) => ( compile_error! { "unreachable!" });
    ( $e:expr ) => (/* … */);
}

- (Aside: if we were to use this kind of syntax, it would be important to keep in mind that this kind of facility could also be one of the best APIs to allow for eager evaluation of macros)

match! concat_idents!(f, o, o) { // <- Note the lack of surrounding parens here.
    ( $foo:ident ) => ( fn $foo() {} );
    ( $($otherwise:tt)* ) => ( compile_error! { "this would be sad" } );
}

view this post on Zulip rylev (Feb 23 2021 at 17:48):

So I've confirmed that this is an issue. If you use a macro to define to scope based macros with the name name, they will collide.

view this post on Zulip rylev (Feb 23 2021 at 17:49):

This is definitely a harder problem to solve than the resursive macros since that could relatively easily be solved with a $self keyword. This problem doesn't seem to have a good solution

view this post on Zulip rylev (Feb 23 2021 at 17:50):

I'm not sure anonymous macro calls are the right way to go (unless I'm actually not understanding them correctly). Ideally we'd like to find something with an existing analogy.


Last updated: Jan 26 2022 at 07:20 UTC