Stream: t-compiler

Topic: edition hygiene


alercah (Jul 31 2018 at 06:48, on Zulip):

@Vadim Petrochenkov \o

alercah (Jul 31 2018 at 06:52, on Zulip):

It would be nice for me to understand about hygiene in general and edition hygiene in particular. I'm especially a bit confused by your comment that edition hygiene isn't part of the language.

Vadim Petrochenkov (Jul 31 2018 at 08:59, on Zulip):

The general idea is pretty simple - each token is produced by some specific macro, each macro is defined by some crate, that crate is built with some --edition option and value of that option is kept in crate metadata.
So, ultimately, we know edition for each token.

Vadim Petrochenkov (Jul 31 2018 at 09:02, on Zulip):

When we do something with that token (parse it, or perform some semantic checks later), we can extract its edition (from its span, all hygiene data is kept through spans) and act differently depending on edition value.

Vadim Petrochenkov (Jul 31 2018 at 09:13, on Zulip):

I'm especially a bit confused by your comment that edition hygiene isn't part of the language

I mean, C++17 is a language, C++11 is a language, Rust 2015 is a language, Rust 2018 is a language, but any attempts to build one library in one language, another library in another language and combine them together is already in the tooling territory.

alercah (Jul 31 2018 at 16:09, on Zulip):

Right, ok, that makes sense. So that is used for things like ensuring that if I pass a new keyword from a previous edition into a new one, it stays a raw identifier and doesn't get reinterpreted as a keyword, right?

alercah (Jul 31 2018 at 16:11, on Zulip):

And I saw the issue about the complexity of handling it for expressions, etc.

alercah (Jul 31 2018 at 16:12, on Zulip):

The one where you can have an incomplete expression returned from a macro really got me. Is it possible that that's an issue we can/should punt on for now, and just let us decide hygiene for other things as we introduce features that require it?

alercah (Jul 31 2018 at 16:13, on Zulip):

like saying "ok, so edition hygiene affects identifiers for name lookup and keyword purposes, but we'll leave everything else unspecified for now"? Is there any other cases where hygiene would come up?

alercah (Jul 31 2018 at 16:20, on Zulip):

err, edition hygiene

alercah (Jul 31 2018 at 16:22, on Zulip):

I know that for hygiene purposes, we have at the very least to deal with name lookup and visibility. I don't fully understand all the options for controlling hygiene, though.

alercah (Jul 31 2018 at 16:26, on Zulip):

I know we have $crate which forces a name to be looked up locally, local_inner_macros which does that for all macros in a macro definition, right? And call_site is the only stable option for proc macros, but we want to let you have definition-site hygiene if you opt to? Can you declare items in definition-site hygiene? Is the behaviour of quote (and the crates with similar functionality) with respect to hygiene understood/defined? Does TokenStream::parse() always use call-site?

Vadim Petrochenkov (Jul 31 2018 at 20:26, on Zulip):

So that is used for things like ensuring that if I pass a new keyword from a previous edition into a new one, it stays a raw identifier and doesn't get reinterpreted as a keyword, right?

Right.
(For keywords this is simple since they are single tokens.)

Is it possible that that's an issue we can/should punt on for now, and just let us decide hygiene for other things as we introduce features that require it?

I think we can do that in practice.
The absolute path issue is probably the only issue here that's urgent or important, since the module reform both 1) focuses everyone's attention on path starts, and 2) makes incompatible changes in that area requiring compatibility between edition.

I know we have $crate which forces a name to be looked up locally, local_inner_macros which does that for all macros in a macro definition, right?

Right.
(Locally means in the root of the current crate, where "current" is the crate that defined the macro... in some sense, see examples with nested macro definitions in https://github.com/rust-lang/rust/pull/51762/commits/b69d51162b5391119e86d1c6e884aa09292a7806).

And call_site is the only stable option for proc macros, but we want to let you have definition-site hygiene if you opt to?

Yes.

Can you declare items in definition-site hygiene?

Yes, they will be local to the macro and effectively inaccessible from the outside, perfectly suits for some internal helper stuff.

Is the behaviour of quote (and the crates with similar functionality) with respect to hygiene understood/defined?

Builtin quote macro generates tokens with def-site hygiene.
Third-party crates working on stable generate tokens with call-site hygiene, since only call-site hygiene is stable.
So its behavior is well-defined in terms of these two hygiene kinds.

Does TokenStream::parse() always use call-site?

Yes.

alercah (Aug 01 2018 at 00:02, on Zulip):

Thanks!

Bit more: in paths or field/method access, how is visibility handled? Is it always by simply looking at the first element in the path/access expression and seeing where it is? How does this relate to a method of a trait? e.g. in a.b(), where does b need to be in scope? at the site of a or the site of b?

If you can elaborate as much as possible here, so that I can someday write this into the reference, I would appreciate that. :)

alercah (Aug 01 2018 at 00:37, on Zulip):

There was a discussion in Discord that led to an interesting case. There is currently an RFC proposing private impls of foreign types (for traits at the moment, but several people have gone "ah but what if we could expand this to inherent impls too?"). If that happened, then what would the following do?

impl String {
  fn len(&self) -> usize { 0 }
}

fn unhygienic_maybe() -> usize {
    let s = "string".to_string();
    imported_macro!(s) // imported_macro! expands to "s.len()", with def-site hygiene on 'len'
}

Would this call the normal String::len or the one declared locally?

(Side note that I just want to get down: perhaps quote should have a way to explicitly invoke call-site hygiene, e.g. by writing #a?)

Vadim Petrochenkov (Aug 01 2018 at 10:01, on Zulip):

in paths or field/method access, how is visibility handled?

Visibility as in privacy, or in the hygiene meaning? (I assume the second.)

Is it always by simply looking at the first element in the path/access expression and seeing where it is?

Not sure what this means exactly, but it's close to what happens.

e.g. in a.b(), where does b need to be in scope? at the site of a or the site of b?

The site of b.
Name resolution hygiene is identifier-based, we see and identifier b, we resolve it from it from its def-site, a is not relevant here (well, it may be relevant for collecting candidates, but not for finding the correct b among them).

The best intuition about def-site hygiene is probably replacing the macro with and inline function and then resolving names from that function as you would usually do that without any macros.

Vadim Petrochenkov (Aug 01 2018 at 10:07, on Zulip):

Would this call the normal String::len or the one declared locally?

Let's assume imported_macro is a function and not a macro, then what len would be in scope in it? Certainly not the locally defined one. Hygienic macro behaves in the same way.

perhaps quote should have a way to explicitly invoke call-site hygiene, e.g. by writing #a?

Yes, perhaps, similarly to https://github.com/rust-lang/rfcs/pull/2498. Hygienic macros do need some kind of opt-out to do anything practical.

nikomatsakis (Aug 06 2018 at 14:25, on Zulip):

@Vadim Petrochenkov we still haven't made any kind of "formal decision" about edition hygiene, and the various models, right? I still lean towards the "take edition from some significant token variant", in part because it seems like something we can readily explain and document, but it seems like we need someone to take charge there and try to write out all the rules and get feedback on them.

Last update: Nov 22 2019 at 05:20UTC