I've managed to minimize #61693 to something pretty small, but depends on
proc_macro2 and I've not managed to strip those out yet. Does anyone have any experience with crafting compiler test cases from code like this?
My current progress w/ the repro is linked in this comment.
I tried removing sync/quote etc from the attribute crate
this works but note the super important space
// note: qux: Qux<Qux<Baz> >, // the space ^ here is important
Oh, wow. I wouldn’t have thought of that.
without the space it builds, with it there's your
^^^^^^^^^^^^^ help: use dyn
: dyn #[dom_struct]`
I haven't looked at the custom derive yet but I assume we can use the same strategy: output a static TokenStream
I’ve not looked much into what’s happening in the compiler yet, just focused on getting a test case that works.
Yeah, that’s probably a good idea. Way easier than what I had been trying.
this is what I tried in the first crate but couldn't get it to work, so I built both tokenstreams with proc_macro::quote and syn::quote
and asserted they were equal, and they were not
so this is where the space comes from
Could probably look at
-Z unpretty output.
I tried to get the pretty=expanded but I wasn't to easily get the expanded proc macros token
do you know if it works with proc macros or only macros ?
I’m not sure. I’ve not got all that much experience with either.
ah yes it does work
to get the errors shown on the attributes etc we might indeed need multiple crates tho
That’s fine, can do that in compiletest.
true (I was mostly responding to the comment about unpretty output)
I'll try to simplify the derive if you're not working on it ?
I’ve stepped away from the computer for a bit. Feel free to, I’d appreciate it.
it's a bit tougher in the second case, it's related to the spans I think, they look weird with syn/quote
I was wondering if it was something like that.
it appears we can't really create/mutate spans
here they might be invalid, I'll show you the output I get
but I'm not familiar with these so I wouldn't know
I'm not even sure the problem is related to them, but there's some
#0 bytes(0..0) spans, which seem like an interesting difference
if I modify the way to create the items with syn, juste quote! statically, the error disappears as well
so I'm a bit stumped @davidtwco
Can you index into the input stream (using
.into_iter().nth(..)) and then use that to get the spans?
Nevermind, I was mixing up which spans were which.
I'll try to see how the spans look in the quoted static input just in case
they are identical to what proc_macro generates then
and the error disappears
So, we think syn/quote is making spans
bytes(0..0) and that's what's causing the issue?
Since that's the only difference?
it's a bit hard to tell honestly :)
it's the only difference I was able to see in the time I looked at it
I'll makesure again the output is identical
Thanks for taking a look, I'll see what I can dig up when I get back to a machine.
the expanded output is indeed identical
why isn't the bare trait giving a warning in the other case ......
I really gtg though, I'll update the derive test so that you can see what I'm talking about :)
Thanks, appreciate the help.
no problem :)
a weird one for sure
and that weird space.... :)
I think the bug here is probably that there's a way to get spans that are
bytes(0..0) at all - I can't figure out how I'd construct those.
@Vadim Petrochenkov I'm not familiar at all with how
proc-macro crates are intended to work, it there supposed to be a way to get what are effectively dummy spans in a
I'm also looking into how the quote crate does it
I'm able to reproduce the results you were getting.
I've been having a look at
syn and nothing is jumping out to me from a few GitHub searches.
it seems like an unlucky collision of a bunch of different factors
Probably from a use of
yeah I don't see another way to create them :)
if that's the case then in this context rustc is giving the weird spans to quote/proc_macro2 ?
Maybe? I have no idea how to craft a test for that.
or if it's even the real root cause of the problem
I can get
Span to have a
#0 by calling
But it doesn't change the range.
(that is, the
Literal::subspan might also be useful if we could get a big enough initial
maybe there's a way to get the proc_macro bridge server/client to create the weird range span :/
Next thing would be to step through with a debugger everywhere that
syn gets a
Span and find out where it gets the dummy span?
yeah interesting thought, we're a bit in the dark still
That's assuming that it is some
set_span call in those crates that do it and not some other function that manipulates tokens or token streams.
it might also be Span::call_site/def_site but from inside proc_macro2
the contents sure look like the syntax DUMMY_SP but for proc_macro::Span
Yeah, that is interesting.
Implies that dummy spans can be valid?
maybe ? at the very least known to happen in some cases let's say
Yeah, that’s reasonable.
how hard it is to actually turn DUMMY_SP into a proc_macro::Span is a testament to rustc's crates privacy design, good job :)
I didn't look into the details, but DUMMY_SP in proc macro tokens means that the tokens were obtained through pretty-printing and then re-parsing the input.
This happens as a fallback when the original tokens are unavailable or invalidated in any way (https://github.com/rust-lang/rust/issues/43081).
it took a while but I figured it out @davidtwco
after trying a lot of things including setting up a rustc driver to force the dummy_sp, I saw the light of day
the dummy_sp was in the input along, PR here
depending on how you override the spans the error can be different
if you clobber them all it will have an error like
^ help: use `dyn`: `dyn #`
so I set it up like the original code did with
quoteand it matches the error you expect
help: use `dyn`: `dyn #[dom_struct]`
Thanks a bunch.
Does that still preserve the original warning too on the field?
(it doesn’t really matter but it’d be interesting)
I've pushed your changes to the repro repository and modified the attribute crate so that it is also token-for-token identical to what
syn was outputting.
it's probably not the most minimal repro of the spirit of the bug, as we're making sure the tokens match the original but it's likely small enough to be debugged, and have it as a regression test
Yeah, I think that's the best that we could hope for.
alright cheers David, enjoy the weekend :)
You too, thanks a ton for the help.