Stream: t-compiler

Topic: #57415 refactoring doctest compilation

QuietMisdreavus (Jan 07 2019 at 21:32, on Zulip):

i'm investigating a possible refactoring of doctest compilation, and i'm wondering: is there some easy hook for a compiler driver to modify a crate after macro expansion, then continue with the rest of compilation as normal, without having to prep and call the phases individually?

QuietMisdreavus (Jan 07 2019 at 21:33, on Zulip):

in compile_input i see several locations that call a controller_entry_point macro but i'm not sure what that's for

QuietMisdreavus (Jan 07 2019 at 21:37, on Zulip):

ooh, now i see it, looks like there are some callbacks on the CompileController, i'll try that

QuietMisdreavus (Jan 07 2019 at 21:42, on Zulip):

oh wait, i can't modify the expanded_crate because it's only a shared reference

QuietMisdreavus (Jan 07 2019 at 22:02, on Zulip):

looking at this more, it doesn't seem like the compile controller is meant for what i wanted to use it for

QuietMisdreavus (Jan 07 2019 at 22:02, on Zulip):

seems more for introspection than in-process modification

QuietMisdreavus (Jan 07 2019 at 22:05, on Zulip):

looks like you can modify the crate before expansion, but not after? i wonder if i can just change the phases or add a new one >_>

QuietMisdreavus (Jan 07 2019 at 22:58, on Zulip):

hmm, changing the expanded_crate to be an owned Crate instead of a borrowed one will affect more than just the after_expansion pass, since other passes also use that field :/

QuietMisdreavus (Jan 07 2019 at 23:04, on Zulip):

and because of the way the ownership works, it would require the crate to be cloned a couple times, if i understand it right

QuietMisdreavus (Jan 07 2019 at 23:05, on Zulip):

is it feasible to turn it from an Option<&Crate> to an Option<&mut Crate>? i'd have to move where after_expansion callbacks are called, but otherwise i don't think that will affect existing callbacks any

QuietMisdreavus (Jan 07 2019 at 23:05, on Zulip):

unless i'm missing some kind of variance implication, as usual

oli (Jan 08 2019 at 09:27, on Zulip):

Cloning the crate multiple times seems like it would severely regress performance. Maybe a custom_expansion callback would be better?

QuietMisdreavus (Jan 08 2019 at 14:00, on Zulip):

@Oli i'm not familiar with custom_expansion, where's that?

oli (Jan 08 2019 at 14:00, on Zulip):

It doesn't exist. I meant creating a new one

QuietMisdreavus (Jan 08 2019 at 14:08, on Zulip):

any existing work i can base off of? i've added stuff to syntax::ext::expand::InvocationCollector before, but i dunno if that's still the right place to put something, especially if it's going to be conditionally applied

oli (Jan 08 2019 at 14:11, on Zulip):

I would have assumed that one would duplicate the after_expansion entry (and change to &mut) and then go backwards from all call sites to after_expansion to figure out a good call site for custom_expansion.
Why would it be conditional? The default impl for the rustc driver could simply be an empty function.

QuietMisdreavus (Jan 08 2019 at 14:12, on Zulip):

oh, i was starting to think of something totally different from a compiler driver

QuietMisdreavus (Jan 08 2019 at 14:13, on Zulip):

adding the new hook would work though

oli (Jan 08 2019 at 14:13, on Zulip):

something like a full-crate proc macro?

QuietMisdreavus (Jan 08 2019 at 14:13, on Zulip):

i was starting to think of directly editing the macro expansion process itself, like what i did for #[doc(include)]

oli (Jan 08 2019 at 14:14, on Zulip):

I think there were ideas like that for custom test drivers

oli (Jan 08 2019 at 14:14, on Zulip):

hmm... can't you just do #![my_custom_attribute_macro] in the crate root for the same effect?

oli (Jan 08 2019 at 14:14, on Zulip):

or are attribute macros run from the outside first?

QuietMisdreavus (Jan 08 2019 at 14:15, on Zulip):

/me stares into middle-distance at the thought of #![doctest]

QuietMisdreavus (Jan 08 2019 at 14:16, on Zulip):

that may be brilliant, but i'm wondering if we're talking past each other again

QuietMisdreavus (Jan 08 2019 at 14:17, on Zulip):

(i gotta run, but i'll be back in ~20 minutes)

oli (Jan 08 2019 at 14:17, on Zulip):

I'm around for a few hours :slight_smile: just ping me

QuietMisdreavus (Jan 08 2019 at 14:58, on Zulip):

okay so the thought of making a proc-macro that does the doctest massaging is just too good

QuietMisdreavus (Jan 08 2019 at 14:58, on Zulip):

but i'm concerned about making sure it runs after regular macro expansion >_>

QuietMisdreavus (Jan 08 2019 at 14:58, on Zulip):

to make sure that any other macros that create fn main have already had the chance to run

QuietMisdreavus (Jan 08 2019 at 14:59, on Zulip):

(i'm thinking of things like the first release of quicli which gave a main! macro that did just that)

QuietMisdreavus (Jan 08 2019 at 16:53, on Zulip):

oof, i need to rethink my approach, plain doctests will likely fail parsing to begin with because they'll just have a bunch of expressions floating in module scope :weary:

QuietMisdreavus (Jan 15 2019 at 19:27, on Zulip):

okay, after experimenting with a vanilla proc-macro to massage doctests, i think it's possible, but probably more cumbersome than it's worth

and i don't know how i'd hook it into the compile process anyway

QuietMisdreavus (Jan 15 2019 at 19:29, on Zulip):

i looked a little more at the compile controller stuff and i'm not sure there's a way to really do something on the libsyntax side without completely reimplementing rustc_driver::driver::phase_1_parse_input with new parsing code >_>

QuietMisdreavus (Jan 15 2019 at 19:31, on Zulip):

am i completely off-base? is attempting to parse something that looks like a doctest (inner attributes, some items, and some expressions that need to be wrapped by a main function) going to cause that much more headache?

Last update: May 24 2020 at 22:50UTC