Stream: t-compiler

Topic: #56900 stack overflow with `include_dir` crate


QuietMisdreavus (Feb 26 2019 at 19:02, on Zulip):

i'm trying to investigate this issue and i'm not totally sure how to debug it. there's something weird happening where if you use include_dir to load in a large file, it causes rustc/rustdoc to overflow its main-thread stack

QuietMisdreavus (Feb 26 2019 at 19:02, on Zulip):

but if you translate the intended proc-macro output into code directly, it builds fine

QuietMisdreavus (Feb 26 2019 at 19:04, on Zulip):

OTOH, something must have changed between stable and nightly, because the memory usage doesn't balloon out of control on nightly :confused:

QuietMisdreavus (Feb 26 2019 at 19:09, on Zulip):

nor on beta, for that matter :thinking:

oli (Feb 26 2019 at 19:14, on Zulip):

O_o So it seems to have accidentally gotten fixed... A regression test needs some large files, right? Can we generate them so we don't have to have them in git?

QuietMisdreavus (Feb 26 2019 at 19:15, on Zulip):

the rustdoc search index for the standard library is sufficient

QuietMisdreavus (Feb 26 2019 at 19:15, on Zulip):

it still takes ages to build tho

QuietMisdreavus (Feb 26 2019 at 19:16, on Zulip):

judging from what the person saw with cargo expand, it looks like it's trying to load the files into byte literals during expansion?

QuietMisdreavus (Feb 26 2019 at 19:19, on Zulip):

i'm not sure how to reduce a test case, though - just manually writing include_bytes!("search-index.js") isn't enough to trigger the behavior

QuietMisdreavus (Feb 26 2019 at 19:19, on Zulip):

the macro call may need to be written by a proc-macro? there may be some weird interaction with the quote crate?

oli (Feb 26 2019 at 19:22, on Zulip):

so it's literally about a macro that just dumps an include_bytes! call without doing anything else?

QuietMisdreavus (Feb 26 2019 at 19:23, on Zulip):

potentially. i haven’t actually tried that

QuietMisdreavus (Feb 26 2019 at 19:24, on Zulip):

include_dir is just a directory scan and file name collection before printing a struct expression to the output token stream, including the include_bytes! calls

QuietMisdreavus (Feb 26 2019 at 19:24, on Zulip):

it does use proc-macro-hack, but hopefully that’s not part of it

QuietMisdreavus (Feb 26 2019 at 21:03, on Zulip):

update: a trivial proc-macro that expands to pub static ASDF: &'static [u8] = include_bytes!("search-index.js") is not enough to trigger the bug

QuietMisdreavus (Feb 26 2019 at 21:18, on Zulip):

...not even when going through quote and using a string literal from outside the proc-macro

QuietMisdreavus (Feb 26 2019 at 21:46, on Zulip):

not even proc-macro-hack by itself will do it :expressionless:

Last update: Nov 22 2019 at 05:45UTC