Stream: rustdoc

Topic: surprising external crates with the json output format


view this post on Zulip Urgau (Jul 31 2021 at 14:26):

Hello, I was experimenting with rustdoc and it's json output format when I realized that for a 5 lines lib.rsproduce over 23000 lines of json when I open it I saw the entire standard library (core, alloc, std, ...) but also some internal crates like rustc_std_workspace_alloc, rust_std_workspace_core but also rustc_demangle. Is that expected ?

view this post on Zulip Joshua Nelson (Jul 31 2021 at 17:25):

Hmm, it does seem weird that it includes items from other crates

view this post on Zulip Joshua Nelson (Jul 31 2021 at 17:25):

@Nixon Enraght-Moony @CraftSpider @Joseph Ryan do one of you happen to remember why rustdoc does that? Or is it unintentional?

view this post on Zulip Joseph Ryan (Jul 31 2021 at 18:12):

Yeah so the reason that the output is so bloated right now is that it takes ALL items in scope (so everything in the prelude and transitively referenced by any external crate that you bring into scope) and includes them in the output. The affected fields are index which is supposed to include the full trait definition for any trait used (even if it's from an external crate) and paths which contains the full module path of any item referenced.

The correct thing to do which we didn't get around to on the first pass implementation was to write a visitor that would figure out which items were actually "reachable" by following all id references from the crate root and only including those in the output. While the output would still contain stuff like blanket impls from std and external crates, it would definitely make the resulting blob much much smaller.

view this post on Zulip Joseph Ryan (Jul 31 2021 at 18:16):

Since removing unreachable output won't change any functionality it shouldn't be too hard to go ahead and write this pruning pass without messing with other parts of the code, if I have some time this week I'll try to make a pass at it, but if anyone else would like to I can walk you through it as well. I'd definitely like to be able to open the json output in my editor instead of poking at it with jq due to it's size :sweat_smile:


Last updated: Oct 11 2021 at 22:34 UTC