Stream: rustdoc

Topic: Json IDs


view this post on Zulip CraftSpider (Jan 20 2021 at 17:13):

Looking into This comment. From what I understand, the issue is that a DefId is based on the compiler's understanding of what is public, while a HirId is based on items, but HirId isn't available to external items and DefId won't work right with items marked public in multiple locations but only visible from one?

view this post on Zulip Joshua Nelson (Jan 20 2021 at 17:18):

@CraftSpider the issue is that, while HirId and DefIds have overlap, neither is a proper subset of the other

view this post on Zulip Joshua Nelson (Jan 20 2021 at 17:18):

not all HirIds are serialized cross-crate, and HirIds are only available in the local crate

view this post on Zulip CraftSpider (Jan 20 2021 at 17:19):

So basically, JSON is trying to represent a concept that neither is quite right for

view this post on Zulip Joshua Nelson (Jan 20 2021 at 17:19):

yes

view this post on Zulip CraftSpider (Jan 20 2021 at 17:23):

Alright, time to stare at the code for a bit, then brainstorm solutions, unless you already have a specific one in mind?

view this post on Zulip Joshua Nelson (Jan 20 2021 at 17:27):

the issue I think is that rustdoc is exposing too much information

view this post on Zulip Joshua Nelson (Jan 20 2021 at 17:27):

if we could just serialize DefIds without HirIds it would be fine

view this post on Zulip Joshua Nelson (Jan 20 2021 at 17:27):

and I would prefer to do that rather than a JsonID or whatever

view this post on Zulip Joshua Nelson (Jan 20 2021 at 17:29):

sorry I keep changing my mind on this

view this post on Zulip Joshua Nelson (Jan 20 2021 at 17:29):

cc @Nixon Enraght-Moony

view this post on Zulip CraftSpider (Jan 20 2021 at 17:30):

What do you mean? HirId is currently unused in the JSON. Do you mean similar to the solution proposed in the original issue, just don't document the unreachable items, or something entirely different?

view this post on Zulip Joshua Nelson (Jan 20 2021 at 17:30):

yes, don't document things that don't have a DefId

view this post on Zulip Joshua Nelson (Jan 20 2021 at 17:31):

hmm, I guess this breaks anyone that wants to do similar things to rustdoc and show the item at that scope

view this post on Zulip Joshua Nelson (Jan 20 2021 at 17:31):

how does rustdoc even handle this?

view this post on Zulip CraftSpider (Jan 20 2021 at 17:33):

        // Stripped modules survive the rustdoc passes (i.e., `strip-private`)
        // if they contain impls for public types. These modules can also
        // contain items such as publicly re-exported structures.
        //
        // External crates will provide links to these structures, so
        // these modules are recursed into, but not rendered normally
        // (a flag on the context).
        if !self.render_redirect_pages {
            self.render_redirect_pages = item.is_stripped();
        }

Appears in html/render/mod.rs, which sounds relevant

view this post on Zulip CraftSpider (Jan 20 2021 at 17:39):

is_stripped seems to special case ImportItems, referencing !i.should_be_displayed. The handling of render_redirect_pages is spread across multiple places, but in one place relates to whether rendering does layout::render or layout::redirect.

view this post on Zulip Joshua Nelson (Jan 20 2021 at 17:40):

@CraftSpider right, but that's not really what I mean - I want to make sure info for doing this is available to other tools if they want it

view this post on Zulip Joshua Nelson (Jan 20 2021 at 17:40):

Not necessarily that they use the same logic

view this post on Zulip Joshua Nelson (Jan 20 2021 at 17:41):

So I guess that does mean it needs a separate id, one that represents both HIR and DefIds

view this post on Zulip CraftSpider (Jan 20 2021 at 17:42):

Yeah. If we want to preserve this information we need a new ID, unless one wants to add some kind of optional 're-exported by' field

view this post on Zulip Joshua Nelson (Jan 20 2021 at 17:43):

Hmm, that might be easier for the consumer

view this post on Zulip Joshua Nelson (Jan 20 2021 at 17:43):

@HeroicKatora weren't you planning to use the JSON backend for something?

view this post on Zulip Joshua Nelson (Jan 20 2021 at 17:43):

How would you prefer to expose info about re-exports?

view this post on Zulip HeroicKatora (Jan 20 2021 at 17:45):

Yep, already started but haven't gotten far. So basically, parsing the json and generating markdown/tex

view this post on Zulip Joshua Nelson (Jan 20 2021 at 17:45):

Oh wow, nice!

view this post on Zulip CraftSpider (Jan 20 2021 at 17:46):

Oh, nice

view this post on Zulip Joshua Nelson (Jan 20 2021 at 17:46):

I presume you want to know about re-exports, right? This is really easy on our end if we just delete that and only emit the canonical definition lol

view this post on Zulip CraftSpider (Jan 20 2021 at 17:47):

Haha. I think in terms of ease, 'ignoring re-exports' < 're-exported at field' < 'custom IDs' :P

view this post on Zulip HeroicKatora (Jan 20 2021 at 17:49):

Yeah, I'd prefer to know. I believe something similar to typedefs would fit best.

view this post on Zulip HeroicKatora (Jan 20 2021 at 17:49):

I'll have to consider it for a bit though and eat, let me get back to you in a bit

view this post on Zulip Nixon Enraght-Moony (Jan 20 2021 at 23:11):

HeroicKatora said:

Yep, already started but haven't gotten far. So basically, parsing the json and generating markdown/tex

If this is public, can you send the link?

view this post on Zulip CraftSpider (Jan 22 2021 at 19:28):

@HeroicKatora Any updates on this?

view this post on Zulip HeroicKatora (Jan 23 2021 at 11:13):

https://github.com/oberien/heradoc/tree/rustdoc/src/frontend/rustdoc
That's the repository and branch, I'm not pushing regularly as I'm not very certain of the implementation yet ;)

view this post on Zulip Joshua Nelson (Jan 23 2021 at 14:46):

types.rs

FYI we plan to make this a separate crate shortly so you can reuse exactly the same types rustdoc uses: https://github.com/rust-lang/rust/pull/81287

view this post on Zulip Joshua Nelson (Jan 23 2021 at 14:51):

target.push("doc");

This can be wrong if the docs are cross compiled, or if a crate has doc = false set. Unfortunately I don't know a more accurate way without building the docs yourself. There's an example of that here: https://github.com/deadlinks/cargo-deadlinks/blob/3e984d56e4e6e92e6ae31d8a09758e57f55fdaa3/src/bin/cargo-deadlinks.rs#L210

view this post on Zulip Joshua Nelson (Jan 23 2021 at 14:52):

I might publish a non-official library for writing rustdoc plugins

view this post on Zulip Joshua Nelson (Jan 24 2021 at 13:28):

Joshua Nelson said:

I might publish a non-official library for writing rustdoc plugins

@HeroicKatora if I made this, would you be interested in using it? mostly it would be refactoring things out of cargo-deadlinks so it wouldn't be a ton of work on my end

view this post on Zulip HeroicKatora (Jan 24 2021 at 14:16):

As a real plugins? Not necessarily. Consuming the json output is okay and retaining control over the execution as a separate process works fine as well. Also I'm not interested in injecting more sources for link resolution (in fact that's pretty much a non-goal for the convertion I'm writing). At least not at the moment. But I'm not too sure about it yet. Maybe something for intra-doc links comes up that can't be easily solved by translating links, or I might need to influence the way it constructs IDs.

view this post on Zulip Joshua Nelson (Jan 24 2021 at 14:16):

not as in buliding from source, just like an easy way to wrap cargo rustdoc

view this post on Zulip HeroicKatora (Jan 24 2021 at 14:26):

That might be more interesting. In particular, getting the file name of the json output etc. and program options? Might be cool to discover how to use it best and what else exists for using the doc tool.

view this post on Zulip Joshua Nelson (Jan 24 2021 at 14:26):

I would have a rustdoc_args to pass things directly I think, otherwise it would get out of date rapidly

view this post on Zulip Joshua Nelson (Jan 24 2021 at 14:27):

but the main benefit is you wouldn't have to guess where cargo is outputting the docs


Last updated: Oct 11 2021 at 22:34 UTC