Stream: rustdoc

Topic: Adding API for OpenSearch


view this post on Zulip CodeSpelunker (Feb 24 2021 at 16:37):

Hi!
It would be really useful if doc.rust-lang.org had some kind of API that returned OpenSearch-compatible application/x-suggestions+json. Such a change would allow autocomplete suggestions when you activate that search engine in a browser URL bar.
What would the work involved with such a change look like? Is this something I could help with working on?

view this post on Zulip Joshua Nelson (Feb 24 2021 at 16:46):

This was discussed a little on Discord. Some points that were brought up:

I'd like to hear more about your use case - what do you want this for?

view this post on Zulip CodeSpelunker (Feb 24 2021 at 16:59):

I make fairly heavy use of Firefox's domain-specific search engines, and I imagine many others do as well.
Integrating with the browser suggestion feature would be very convenient in allowing previews of crate names and functions before the search is completed.
As for the XML point: I haven't looked into this much, but I believe this is just the manifest which provides the "Add Search Engine" menu option. I believe it would be fairly straightforward to host this statically.
The main interesting bit as I see it is the API that would be called to populate the suggestions.

view this post on Zulip apiraino (Feb 24 2021 at 17:02):

@CodeSpelunker if you want to search in Rust docs from your url bar you have these options:
1) the rust firefox add-on: https://addons.mozilla.org/de/firefox/addon/rust-search-extension
2) !bang syntax from duckduckgo : !rustdoc Option
3) roll your own search OpenDocument XML: https://apiraino.github.io/firefox-custom-search :-)

view this post on Zulip Joshua Nelson (Feb 24 2021 at 17:04):

if it's as simple as hosting the static manifest, that seems fine to me

view this post on Zulip apiraino (Feb 24 2021 at 17:06):

yeah, it's a simple XML file and then add it to a <link.... in the headers of the website

view this post on Zulip Joshua Nelson (Feb 24 2021 at 17:06):

@apiraino want to make a PR for that? :wink:

view this post on Zulip apiraino (Feb 24 2021 at 17:06):

I can open a PR it you want

view this post on Zulip apiraino (Feb 24 2021 at 17:06):

oh right you got me :D

view this post on Zulip CodeSpelunker (Feb 24 2021 at 17:09):

@apiraino, this looks very useful, and might even obviate what I'm talking about.
@Joshua Nelson, I believe the XML defines a few things, which FF/Chrome parses to register the search engine. Among that metadata is the search engine name, the standard search API (like when you just press Enter or select a suggestion), and the suggestions API returning application/x-suggestions+json.

view this post on Zulip Joshua Nelson (Feb 24 2021 at 17:09):

and the suggestions API returning application/x-suggestions+json.

How would that work for a static server?

view this post on Zulip CodeSpelunker (Feb 24 2021 at 17:11):

That's what I'm not so sure about.
I'd also like to look a little more closely at how the extension @apiraino had linked works. It might be doing something differently than the other search engines I've seen.

view this post on Zulip CodeSpelunker (Feb 24 2021 at 17:12):

I don't know enough about service workers and lambda to give you a good answer on how that would be implemented in this case

view this post on Zulip CodeSpelunker (Feb 24 2021 at 17:20):

If anyone else is interesting in looking through that extension's source code: https://github.com/huhu/rust-search-extension

view this post on Zulip CodeSpelunker (Feb 24 2021 at 17:38):

@apiraino This extension won't work for me.
There is no way to customize the prefix shortcut through the Settings interface, and the suggestions get mangled with my default search engine (Google) settings. Underneath the extension results, I also get Google autocomplete results for rs std::vec.
So, back to square one here.

view this post on Zulip GuillaumeGomez (Feb 24 2021 at 18:37):

To be noted, it has to be done on doc.rust-lang.org side and not in rustdoc

view this post on Zulip GuillaumeGomez (Feb 24 2021 at 18:38):

and I don't know how it works there...

view this post on Zulip apiraino (Feb 24 2021 at 20:59):

CodeSpelunker said:

the suggestions get mangled with my default search engine (Google) settings. Underneath the extension results, I also get Google autocomplete results for rs std::vec.

@CodeSpelunker what a drag. But thanks for reporting on that

view this post on Zulip apiraino (Feb 24 2021 at 21:02):

GuillaumeGomez said:

To be noted, it has to be done on doc.rust-lang.org side and not in rustdoc

didnt look yet at the codebase , you mean the patch should be pushed here or not here?

view this post on Zulip Joshua Nelson (Feb 24 2021 at 21:04):

not there

view this post on Zulip apiraino (Feb 24 2021 at 21:05):

I see. I thought this file here really had the header I was looking for

view this post on Zulip Joshua Nelson (Feb 24 2021 at 21:06):

I'm unclear what you're looking for.

view this post on Zulip Joshua Nelson (Feb 24 2021 at 21:06):

I don't know how this can work as a static file.

view this post on Zulip apiraino (Feb 24 2021 at 21:10):

hmm ok I'll check later a bit the code and be back with a bit more context. I'm thinking two things that may be both wrong:
1) doc.rust-lang.org is an artifact of compiling librustdoc (that html directory)
2) when I compile my crate I use the same static HTML+css assets as doc.rust-lang.org

view this post on Zulip Joshua Nelson (Feb 24 2021 at 21:11):

yes, that's all right, but I don't know how you can do opensearch with static files

view this post on Zulip Joshua Nelson (Feb 24 2021 at 21:11):

oh wait you were asking specifically about the header

view this post on Zulip Joshua Nelson (Feb 24 2021 at 21:12):

sorry, that's the right place for the header

view this post on Zulip apiraino (Feb 24 2021 at 21:13):

and then add another static asset (the XML file)

view this post on Zulip Joshua Nelson (Feb 24 2021 at 21:14):

that would probably be in html/static

view this post on Zulip GuillaumeGomez (Feb 24 2021 at 21:18):

I guess I'll see in the PR, but I don't think adding yet another file for such a specific feature is worth it for rustdoc

view this post on Zulip GuillaumeGomez (Feb 24 2021 at 21:19):

But if the file is always the same and is rather small, then I guess it's fine, even though I think it's too much

view this post on Zulip apiraino (Feb 24 2021 at 21:55):

@GuillaumeGomez for this feature to work it is mandatory to have a 10-ish lines XML similar to what docs.rs already has
https://docs.rs/-/static/opensearch.xml
and add a <link... header to the HTML
No way around this (afaik). I perfectly understand you want to avoid bloat so it will be fine if you don't want to merge :thumbs_up:

view this post on Zulip GuillaumeGomez (Feb 24 2021 at 22:19):

So just like I said: it seems better to put it into doc.rust-lang.org instead

view this post on Zulip GuillaumeGomez (Feb 24 2021 at 22:20):

I don't see why users generating doc locally should care about that


Last updated: Oct 11 2021 at 22:34 UTC