Stream: rustdoc

Topic: @has checks "no closing quotation"


view this post on Zulip Noah Lev (Feb 08 2021 at 21:35):

What's wrong with these @has checks? I'm getting ValueError: No closing quotation from htmldocck.

// @has "foo/index.html" "//p" "This is the “start” of the ‘document’! How’d you know that “it’s” the start?"
// @has "foo/index.html" "//h1" "Header with “smart punct’”"
// @has "foo/index.html" '//a[@href="https://www.rust-lang.org"]' "link with “smart punct’” – yessiree!"
// @has "foo/index.html" '//code' 'this inline code -- it shouldn\'t have "smart punct"'
// @has "foo/index.html" '//pre' 'let x = "don\'t smart-punct me -- please!";'
// @has "foo/index.html" '//pre' 'I say "don\'t smart-punct me -- please!"'

view this post on Zulip Noah Lev (Feb 08 2021 at 21:35):

I'm pretty sure I escaped all of the ' marks.

view this post on Zulip Noah Lev (Feb 08 2021 at 21:39):

It looks like it's hiccuping on this part:

"foo/index.html" '//code' 'this inline code -- it shouldn\'t have "smart punct"'

view this post on Zulip Joshua Nelson (Feb 08 2021 at 22:16):

I don't think it reads escaped quotations

view this post on Zulip Joshua Nelson (Feb 08 2021 at 22:16):

You could add support for that if you like, seems useful

view this post on Zulip Joshua Nelson (Feb 08 2021 at 22:17):

Or just use double quotes instead

view this post on Zulip GuillaumeGomez (Feb 08 2021 at 22:45):

It should handle escape ;)

view this post on Zulip GuillaumeGomez (Feb 08 2021 at 22:46):

Interesting that we didn't have this issue until now

view this post on Zulip CraftSpider (Feb 08 2021 at 23:34):

It's because you're using single-quotes around the content

view this post on Zulip CraftSpider (Feb 08 2021 at 23:35):

Python shlex interpets "foo\"" as foo", but 'foo\'' as 'foo', ' (an unclosed second string)

view this post on Zulip CraftSpider (Feb 08 2021 at 23:36):

(And "foo'" as foo')

view this post on Zulip Noah Lev (Feb 08 2021 at 23:46):

Joshua Nelson said:

Or just use double quotes instead

I can't do that because I use double quotes too.

view this post on Zulip CraftSpider (Feb 08 2021 at 23:48):

Camelid said:

Joshua Nelson said:

Or just use double quotes instead

I can't do that because I use double quotes too.

You can escape inside double quotes, but not single quotes. So "... ' ... \"this things\"" should work

view this post on Zulip Noah Lev (Feb 08 2021 at 23:48):

Trying that now...

view this post on Zulip Noah Lev (Feb 08 2021 at 23:48):

CraftSpider said:

Python shlex interpets "foo\"" as foo", but 'foo\'' as 'foo', ' (an unclosed second string)

That's really bizarre behavior. I wonder why it does that...

view this post on Zulip Noah Lev (Feb 08 2021 at 23:48):

Indeed, that fixed it. Thanks so much @CraftSpider!

view this post on Zulip CraftSpider (Feb 08 2021 at 23:51):

In bash, try echo "foo\"" 'foo\'', to see why shlex does it like that :P

view this post on Zulip Noah Lev (Feb 08 2021 at 23:51):

I think you might have one extra ' there?

view this post on Zulip CraftSpider (Feb 08 2021 at 23:53):

Yeah, fixed. One more ' ends the line. Basically, shlex is supposed to behave like a shell does, and thus 'expands' things in ", but doesn't in '.

view this post on Zulip Noah Lev (Feb 08 2021 at 23:53):

Huh, I think I see. Still a bit confusing. Is that why it's called shlex?

view this post on Zulip CraftSpider (Feb 08 2021 at 23:54):

Yeah. According to the python shlex documentation:

The shlex class makes it easy to write lexical analyzers for simple syntaxes resembling that of the Unix shell. This will often be useful for writing minilanguages, (for example, in run control files for Python applications) or for parsing quoted strings.

view this post on Zulip CraftSpider (Feb 08 2021 at 23:56):

It can be configured to treat ' and " the same, which might be less confusing for people of the future

view this post on Zulip Noah Lev (Feb 09 2021 at 00:44):

How do you configure it that way? Do you want to open a PR for that?

view this post on Zulip Joshua Nelson (Feb 09 2021 at 01:13):

I would rather make it more clear that it uses shlex

view this post on Zulip Joshua Nelson (Feb 09 2021 at 01:13):

Or use something other than shlex

view this post on Zulip Joshua Nelson (Feb 09 2021 at 01:13):

"sort of but not exactly the same as shlex" sounds like the worst of both worlds

view this post on Zulip CraftSpider (Feb 09 2021 at 01:48):

So, a couple options:
1) Document clearly that htmldocck (and jsondocck) use shlex, with all the conditions that entails
2) Configure shlex, by setting 'escapedquotes' to include '
3) Use something else / roll our own simpler parser, with just like '', "", and \

view this post on Zulip Noah Lev (Feb 09 2021 at 20:19):

I'm fine with (2) or (3), but I would rather not do (1). It didn't cross my mind at all that @has checks would be based on shell syntax, and it seems better to have behavior consistent with languages like Python and JS where ' and " are pretty much identical rather than with shell syntax.

view this post on Zulip GuillaumeGomez (Feb 09 2021 at 20:26):

2) would be nice!

view this post on Zulip Joshua Nelson (Feb 09 2021 at 20:28):

I'm ok with 2) if other people like it


Last updated: Oct 11 2021 at 22:34 UTC