Stream: t-compiler/help

Topic: ui test annotations

ange (Apr 11 2019 at 23:13, on Zulip):

What is the point of machine-checked annotations in testcases when you have to capture all error output for the ui tests anyway?

ange (Apr 11 2019 at 23:16, on Zulip):

(I'm asking because getting the exact compiler output right for a testcase that cannot yet pass is much more involved than simply saying "I'm expecting this error here")

davidtwco (Apr 11 2019 at 23:50, on Zulip):

If you pass --bless then it updates all the ui test output. It's easy to do this and miss that you've accidentally blessed a regression (a missing error, say) if there are a lot of changed tests. By also using annotations, you need to acknowledge that a test lost an error.

ange (Apr 11 2019 at 23:56, on Zulip):

Thanks, that makes sense. So I guess the reason you can't just have an annotations-cover-all-expected-error-output header command is to make sure one doesn't miss subtle changes in the formatting of the output and instead needs to explicitly --bless them?

davidtwco (Apr 12 2019 at 07:02, on Zulip):

Exactly, without stderr files we’d miss small things like a span changing.

Last update: Sep 18 2020 at 20:00UTC