Stream: t-compiler/wg-diagnostics

Topic: elm error message catalog

nikomatsakis (Oct 22 2019 at 13:06, on Zulip):

Hey @Esteban Küber and other @WG-diagnostics folks:

Did you all see the new elm post about syntax errors? Some cool stuff in there. One thing I particularly found interesting was this:

So language designers never hear about this problem. I only understood its magnitude once elm/error-message-catalog got going. That repo solicits confusing error messages in hopes of finding ways to improve. I think projects like that legitimize the idea that "error messages should be better" such that I started hearing from a broader range of people. (Not just the very non-random sample of users that participate online!)

I wonder if there are steps we can take to help encourage people to report confusing error messages. We already get enough, so maybe we don't need to do any new work here -- and I'm not sure if a dedicated repo is what we need -- maybe an issue template or something? Not sure. Anyway I thought it was cool!

(hat tip @boats, who sent it to me)

nikomatsakis (Oct 22 2019 at 13:07, on Zulip):

I feel like we've been doing these syntax oriented errors for a while now :)

davidtwco (Oct 22 2019 at 13:09, on Zulip):

That's a awesome post. There's always room to do more, but I think we already (and by we, I mean @Esteban Küber mostly) do a pretty good job at encouraging people to report confusing error messages and trying to improve them.

centril (Oct 22 2019 at 13:13, on Zulip):

@nikomatsakis I read the post earlier; some points:

  1. We already get a lot of these reports; we track them with D-newcomer, D-confusing, etc.
  2. We need to balance the happy path of the parser (the actual grammar of the language) with the amount of code required to do recovery and how invasive such recovery is. If an error message can be isolated into a dedicated method then fine, but I'm wary if it is not and requires tracking a lot of state.
  3. We need to balance these long error messages which may be good for newcomers with inflating how quickly an error message can be read for intermediate and regular users. Longer diagnostics should be activated with --teach
  4. I think we already offer good, actionable, and often MachineApplicable suggestions. I'm not a fan of the anthropomorphization of the compiler, it doesn't make errors more actionable and just increases wordiness
centril (Oct 22 2019 at 13:14, on Zulip):

I feel like we've been doing these syntax oriented errors for a while now :slight_smile:

Yes, the majority of the code in the parser is probably for such errors.

centril (Oct 22 2019 at 13:18, on Zulip):

[...] I think projects like that legitimize the idea that "error messages should be better" [...]

I think that's already an idea we back, but as aforementioned, diagnostics have a cost, and we should be cognizant of the bugs they introduce (ICEs, deviations from the actual grammar) and the difficulty of reading the code.

Esteban Küber (Oct 22 2019 at 23:11, on Zulip):

1) regarding style, were a bit more terse and less friendly in the copy we choose to use, but we're friendlier than other compilers out there. I feel we strike a good middle point
2) We should probably resuscitate --teach as a concept. I've spent a lot of time trying to add info to the existing default errors, but there's so much more we could mention with teach.
3) There's a lot of code for diagnostics in the compiler already. We effectively support a superset of the language. But as we get increasingly more user friendly, it's getting harder to stay with no impact to the performance of the happy path.
4) We do have a bias towards people with some level of experience writing code. Catering to absolute newcomers would probably be best served by the teach flag and possibly a more aggressive culling of errors.

varkor (Oct 23 2019 at 00:57, on Zulip):

I prefer the terser error messages — the paragraph of text works when you just have one error, but not 10

Last update: Mar 31 2020 at 02:10UTC