Stream: t-compiler/wg-meta

Topic: reducer ICE-breakers


nikomatsakis (Nov 14 2019 at 15:22, on Zulip):

The other idea we had was to have a ICE-breaker around bisecting and reducing -- we need a good name for this -- but I definitely think we should do it. Leaving a thread here.

nikomatsakis (Nov 14 2019 at 15:22, on Zulip):

We did have an issue on it, I think

pnkfelix (Nov 14 2019 at 19:03, on Zulip):

boy I really better finish writing the blog post I started drafting on this subject (on reducing in particular)

Santiago Pastorino (Nov 14 2019 at 19:08, on Zulip):

@nikomatsakis btw, what would the bisecting and reducing wg would be about exactly?

nikomatsakis (Nov 14 2019 at 19:08, on Zulip):

@Santiago Pastorino sometimes we have a bug report

nikomatsakis (Nov 14 2019 at 19:08, on Zulip):

that is like "something crashed in my repo"

nikomatsakis (Nov 14 2019 at 19:08, on Zulip):

the idea would be to have people who just try to turn that from "something crashed" to

nikomatsakis (Nov 14 2019 at 19:08, on Zulip):

"here is a self-contained playground example that broke because of PR #123"

Santiago Pastorino (Nov 14 2019 at 19:09, on Zulip):

I see

nikomatsakis (Nov 14 2019 at 19:10, on Zulip):

So I think an alternative could be to try and have a dedicated set of people (sort of like a standard working group) oriented around this thing, but I think I'd sort of prefer the "just ping a bunch of people and at least one or two of them get interested" and see if it works

Santiago Pastorino (Nov 14 2019 at 19:12, on Zulip):

yeah makes sense

Cem Karan (Nov 14 2019 at 19:16, on Zulip):

Do they give access to their repos? If they're saying that something crashed and nothing else, then things will be... difficult.

pnkfelix (Nov 14 2019 at 19:19, on Zulip):

we tend to ask for the source crate and the build/test invocation

pnkfelix (Nov 14 2019 at 19:20, on Zulip):

(at the very least)

Cem Karan (Nov 14 2019 at 19:22, on Zulip):

OK, please forgive my ignorance, but would it be possible to script git bisect to do most of this work for you? Or is the goal to really minimize the output of git bisect?

pnkfelix (Nov 14 2019 at 19:23, on Zulip):

(as an example, i spent a while today hunting for some way to reproduce #65774, before giving up and closing the bug)

pnkfelix (Nov 14 2019 at 19:24, on Zulip):

@Cem Karan oh, hold on: I think @nikomatsakis failed to provide context about what we are reducing and bisecting-over

pnkfelix (Nov 14 2019 at 19:24, on Zulip):

The "reduction" under discussion here is not the git history (which is what git bisect attacks)

pnkfelix (Nov 14 2019 at 19:25, on Zulip):

the reduction is instead of the input source crate (or crates)

pnkfelix (Nov 14 2019 at 19:25, on Zulip):

and likewise the bisection is over that source code

pnkfelix (Nov 14 2019 at 19:26, on Zulip):

to try to reduce the source code to some minimal amount where the problem of interest still arises

Cem Karan (Nov 14 2019 at 19:29, on Zulip):

@pnkfelix OK, thanks for the explanation, but if the crate or crates are also under git, then could we do git bisect on those crates to at least get a clue as to what chunk of code is causing the issue? I know it isn't ideal as different people have different qualities of committing, but it will pare down the amount of code that needs to be reviewed. Once that's done, then you can go in by hand to find the real problem area.

Or am I still missing the point???

pnkfelix (Nov 14 2019 at 19:30, on Zulip):

yes, that is also a technique for identifying smoking guns

pnkfelix (Nov 14 2019 at 19:31, on Zulip):

one that we are familiar with; after all, we use it ourself (for cargo-bisect-rustc)

pnkfelix (Nov 14 2019 at 19:31, on Zulip):

And its not unreasonable to include it under the techniques available for trying to identify the problem area

pnkfelix (Nov 14 2019 at 19:34, on Zulip):

But even with that identified git commit in hand, the rustc developers will still ask you to provide a mcve

pnkfelix (Nov 14 2019 at 19:36, on Zulip):

and of course, while the diff from a single commit may be small, that diff is usually not sufficient the bug report. So you'd still need to figure out how best to reduce the original test (either by building up a new example from scratch, informed by the identified commit; or by reducing down the original crate source)

Cem Karan (Nov 14 2019 at 19:37, on Zulip):

@pnkfelix :sweat_smile: I should have realized that you guys would already have thought of that!

pnkfelix (Nov 14 2019 at 19:37, on Zulip):

Note that cargo-bisect-rustc is tailored solely for bisecting the rustc development history, not arbitrary crates. :smiley:

Cem Karan (Nov 14 2019 at 19:42, on Zulip):

Yes, and I just thought of something else; the rust project has done a really, really good job of trying to ensure that PRs will always compile at the very least before they are accepted. Random crates out the wild may not be that clean, so git bisect might not even be a good option.

Cem Karan (Nov 14 2019 at 19:43, on Zulip):

OK, this is a wild spitballing type of idea, which is likely to be difficult to do, but...

Cem Karan (Nov 14 2019 at 19:45, on Zulip):

Since rustc has access to the complete AST, is it possible to selectively delete portions of the tree? If the crate compiles and the problem still exists, repeat the process, otherwise back out what you did, and try to delete a different portion. Over time, you'll have a minimal code base that you can test.

I'll be the first to admit that it would be difficult to implement, but it might be a starting point for a better idea...

pnkfelix (Nov 14 2019 at 19:46, on Zulip):

no its not a bad idea at all. The blog post I'm in the process of authoring describes a set of transformations similar to that (though not always deleting)

pnkfelix (Nov 14 2019 at 19:46, on Zulip):

for example, replacing function bodies with loop { } is almost always "valid"

pnkfelix (Nov 14 2019 at 19:47, on Zulip):

so I indeed have often mused about trying to implement a tool that mechanizes the search over these reducing transformations

pnkfelix (Nov 14 2019 at 19:47, on Zulip):

or leveraging something like quickcheck to do the search

Cem Karan (Nov 14 2019 at 19:47, on Zulip):

quickcheck/proptest would be a good way to go...

pnkfelix (Nov 14 2019 at 19:47, on Zulip):

but I wanted to first document the transformations I find useful for reducing by hand

pnkfelix (Nov 14 2019 at 19:48, on Zulip):

and let some one else worry about implementing the reduction tool

Cem Karan (Nov 14 2019 at 19:50, on Zulip):

What about using unimplemented!() instead of loop {}? If a function with loop{} is called, the program will just hang, which may make finding the bug harder.

Alternatively, if the function's return value implements Default, you might just replace the function body with Default::default(), which should always produce a valid, if unexpected, value.

pnkfelix (Nov 14 2019 at 19:50, on Zulip):

for example, replacing function bodies with loop { } is almost always "valid"

(when one is debugging ICE's, that is)

pnkfelix (Nov 14 2019 at 19:50, on Zulip):

sorry, I forgot to include that detail before

pnkfelix (Nov 14 2019 at 19:51, on Zulip):

A lot of the bugs we are trying to reduce are solely compile-time issues

pnkfelix (Nov 14 2019 at 19:51, on Zulip):

and so the runtime behavior is irrelevant and can be discarded entirely

Cem Karan (Nov 14 2019 at 19:51, on Zulip):

OK, then you're right, it doesn't matter what goes in the function body.

Cem Karan (Nov 14 2019 at 19:52, on Zulip):

Actually... what about procedural macros?

pnkfelix (Nov 14 2019 at 19:53, on Zulip):

well the behavior of those may or may not need to be preserved, depending on the bug

pnkfelix (Nov 14 2019 at 19:53, on Zulip):

so I'm not really talking about blindly replacing all method bodies and throwing up your hands if things go awry afterwards

pnkfelix (Nov 14 2019 at 19:55, on Zulip):

(you cannot do such blind replacement anyway. Reason 1: you may need one or more bodies to reproduce the bug. Especially if impl Trait in return position is involved. Reason 2: const fn does not support loop { } as body.)

Cem Karan (Nov 14 2019 at 19:57, on Zulip):

so I'm not really talking about blindly replacing all method bodies and throwing up your hands if things go awry afterwards

Makes sense; so what you need to do is figure out what can be replaced, and what you can replace it with.

(you cannot do such blind replacement anyway. Reason 1: you may need one or more bodies to reproduce the bug. Especially if impl Trait in return position is involved. Reason 2: const fn does not support loop { } as body.)

Does const fn support unimplemented!()?

Cem Karan (Nov 14 2019 at 19:58, on Zulip):

Can't help with reason 1

Cem Karan (Nov 14 2019 at 20:03, on Zulip):

As for not blindly replacing method bodies, let's take a look at that. One method of solving this problem is to do exactly that, randomly replacing method bodies and recompiling. One of two things will happen; either the compile will fail, or it will succeed. If it fails, then back out the changes, if it succeeds, then keep the body swap. This can continue until you have some minimal set of methods that you can't replace any more. At that point, you can start deleting methods/functions that have had their bodies replaced. This will further reduce the code. It may be enough to get to the root of the bug, or it may not, no way to know without trying.

pnkfelix (Nov 14 2019 at 21:49, on Zulip):

I think we are in vigorous agreement?

pnkfelix (Nov 14 2019 at 21:55, on Zulip):

By the way, a reason I choose to use loop { } rather than unimplemented!() is that it removes the dependence on that unimplemented!() macro (and the underlying panic machinery).

pnkfelix (Nov 14 2019 at 21:56, on Zulip):

For many bugs, this distinction doesn't matter.

pnkfelix (Nov 14 2019 at 21:57, on Zulip):

But there are some bugs where you are working in a #![no_core] scenario, and you do not have that macro available.

pnkfelix (Nov 14 2019 at 21:59, on Zulip):

Does const fn support unimplemented!()?

No. Not yet at least.

Last update: Dec 12 2019 at 00:55UTC