Stream: t-compiler

Topic: cargo-bisect-rustc next steps


Santiago Pastorino (Oct 25 2019 at 17:57, on Zulip):

hey @simulacrum so, about what we layed out in the Hackmd document

Santiago Pastorino (Oct 25 2019 at 17:57, on Zulip):

in particular

Santiago Pastorino (Oct 25 2019 at 17:57, on Zulip):
    - make sure this is easy to override with cargo +stage1 bisect-rustc
Santiago Pastorino (Oct 25 2019 at 17:58, on Zulip):

unsure what you exactly meant, but my patch if you do that would use stage1 as the default nightly

Santiago Pastorino (Oct 25 2019 at 17:58, on Zulip):

I guess that's what you were proposing

simulacrum (Oct 25 2019 at 17:58, on Zulip):

yep, that's what I expected

Santiago Pastorino (Oct 25 2019 at 17:58, on Zulip):

and about ...

Santiago Pastorino (Oct 25 2019 at 17:58, on Zulip):
    - print nice error message for beta/stable as default, see task below
Santiago Pastorino (Oct 25 2019 at 17:58, on Zulip):

unsure what's that exactly

simulacrum (Oct 25 2019 at 17:59, on Zulip):

(though I think stage1 was a bad example, I guess, since stage 1 is usually an unpublished build that we can't do much with)

Santiago Pastorino (Oct 25 2019 at 17:59, on Zulip):

yeah, but I got what you meant, it may have been cargo +nightly-2019-08-01 or something like that

simulacrum (Oct 25 2019 at 18:00, on Zulip):

yep

Santiago Pastorino (Oct 25 2019 at 18:00, on Zulip):

ok

simulacrum (Oct 25 2019 at 18:00, on Zulip):

for - print nice error message for beta/stable as default, see task below

I meant that cargo +stable bisect-rustc (or just stable being the default in the environment) should for now just say "test with nightly first please; then run cargo +nightly bisect-rustc"

Santiago Pastorino (Oct 25 2019 at 18:01, on Zulip):

ahh ok ok

Santiago Pastorino (Oct 25 2019 at 18:01, on Zulip):

do we have a nice way to differenciate that over someone just having stable as their default toolchain?

Santiago Pastorino (Oct 25 2019 at 18:01, on Zulip):

my current code won't differenciate both but I'm not sure if there's a way in which we can

simulacrum (Oct 25 2019 at 18:01, on Zulip):

no, nor do I think we should really

Santiago Pastorino (Oct 25 2019 at 18:02, on Zulip):

basically if the default toolchain is beta or stable it just ignores that and goes to a nightly

Santiago Pastorino (Oct 25 2019 at 18:02, on Zulip):

I think that + maybe some cool messages are the best we can do

Santiago Pastorino (Oct 25 2019 at 18:03, on Zulip):

like your default toolchain is stable/beta using latest nightly instead or something like that

simulacrum (Oct 25 2019 at 18:05, on Zulip):

sure, that works

simulacrum (Oct 25 2019 at 18:05, on Zulip):

I was suggesting to just not even try that hard

Santiago Pastorino (Oct 25 2019 at 18:05, on Zulip):

and last question ...

Santiago Pastorino (Oct 25 2019 at 18:06, on Zulip):
Find a starting point going geometrically (2x each time, with jump of at most 1 month) backwards
  [today; 2 days ago; 4 days ago; 8 days ago; 16 days ago; 1 month ago; 2 months ago]
Santiago Pastorino (Oct 25 2019 at 18:06, on Zulip):

how this differs with what we are doing today?

simulacrum (Oct 25 2019 at 18:06, on Zulip):

it might not

Santiago Pastorino (Oct 25 2019 at 18:06, on Zulip):

yeah, I was guessing that you wanted my stuff to play nicely with it :)

simulacrum (Oct 25 2019 at 18:06, on Zulip):

we also might want to change the max jump to e.g. a week or so

Santiago Pastorino (Oct 25 2019 at 18:06, on Zulip):

and it does

Santiago Pastorino (Oct 25 2019 at 18:07, on Zulip):

yeah, the max jump is really a thing

Santiago Pastorino (Oct 25 2019 at 18:07, on Zulip):

to avoid cases like the one we discussed yesterday

Santiago Pastorino (Oct 25 2019 at 18:07, on Zulip):

I think a week is good enough

Santiago Pastorino (Oct 25 2019 at 18:07, on Zulip):

maybe an algorithm like ...

simulacrum (Oct 25 2019 at 18:07, on Zulip):

ideally we'd not need to -- maybe via some of the finer-grained checks we were talking about -- but a week seems fine too.

Santiago Pastorino (Oct 25 2019 at 18:08, on Zulip):

1 day, 2 days, 3 days, ... 7 days, 2 weeks, 3 weeks, ...

Santiago Pastorino (Oct 25 2019 at 18:08, on Zulip):

if we are aiming to find a regression tipically between two stables we shouldn't jump really long I think

Santiago Pastorino (Oct 25 2019 at 18:09, on Zulip):

I mean, the tool is general purpose of course, but seems like the most frequent use case would be that one, right?

Santiago Pastorino (Oct 25 2019 at 18:09, on Zulip):

maybe even something like

simulacrum (Oct 25 2019 at 18:09, on Zulip):

yeah it seem reasonable to not be too aggressive

Santiago Pastorino (Oct 25 2019 at 18:09, on Zulip):

1 day, 2 days, 3 days, ... 7 days, 2 weeks, 3 weeks, ... 7 weeks, 9 weeks, 11 weeks, something like that

simulacrum (Oct 25 2019 at 18:09, on Zulip):

I largely think we should just try things and sort of "see what sticks"

Santiago Pastorino (Oct 25 2019 at 18:09, on Zulip):

so each 7 repetitions we can * 2 the thing

Santiago Pastorino (Oct 25 2019 at 18:10, on Zulip):

I largely think we should just try things and sort of "see what sticks"

agreed

simulacrum (Oct 25 2019 at 18:10, on Zulip):

I'd maybe not go day-by-day for the first 7, at least 2 day increments seems good

Santiago Pastorino (Oct 25 2019 at 18:11, on Zulip):

:+1:

Santiago Pastorino (Oct 25 2019 at 18:11, on Zulip):

another thing related to that is that you were looking for a starting good release

Santiago Pastorino (Oct 25 2019 at 18:11, on Zulip):

I was wondering, why was that

Santiago Pastorino (Oct 25 2019 at 18:12, on Zulip):

well with the jumps you were doing I think it makes sense

Santiago Pastorino (Oct 25 2019 at 18:12, on Zulip):

but if we were to jump short we could just start with non working and keep finding non working until we see a working one

simulacrum (Oct 25 2019 at 18:17, on Zulip):

hm, yeah, that might work

simulacrum (Oct 25 2019 at 18:17, on Zulip):

not quite bisection but that seems fine yeah

simulacrum (Oct 25 2019 at 18:18, on Zulip):

to be clear I have not put all that much thought into this

Santiago Pastorino (Oct 25 2019 at 18:18, on Zulip):

yeah, won't be that bisection or would be bisection between 2 weeks and between commits then

Santiago Pastorino (Oct 25 2019 at 18:18, on Zulip):

the problem is that the tool would bisect when you give two points

Santiago Pastorino (Oct 25 2019 at 18:19, on Zulip):

if there are not two points there is not much we can do rather than jumping in some way or another

simulacrum (Oct 25 2019 at 18:26, on Zulip):

indeed

Santiago Pastorino (Oct 28 2019 at 14:49, on Zulip):

@simulacrum I was mentioning @nikomatsakis that the changes we have discussed are more or less ready

Santiago Pastorino (Oct 28 2019 at 14:49, on Zulip):

according to https://hackmd.io/Ipu9wSL4SCyx9_adR6GKLg?view the pending things would be to release and to integrate with bots (triagebot?)

Santiago Pastorino (Oct 28 2019 at 14:50, on Zulip):

the question is how exactly to integrate with bots

Santiago Pastorino (Oct 28 2019 at 14:50, on Zulip):

my guess is @triagebot cargo-bisect-rustc https://github.com/spastorino/my-regressed-app

simulacrum (Oct 28 2019 at 14:50, on Zulip):

That seems pretty hard, like, nowhere near MVP level

Santiago Pastorino (Oct 28 2019 at 14:51, on Zulip):

and that could even take the same arguments the tool takes, like start and end

Santiago Pastorino (Oct 28 2019 at 14:51, on Zulip):

That seems pretty hard, like, nowhere near MVP level

really?

Santiago Pastorino (Oct 28 2019 at 14:51, on Zulip):

I'd have guessed that was a simple thing to do

simulacrum (Oct 28 2019 at 14:51, on Zulip):

Sure -- the bot needs to run arbitrary rust code, etc

simulacrum (Oct 28 2019 at 14:51, on Zulip):

we do have some sandboxing for this -- e.g. rustwide -- but it's not really trivial

Pietro Albini (Oct 28 2019 at 14:51, on Zulip):

you'd need to switch cargo-bisect-rustc to run the build inside rustwide

Pietro Albini (Oct 28 2019 at 14:52, on Zulip):

and we'd probably need to upgrade the instance

Santiago Pastorino (Oct 28 2019 at 14:52, on Zulip):

why arbitrary?, if the bot has installed the bisect tool isn't that enough?

Santiago Pastorino (Oct 28 2019 at 14:52, on Zulip):

well you meant because of the tests of the app?

simulacrum (Oct 28 2019 at 14:52, on Zulip):

to bisect you need to build which means build scripts etc

Pietro Albini (Oct 28 2019 at 14:52, on Zulip):

yeah but I'm not executing random code from a git repo without a sandbox

simulacrum (Oct 28 2019 at 14:52, on Zulip):

(Build scripts, proc macros) are arbitrary rust code

Santiago Pastorino (Oct 28 2019 at 14:53, on Zulip):

yeah it makes sense

simulacrum (Oct 28 2019 at 14:53, on Zulip):

in any case I think bots would be nice but ultimately running it is not -- ideally -- hard to do locally

simulacrum (Oct 28 2019 at 14:54, on Zulip):

So I'd rather see us say "hey, person who filed issue, can you run this tool?"

simulacrum (Oct 28 2019 at 14:54, on Zulip):

and/or we can do it ourselves (similar to icebreakers)

Santiago Pastorino (Oct 28 2019 at 14:54, on Zulip):

yeah, people can just do cargo install bisect-rustc; cargo-bisect-rustc

Santiago Pastorino (Oct 28 2019 at 14:55, on Zulip):

maybe we can have the output of the tool be something like a github issue report that people can paste on github quickly

simulacrum (Oct 28 2019 at 14:55, on Zulip):

yeah I think we talked about a "markdown" report

Santiago Pastorino (Oct 28 2019 at 14:56, on Zulip):

@nikomatsakis would that be enough?

nikomatsakis (Oct 29 2019 at 13:53, on Zulip):

+1 from me

nikomatsakis (Oct 29 2019 at 13:53, on Zulip):

I think a tool they can run with a report they can copy-and-paste is awesome

Santiago Pastorino (Oct 29 2019 at 14:10, on Zulip):

ideas on things that a report can contain?

Santiago Pastorino (Oct 29 2019 at 14:10, on Zulip):

I guess we could mention in what nightly regressed and in what commit specifically, but if people are going to copy and paste what relevant information for us can we include?

Santiago Pastorino (Oct 29 2019 at 14:11, on Zulip):

I guess the report should look contain some place holders to fill, like feel here a url of a sample application that reproduces this or something like that

nikomatsakis (Oct 29 2019 at 14:11, on Zulip):

hmm:

Santiago Pastorino (Oct 29 2019 at 14:11, on Zulip):

anyway if we are pointed to a specific commit I guess we can figure out what the error is but still ...

nikomatsakis (Oct 29 2019 at 14:11, on Zulip):

that's probably enough

Santiago Pastorino (Oct 29 2019 at 14:12, on Zulip):

yes

Santiago Pastorino (Oct 29 2019 at 14:12, on Zulip):

what I meant is ... it's not just copy and paste

Santiago Pastorino (Oct 29 2019 at 14:12, on Zulip):

we could print something like

Santiago Pastorino (Oct 29 2019 at 14:14, on Zulip):
searched between: 2019-10-29 and 2019-08-20
nightly: 2019-09-21
commit: fa12334fa
error: <paste of the error here>
where to find the code: <PLEASE GIVE US INSTRUCTIONS ON WHAT YOUR APPLICATION DOES OR A URL TO FETCH>
Santiago Pastorino (Oct 29 2019 at 14:15, on Zulip):

@nikomatsakis that's what I meant, it won't be just a copy and paste text, it would be something that you can mostly copy and paste but you need to edit a bit

simulacrum (Oct 29 2019 at 14:59, on Zulip):

I could imagine some heuristics - e.g. if we're in a GitHub repo we could look at the origin remote

Santiago Pastorino (Oct 29 2019 at 15:25, on Zulip):

yeah

Last update: Nov 20 2019 at 01:10UTC