Stream: t-compiler/wg-parallel-rustc

Topic: plan to test parallel rustc


Alex Crichton (Dec 16 2019 at 21:09, on Zulip):

I've posted this online now -- https://internals.rust-lang.org/t/plan-to-test-parallel-rustc/11487

Alex Crichton (Dec 16 2019 at 21:09, on Zulip):

we can discuss feedback here if there is any! (or on thread if it's more appropriate)

Santiago Pastorino (Dec 16 2019 at 21:21, on Zulip):

one thing that I'm thinking of are potential clippy, miri and things like that breakages that if you have those and try to do rustup update you may not escape for a while from the parallel version that was built

Santiago Pastorino (Dec 16 2019 at 21:22, on Zulip):

I think that's ok anyway ...

nikomatsakis (Dec 16 2019 at 21:22, on Zulip):

I guess plausibly downstream toolstate might die pretty badly

nikomatsakis (Dec 16 2019 at 21:22, on Zulip):

though I'm not sure how

nikomatsakis (Dec 16 2019 at 21:22, on Zulip):

not sure if we can test that in advance? i.e., if clippy does something not thread safe?

nikomatsakis (Dec 16 2019 at 21:23, on Zulip):

(do standard PR builds or try builds test tools?)

Santiago Pastorino (Dec 16 2019 at 21:24, on Zulip):

just in case, I was referring to the time window between the nightly with parallel enabled goes out and the next nightly version

Santiago Pastorino (Dec 16 2019 at 21:24, on Zulip):

if between those a change that breaks miri, clippy or any of the other components lands

Santiago Pastorino (Dec 16 2019 at 21:24, on Zulip):

you may not escape the parallel nightly for a while unless you do something

Santiago Pastorino (Dec 16 2019 at 21:26, on Zulip):

but just a rustup update won't allow you to update because next versions will have a broken component

simulacrum (Dec 16 2019 at 21:30, on Zulip):

the advantage to doing it this week is that all tools must build (modulo miri)

simulacrum (Dec 16 2019 at 21:30, on Zulip):

since it's beta week

Santiago Pastorino (Dec 16 2019 at 21:31, on Zulip):

:+1:

Zoxc (Dec 17 2019 at 07:19, on Zulip):

Have you checked that parallel compiler passes tests? I know there was one test where the single thread compiler outputs an extra cycle error.

simulacrum (Dec 17 2019 at 12:33, on Zulip):

hm, we're seeing the opposite

simulacrum (Dec 17 2019 at 12:34, on Zulip):

i.e., one less cycle failure with parallel

Zoxc (Dec 17 2019 at 12:38, on Zulip):

That sounds like the same thing =P

simulacrum (Dec 17 2019 at 12:49, on Zulip):

oh I completely missed the "single threaded" bit

nikomatsakis (Dec 17 2019 at 19:55, on Zulip):

Seems like a concern :)

Alex Crichton (Dec 17 2019 at 21:50, on Zulip):

parallelism was enabled in https://github.com/rust-lang/rust/pull/67362 and is being disabled in https://github.com/rust-lang/rust/pull/67379

Alex Crichton (Dec 17 2019 at 21:50, on Zulip):

@simulacrum I assume you've got a method in mind of how to land that while making sure the previous PR makes its way into nightlie?

simulacrum (Dec 17 2019 at 21:51, on Zulip):

we're less than 4 hours out from nightly "fork" and the rollup won't start testing for at least an hour or two, so it's pretty much guaranteed that everything sync up appropriately

Zoxc (Dec 17 2019 at 23:26, on Zulip):

@nikomatsakis It could just be that the parallel compiler chooses to break the cycle in a location it can't be recovered from. Recovering from cycles is really questionable in general and it's probably not something we should do.

nikomatsakis (Dec 18 2019 at 15:37, on Zulip):

anyway I don't think it has to block doing some experiments w/ nightly

nikomatsakis (Dec 18 2019 at 15:37, on Zulip):

but we should file an issue to investigate or something at least?

simulacrum (Dec 18 2019 at 15:47, on Zulip):

yeah I think an issue would be good

Alex Crichton (Dec 18 2019 at 16:44, on Zulip):

I've posted a dedicated thread -- https://internals.rust-lang.org/t/help-test-parallel-rustc/11503 -- for gathing measurements now

Alex Crichton (Dec 18 2019 at 16:44, on Zulip):

@simulacrum I see the revert already landed, thanks for managing that!

simulacrum (Dec 18 2019 at 16:45, on Zulip):

Yep tonight's nightly should be back to normal

Santiago Pastorino (Dec 18 2019 at 18:51, on Zulip):

I was wondering what rayon did we ship the parallel version with

Santiago Pastorino (Dec 18 2019 at 18:51, on Zulip):

it seems like it was master's rust-lang/rustc-rayon, right?

Santiago Pastorino (Dec 18 2019 at 18:51, on Zulip):

didn't we want to use Niko's fork?

simulacrum (Dec 18 2019 at 19:01, on Zulip):

we intentionally decided not to I think

simulacrum (Dec 18 2019 at 19:02, on Zulip):

since it's pretty alpha-level

Santiago Pastorino (Dec 18 2019 at 19:38, on Zulip):

:+1:

Last update: Jul 02 2020 at 19:50UTC