Stream: t-compiler/wg-parallel-rustc

Topic: will we meet today?


nikomatsakis (Nov 04 2019 at 19:04, on Zulip):

I think I would rather not meet today, though I have no objection to y'all meeting -- I've just got some catch-up work to do

nikomatsakis (Nov 04 2019 at 19:04, on Zulip):

Well, let me phrase that a bit more open-ended

nikomatsakis (Nov 04 2019 at 19:04, on Zulip):

Will we meet today? Do we have things to discuss?

nikomatsakis (Nov 04 2019 at 19:04, on Zulip):

(cc @WG-parallel-rustc)

simulacrum (Nov 04 2019 at 19:05, on Zulip):

I think if we did it would be to go over the doc that @Alex Crichton prepared

simulacrum (Nov 04 2019 at 19:05, on Zulip):

I don't think that needs to be done synchronously though

Santiago Pastorino (Nov 04 2019 at 19:06, on Zulip):

on my part I went over CrateMetadata fields, @Vadim Petrochenkov also did some work

Santiago Pastorino (Nov 04 2019 at 19:06, on Zulip):

unsure if someone saw all that and have thoughts

simulacrum (Nov 04 2019 at 19:06, on Zulip):

https://hackmd.io/oUdvUU2lTk2ZxfOWQBT9xQ?both is that doc

simulacrum (Nov 04 2019 at 19:07, on Zulip):

I don't think I have any sync-needing thoughts though, so I'm fine skipping a week

Paul Faria (Nov 04 2019 at 19:07, on Zulip):

I only have to get another review from @oli on my PR (extracting interior mutability from Body.cache)

simulacrum (Nov 04 2019 at 19:07, on Zulip):

I've been working on some of the Session stuff, putting up (smallish) PRs as I go

simulacrum (Nov 04 2019 at 19:08, on Zulip):

but again don't feel that we need a meeting for that to unblock me

Alex Crichton (Nov 04 2019 at 19:08, on Zulip):

I don't mind going either way

simulacrum (Nov 04 2019 at 19:09, on Zulip):

sounds like we don't have any particular need for a sync meeting, so I think let's cancel

simulacrum (Nov 11 2019 at 13:48, on Zulip):

@nikomatsakis Are we meeting today? I don't have anything in particular for agenda, I think...

My work on Session has mostly stalled out -- everything left is mostly either fine or just a pre-TyCtxt query (Once), which is also fine

Santiago Pastorino (Nov 11 2019 at 13:51, on Zulip):

I was wondering the same if we are having the meeting today

Santiago Pastorino (Nov 11 2019 at 13:52, on Zulip):

also @simulacrum If I can be of help on something let me know

Alex Crichton (Nov 11 2019 at 14:14, on Zulip):

I unfortunately won't be able to make it if we meet

nikomatsakis (Nov 11 2019 at 15:21, on Zulip):

I am happy to skip this week. It seems like we're approaching a point where the "shared state fears" are pretty minimal

nikomatsakis (Nov 11 2019 at 15:21, on Zulip):

And it's mostly a perf question at this point?

simulacrum (Nov 11 2019 at 16:18, on Zulip):

Indeed, that's my impression -- let's skip then, I don't think a sync meeting is super helpful since we have nothing in particular to discuss

simulacrum (Nov 11 2019 at 16:19, on Zulip):

We probably should figure out a next step for perf work -- I'm unclear if we have any known culprits at this point

Alex Crichton (Nov 11 2019 at 17:02, on Zulip):

I'm under the impression at least that the main culprit is the investigation I did a week or so ago?

Alex Crichton (Nov 11 2019 at 17:03, on Zulip):

the doc about spinning up threads and all time going into the system?

Alex Crichton (Nov 11 2019 at 17:03, on Zulip):

basically the summary at the top of https://hackmd.io/oUdvUU2lTk2ZxfOWQBT9xQ?view#Alex-Profiles-Cargo-and--j28

simulacrum (Nov 11 2019 at 18:30, on Zulip):

hm, I seemed to recall the conclusion from that was "this is not great, but not a major contributor"

simulacrum (Nov 11 2019 at 18:30, on Zulip):

and in any case that we're still not seeing amazing gains that we were hoping for on the single-crate case, where it matters a little more in some sense

simulacrum (Nov 18 2019 at 16:18, on Zulip):

@WG-parallel-rustc Do we want to meet today? I don't know that we have anything to discuss

Santiago Pastorino (Nov 18 2019 at 16:19, on Zulip):

I don't have anything to discuss

Santiago Pastorino (Nov 18 2019 at 16:22, on Zulip):

still at some point (when everyone considers so) would be nice to get together to keep shaping ideas to pursue maybe?

nikomatsakis (Nov 18 2019 at 16:30, on Zulip):

I too could use the time this week but

nikomatsakis (Nov 18 2019 at 16:30, on Zulip):

I do want to sort of unstall

nikomatsakis (Nov 18 2019 at 16:31, on Zulip):

I think though that mostly we need to try out new rayon scheduler?

Santiago Pastorino (Nov 18 2019 at 16:33, on Zulip):

so as we were talking in private with Niko

Santiago Pastorino (Nov 18 2019 at 16:34, on Zulip):

it turned out complicated to me to run perf (it segfaults, it happens to other Archlinux users), but I can work now on getting rustc-rayon use the new scheduler

Santiago Pastorino (Nov 18 2019 at 16:34, on Zulip):

then ask someone to run perf

Paul Faria (Nov 18 2019 at 16:37, on Zulip):

I might be able to run perf. I've got Ubuntu and pop-os machines. Not going to make the meeting today though.

Alex Crichton (Nov 18 2019 at 16:47, on Zulip):

I'm also worried about perpetually stalling meetings too long which could stymie progress

Alex Crichton (Nov 18 2019 at 16:48, on Zulip):

I also get the feeling though that the work items were generated by me and nothing else has really been generated since then

Alex Crichton (Nov 18 2019 at 16:48, on Zulip):

and I'm only sort of tangentially helping out?

simulacrum (Nov 18 2019 at 16:53, on Zulip):

I think we're semi blocked on the rayon work

simulacrum (Nov 18 2019 at 16:54, on Zulip):

Like, I don't know that we have any other major thoughts

simulacrum (Nov 18 2019 at 16:55, on Zulip):

I do agree that we're somewhat stalled but I think it's also somewhat unavoidable

Santiago Pastorino (Nov 18 2019 at 17:33, on Zulip):

I'm pretty sure that I can do the rayon stuff today

Santiago Pastorino (Nov 18 2019 at 17:33, on Zulip):

jumping into it right now

simulacrum (Nov 18 2019 at 17:39, on Zulip):

@Santiago Pastorino Presumably though we'd not have data by the meeting though?

simulacrum (Nov 18 2019 at 17:40, on Zulip):

I think it may make sense to meet though to try and work out a plan for the next steps

simulacrum (Nov 18 2019 at 17:40, on Zulip):

e.g. are we intending to keep trying to push for better performance until we have some x% win, or do we have other goals in mind

Santiago Pastorino (Nov 18 2019 at 17:41, on Zulip):

unsure, let me see, may be able to handle a working thing to @Alex Crichton so we can re-run perf

Santiago Pastorino (Nov 18 2019 at 17:41, on Zulip):

will comment here ASAP

Santiago Pastorino (Nov 18 2019 at 17:51, on Zulip):
test iter::test::check_partial_cmp_to_seq ... ok
test iter::test::check_slice_indexed ... ok
thread '<unnamed>' panicked at 'attempt to subtract with overflow', rayon-core/src/sleep/mod.rs:346:17
test iter::test::check_slice_mut_indexed ... ok
Rayon: detected unexpected panic; aborting
error: process didn't exit successfully: `/home/santiago/src/oss/rayon/target/debug/deps/rayon-d8f3641836783ef4` (signal: 6, SIGABRT: process abort signal)
simulacrum (Nov 18 2019 at 17:52, on Zulip):

(I don't think it's necessarily urgent to have the data, to be clear)

simulacrum (Nov 18 2019 at 17:52, on Zulip):

In the sense that I don't think our meeting would significantly change with it

Santiago Pastorino (Nov 18 2019 at 17:52, on Zulip):

@nikomatsakis trying to run tests on your rayon fork, master works ok, latch-target-thread shows that problem

Santiago Pastorino (Nov 18 2019 at 17:53, on Zulip):

In the sense that I don't think our meeting would significantly change with it

:+1:, I'm just going over this

Santiago Pastorino (Nov 18 2019 at 17:54, on Zulip):

nikomatsakis trying to run tests on your rayon fork, master works ok, latch-target-thread shows that problem

were tests working for you?

Santiago Pastorino (Nov 18 2019 at 18:03, on Zulip):

anyway, fixed the issue

nikomatsakis (Nov 18 2019 at 18:22, on Zulip):

one other data point: I'm on PTO next week :)

nikomatsakis (Nov 18 2019 at 18:23, on Zulip):

that said, I don't think we have to meet necessarily, but I do want to be making progress

nikomatsakis (Nov 18 2019 at 18:23, on Zulip):

I'm happy if @Santiago Pastorino is working on the rebase

nikomatsakis (Nov 18 2019 at 18:23, on Zulip):

(and i'll chat with them a bit in the other topic about it)

nikomatsakis (Nov 18 2019 at 18:23, on Zulip):

there is also @Zoxc's comment about their other PRs

nikomatsakis (Nov 18 2019 at 18:23, on Zulip):

which could be further work items

simulacrum (Nov 18 2019 at 19:12, on Zulip):

Do we have any agenda? i.e., at least one item to discuss?

simulacrum (Nov 18 2019 at 19:13, on Zulip):

if not then I guess we defacto don't meet

simulacrum (Nov 18 2019 at 19:13, on Zulip):

but otherwise let's meet

Alex Crichton (Nov 18 2019 at 19:53, on Zulip):

hm so the conclusion is we're not meeting today, right?

Santiago Pastorino (Nov 18 2019 at 19:54, on Zulip):

hm so the conclusion is we're not meeting today, right?

seems like we will meet to go over the rebasing of rayon, but I guess that can only be @nikomatsakis helping me a bit?

Santiago Pastorino (Nov 18 2019 at 19:54, on Zulip):

@nikomatsakis please confirm your idea

Santiago Pastorino (Nov 18 2019 at 19:55, on Zulip):

that would be in 6 minutes, right?

nikomatsakis (Nov 18 2019 at 19:55, on Zulip):

yes

nikomatsakis (Nov 18 2019 at 19:55, on Zulip):

I'd say anyone who wants to join can

nikomatsakis (Nov 18 2019 at 19:55, on Zulip):

but it's more of a "hack session" today?

nikomatsakis (Nov 18 2019 at 19:55, on Zulip):

(sorry, finishing up something else and hence laggy)

Alex Crichton (Nov 18 2019 at 19:57, on Zulip):

ok I think I will sit out then, but if you need me just holler

Alex Crichton (Nov 18 2019 at 19:57, on Zulip):

(I don't think I'd be too useful anyway for rebasing rayon)

nikomatsakis (Nov 18 2019 at 20:04, on Zulip):

agreed

nikomatsakis (Nov 18 2019 at 20:04, on Zulip):

@Santiago Pastorino ok -- around?

nikomatsakis (Nov 18 2019 at 20:04, on Zulip):

should we use zoom for chat?

Santiago Pastorino (Nov 18 2019 at 20:04, on Zulip):

yes, I'm ... ahh I was already on zoom

nikomatsakis (Nov 18 2019 at 20:04, on Zulip):

ah right the calendar invite has one

Santiago Pastorino (Nov 18 2019 at 20:05, on Zulip):

but whatever you prefer is fine

nikomatsakis (Nov 18 2019 at 20:05, on Zulip):

omw

Santiago Pastorino (Nov 18 2019 at 21:00, on Zulip):
    Updating git repository `https://github.com/spastorino/rustc-rayon`
    Updating crates.io index
error: failed to select a version for `crossbeam-channel`.
    ... required by package `rustc-rayon-core v0.3.0 (https://github.com/spastorino/rustc-rayon?branch=latch-target-thread-rustc#a8dc7ab6)`
    ... which is depended on by `rustc v0.0.0 (/home/santiago/src/oss/rust1/src/librustc)`
    ... which is depended on by `rustc_codegen_ssa v0.0.0 (/home/santiago/src/oss/rust1/src/librustc_codegen_ssa)`
    ... which is depended on by `rustc-main v0.0.0 (/home/santiago/src/oss/rust1/src/rustc)`
versions that meet the requirements `^0.3.9` are: 0.3.9

all possible versions conflict with previously selected packages.

  previously selected package `crossbeam-channel v0.3.8`
    ... which is depended on by `rls v1.40.0 (/home/santiago/src/oss/rust1/src/tools/rls)`

failed to select a version for `crossbeam-channel` which could resolve this conflict
failed to run: /home/santiago/src/oss/rust1/build/x86_64-unknown-linux-gnu/stage0/bin/cargo build --manifest-path /home/santiago/src/oss/rust1/src/bootstrap/Cargo.toml
Build completed unsuccessfully in 0:00:05
Santiago Pastorino (Nov 18 2019 at 21:00, on Zulip):

I wonder why this happens

Santiago Pastorino (Nov 18 2019 at 21:01, on Zulip):

rls explicitly states 0.3.9

Santiago Pastorino (Nov 18 2019 at 21:03, on Zulip):

ahh I see, the error is just weird

Santiago Pastorino (Nov 25 2019 at 16:07, on Zulip):

will we meet today?

Santiago Pastorino (Nov 25 2019 at 16:09, on Zulip):

I think it may be a good idea to try to discuss and get actionables out of the perf numbers we have gathered

simulacrum (Nov 25 2019 at 16:23, on Zulip):

yeah, I've been working on trying to identify causes

simulacrum (Nov 25 2019 at 16:23, on Zulip):

https://hackmd.io/CBdNMYStS_GH2MawrgARfw

simulacrum (Nov 25 2019 at 16:23, on Zulip):

leaving notes there

simulacrum (Nov 25 2019 at 18:07, on Zulip):

otoh I believe Niko is not here this week

simulacrum (Nov 25 2019 at 18:07, on Zulip):

@Alex Crichton will you be able to make it today?

Alex Crichton (Nov 25 2019 at 18:17, on Zulip):

Yes

Santiago Pastorino (Nov 25 2019 at 18:33, on Zulip):

yeah, I'm just remembering that Niko is on vacations this full week

Santiago Pastorino (Nov 25 2019 at 18:34, on Zulip):

unsure if we will do Zoom or just here Zulip but if we do Zoom would be nice to record

simulacrum (Nov 25 2019 at 18:45, on Zulip):

Zoom should be fine

simulacrum (Nov 25 2019 at 20:01, on Zulip):

https://hackmd.io/CBdNMYStS_GH2MawrgARfw

Alex Crichton (Nov 25 2019 at 20:03, on Zulip):

https://mozilla.zoom.us/j/2744183693

Alex Crichton (Nov 25 2019 at 20:03, on Zulip):

@simulacrum @Santiago Pastorino ^

simulacrum (Nov 25 2019 at 20:20, on Zulip):

https://github.com/Mark-Simulacrum/jobserver-rs/commit/fb759778df3b1d4524ee1116b975137002668b22

Alex Crichton (Nov 25 2019 at 20:55, on Zulip):

ok recording says it's being processed

Alex Crichton (Nov 25 2019 at 20:56, on Zulip):

@simulacrum that seems reaosnable to me at least

Alex Crichton (Nov 25 2019 at 20:56, on Zulip):

just never used ipc semaphores myself heh

simulacrum (Nov 25 2019 at 20:56, on Zulip):

first time for me too :)

Alex Crichton (Nov 25 2019 at 22:50, on Zulip):

https://mozilla.zoom.us/recording/share/VLce4oHRtki9v9jGXPh1oSxdM7FfU6vvfTKDDFeoZhWwIumekTziMw

Alex Crichton (Nov 25 2019 at 22:50, on Zulip):

ok recording is up ^

Santiago Pastorino (Nov 26 2019 at 13:23, on Zulip):

:+1:

Santiago Pastorino (Nov 26 2019 at 13:25, on Zulip):

transcripts are funny

Santiago Pastorino (Nov 26 2019 at 13:25, on Zulip):

"NASA data so well actually I thought"

Santiago Pastorino (Nov 26 2019 at 13:26, on Zulip):

"This is did you, this is about the shirting of the locks on the x ray"

Zoxc (Nov 26 2019 at 13:33, on Zulip):

It was also from the wrong person at times, which is a bit surprising. I guess it ran on the combined audio?

Alex Crichton (Nov 26 2019 at 14:23, on Zulip):

Yes

Last update: Mar 30 2020 at 23:55UTC