Stream: t-compiler

Topic: planning meeting 2019.05.10


nikomatsakis (May 10 2019 at 14:00, on Zulip):

Dear @T-compiler/meeting -- there will be a planning meeting shortly! This is our first time trying such a meeting. The idea is to decide what meetings (if any) to schedule for the next 3 weeks:

It is ok not to schedule anything. We will be selecting from the list of proposed meetings, of which presently there are only two. (But I'm planning to submit one more that I forgot about -- almost done.)

nikomatsakis (May 10 2019 at 14:00, on Zulip):

To begin with, and to let people show up, let's have an announcements period for the next 5-10 minutes =)

nikomatsakis (May 10 2019 at 14:00, on Zulip):

# Announcements

nikomatsakis (May 10 2019 at 14:05, on Zulip):

Anybody around? =)

nikomatsakis (May 10 2019 at 14:06, on Zulip):

If not, maybe we'll postpone this for another time =)

nikomatsakis (May 10 2019 at 14:06, on Zulip):

It occurs to me that we should formalize the :wave: emoji thing for registering presence =)

mw (May 10 2019 at 14:07, on Zulip):

all three topics seem worth discussing in some form

mw (May 10 2019 at 14:08, on Zulip):

and they all seem complex enough to warrant their own meeting, I'd say

nikomatsakis (May 10 2019 at 14:09, on Zulip):

Let me ping @eddyb on discord

nikomatsakis (May 10 2019 at 14:09, on Zulip):

I know @pnkfelix is not feeling well

nikomatsakis (May 10 2019 at 14:09, on Zulip):

Not sure about other folks

mw (May 10 2019 at 14:09, on Zulip):

although I'm not so sure what the discussion around parallel rustc would look like?

nikomatsakis (May 10 2019 at 14:09, on Zulip):

Well, shall we get started?

nikomatsakis (May 10 2019 at 14:09, on Zulip):

We can discuss

mw (May 10 2019 at 14:09, on Zulip):

mostly raising awareness of the plan and getting buy-in for flipping the switch?

nikomatsakis (May 10 2019 at 14:09, on Zulip):

I don't know if it's the right venue for that exactly

nikomatsakis (May 10 2019 at 14:10, on Zulip):

mostly raising awareness of the plan and getting buy-in for flipping the switch?

yes, but I was also imagining trying to decide on the acceptable perf loss, as well as what kind of doc etc we felt we needed, and maybe how to get it

nikomatsakis (May 10 2019 at 14:11, on Zulip):

Well, the three proposals are as follows:

nikomatsakis (May 10 2019 at 14:11, on Zulip):
nikomatsakis (May 10 2019 at 14:11, on Zulip):
nikomatsakis (May 10 2019 at 14:11, on Zulip):
nikomatsakis (May 10 2019 at 14:12, on Zulip):

I would classify the middle one as "semi-technical" , the others... kind of either way. that was wrong :)

mw (May 10 2019 at 14:12, on Zulip):

formatting seems the least urgent to me

nikomatsakis (May 10 2019 at 14:13, on Zulip):

Yeah, I'm not entirely sure if we need a meeting on it to be honest

nikomatsakis (May 10 2019 at 14:13, on Zulip):

The plan seems pretty fleshed out and I don't know if there's a lot of controversy to discuss or anything

nikomatsakis (May 10 2019 at 14:13, on Zulip):

I could imagine just doing an fcp merge

nikomatsakis (May 10 2019 at 14:13, on Zulip):

In any case, I agree that the other two are more urgent

nikomatsakis (May 10 2019 at 14:14, on Zulip):

Maybe we can talk more about what we might hope to get out of this?

mw (May 10 2019 at 14:14, on Zulip):

(cc @lwshang )

Zoxc (May 10 2019 at 14:15, on Zulip):

I just think parallel rustc just has to be equal or faster for most uses cases on 2 core systems (since people don't use 1 core systems anymore)

nikomatsakis (May 10 2019 at 14:16, on Zulip):

Hmm

mw (May 10 2019 at 14:16, on Zulip):

well, in practice we have perf.rlo for deciding

nikomatsakis (May 10 2019 at 14:16, on Zulip):

I think that's a good criteria

nikomatsakis (May 10 2019 at 14:16, on Zulip):

But I think we should also consider the sequential overhead since there are also cases where cores would be saturated by having multiple crates

mw (May 10 2019 at 14:17, on Zulip):

I think we should use perf.rlo for testing

nikomatsakis (May 10 2019 at 14:17, on Zulip):

But I know we are considering it, in any case :)

mw (May 10 2019 at 14:17, on Zulip):

we can set that to two threads

Zoxc (May 10 2019 at 14:17, on Zulip):

perf.rlo doesn't test crate graphs, which are important

nikomatsakis (May 10 2019 at 14:17, on Zulip):

I was wondering about the concerns that @nnethercote raised

mw (May 10 2019 at 14:17, on Zulip):

I thought it did

nikomatsakis (May 10 2019 at 14:18, on Zulip):

I guess perf also measures wallclock

nikomatsakis (May 10 2019 at 14:18, on Zulip):

Well, in any case, I dont' thnk we should have the meeting now :)

mw (May 10 2019 at 14:18, on Zulip):

(cc @simulacrum ad crate graphs)

nikomatsakis (May 10 2019 at 14:18, on Zulip):

Let me ask this question: if we did have such a meeting, who are people that should attend it

nikomatsakis (May 10 2019 at 14:18, on Zulip):

It might not be possible to get everyone but it'd be nice to try and pick a date that works for them

mw (May 10 2019 at 14:18, on Zulip):

any potential maintainers of the parallel code

nikomatsakis (May 10 2019 at 14:19, on Zulip):

Apart from performance thresholds, the other thing I hoped to discuss was whether we were prepared for maintenance

nikomatsakis (May 10 2019 at 14:19, on Zulip):

By which I mostly mean: how much documentation do we have/need on how the threading system works etc

nikomatsakis (May 10 2019 at 14:20, on Zulip):

I noticed on a recent PR that @eddyb said something like "I don't understand how the threading system works tbh" and cc'd others

nikomatsakis (May 10 2019 at 14:20, on Zulip):

I feel like this is a bit of a red flag to me ;)

nikomatsakis (May 10 2019 at 14:20, on Zulip):

(But maybe we don't want to block deployment on that)

nikomatsakis (May 10 2019 at 14:20, on Zulip):

Well there I go again

nikomatsakis (May 10 2019 at 14:20, on Zulip):

Having the meeting

nikomatsakis (May 10 2019 at 14:21, on Zulip):

My impression is that the amount of discussion already suggests we have enough to talk about :)

mw (May 10 2019 at 14:21, on Zulip):

the next cycle starts soonish

mw (May 10 2019 at 14:21, on Zulip):

would be good if we discussed this before

nikomatsakis (May 10 2019 at 14:21, on Zulip):

(One caveat: I personally will be able away May 24)

nikomatsakis (May 10 2019 at 14:22, on Zulip):

OK, so who might be nice to attend? I think myself, @eddyb, @mw, @Zoxc, and it sounds like maybe @simulacrum because perf.

It'd be nice to get @lwshang or @nnethercote but time zones may interfere

nikomatsakis (May 10 2019 at 14:23, on Zulip):

the next cycle starts soonish

May 23rd 2019

nikomatsakis (May 10 2019 at 14:24, on Zulip):

if we want to catch that train, we have to act very fast

Wesley Wiser (May 10 2019 at 14:24, on Zulip):

(eddyb mentioned last week that they'd be unable to attend this meeting)

mw (May 10 2019 at 14:24, on Zulip):

OK, should we do next week then?

nikomatsakis (May 10 2019 at 14:24, on Zulip):

Sounds like it? Is that soon enough?

Zoxc (May 10 2019 at 14:24, on Zulip):

We kind of have a plan in place, do we need to revise it quickly?

mw (May 10 2019 at 14:25, on Zulip):

we should make sure we have any data we need before the meeting, yes

nikomatsakis (May 10 2019 at 14:25, on Zulip):

Definitely. I guess we might aim to make a "go/no-go" decision

mw (May 10 2019 at 14:26, on Zulip):

i.e. should we try to do some perf runs before the meeting?

simulacrum (May 10 2019 at 14:26, on Zulip):

I can try to help with some crate-graph impls for perf, I don't think it'd be too hard to make that happen if I invested time in it

nikomatsakis (May 10 2019 at 14:27, on Zulip):

sounds like want to know:

Anything else?

Wesley Wiser (May 10 2019 at 14:27, on Zulip):

I'm personally curious how much of an impact this will have on bootstrap time on a many core machine

mw (May 10 2019 at 14:27, on Zulip):

I thought perf.rlo would measure entire cargo build invocations :(

nikomatsakis (May 10 2019 at 14:28, on Zulip):

I'm personally curious how much of an impact this will have on bootstrap time on a many core machine

can we measure this too? Not sure what you mean by "many core", but e.g. I have a 14-core desktop

simulacrum (May 10 2019 at 14:28, on Zulip):

@mw currently we just benchmark rustc but I suspect it's fairly easy to move where perf stat sits in that process

Wesley Wiser (May 10 2019 at 14:28, on Zulip):

8+ core

simulacrum (May 10 2019 at 14:28, on Zulip):

We could maybe try 1, 2, 4, 8 threads on perf server? To get relative speedup per thread (i.e., how much we do in parallel, to an extent)

nikomatsakis (May 10 2019 at 14:29, on Zulip):

Seems good

nikomatsakis (May 10 2019 at 14:29, on Zulip):

If we are able to do all of that, yeah

nikomatsakis (May 10 2019 at 14:29, on Zulip):

The more data the better

nikomatsakis (May 10 2019 at 14:29, on Zulip):

in particular, if we're going to announce this, we're going to want to be able to show benefits

nikomatsakis (May 10 2019 at 14:29, on Zulip):

OK, so can we get a comprehensive list of the things we're aiming to do data-wise?

nikomatsakis (May 10 2019 at 14:29, on Zulip):

We could append it to the issue

simulacrum (May 10 2019 at 14:29, on Zulip):

perf server has 8 threads (4 cores, I guess?) so we might not get great results there

nikomatsakis (May 10 2019 at 14:30, on Zulip):

(I could also try running it on my desktop if you have a script I can run)

mw (May 10 2019 at 14:30, on Zulip):

that's interesting data too, I think

nikomatsakis (May 10 2019 at 14:30, on Zulip):

It's mostly idle these days :'(

nikomatsakis (May 10 2019 at 14:31, on Zulip):

(but I think perf server is most imp't, and 4 cores seems kind of "representative" of what most folks have)

simulacrum (May 10 2019 at 14:31, on Zulip):
nikomatsakis (May 10 2019 at 14:32, on Zulip):

full crate graph measurements? (definitely targeted cases: e.g., rustc bootstrap, servo)

I would add cranelift to the list

mw (May 10 2019 at 14:32, on Zulip):

something like ripgrep might also be an interesting case

mw (May 10 2019 at 14:32, on Zulip):

@Zoxc we have to make sure that we measure the right thing here

mw (May 10 2019 at 14:33, on Zulip):

i.e. the right combination of flags for cargo and rustc

Zoxc (May 10 2019 at 14:33, on Zulip):

Lots of cores (14+) might be slower as there is quite a few global locks too. Let's get eddyb to run some tests with 24 cores =P

mw (May 10 2019 at 14:33, on Zulip):

whoever does benchmarking should talk to @Zoxc first about which flags to use exactly

simulacrum (May 10 2019 at 14:34, on Zulip):

So -- to be clear -- if we want these, someone (probably me?) needs to invest I suspect 2-3 hours to get this setup; getting flags for cargo and rustc from @Zoxc is a prerequisite

nikomatsakis (May 10 2019 at 14:34, on Zulip):

I'm starting a hackmd doc to collect benchmark stuff

Zoxc (May 10 2019 at 14:34, on Zulip):

Also there's 2 other items to talk about =P

simulacrum (May 10 2019 at 14:34, on Zulip):

(and probably a current parallel compiler try build)

nikomatsakis (May 10 2019 at 14:34, on Zulip):

Also there's 2 other items to talk about =P

I was going to bring that up :) but I thiink this is the most pressing; it seems like we're almost done tho

nikomatsakis (May 10 2019 at 14:36, on Zulip):

To what extent @simulacrum do you think we can have a "script" that we just run

nikomatsakis (May 10 2019 at 14:36, on Zulip):

That would make it much easier to do that on e.g. my desktop and/or eddyb's machine

simulacrum (May 10 2019 at 14:36, on Zulip):

I would say 95% that should be feasible

nikomatsakis (May 10 2019 at 14:37, on Zulip):

ok

simulacrum (May 10 2019 at 14:37, on Zulip):

(but all of this is blocked on try compiler build and @Zoxc providing flags, to an extent)

nikomatsakis (May 10 2019 at 14:37, on Zulip):

OK

nikomatsakis (May 10 2019 at 14:37, on Zulip):

Let's get those going ASAP

nikomatsakis (May 10 2019 at 14:37, on Zulip):

Seems like we're done with this topic then?

nikomatsakis (May 10 2019 at 14:37, on Zulip):

We'll schedule for next friday?

mw (May 10 2019 at 14:37, on Zulip):

would work for me

simulacrum (May 10 2019 at 14:37, on Zulip):

should work for me, yes

nikomatsakis (May 10 2019 at 14:38, on Zulip):

@Zoxc ?

Zoxc (May 10 2019 at 14:39, on Zulip):

Sure

nikomatsakis (May 10 2019 at 14:40, on Zulip):

OK

nikomatsakis (May 10 2019 at 14:40, on Zulip):

Take a look at this github comment if you like, I tried to summarize very briefly =)

nikomatsakis (May 10 2019 at 14:41, on Zulip):

So there were 2 other proposals, formatting + maintenance.

I won't be here May 24 but we could possibly discuss formatting then if somebody else wants to lead. I also think plausibly we don't need a meeting for that.

We could discuss maintenance on May 31, the non-technical day.

nikomatsakis (May 10 2019 at 14:41, on Zulip):

Or we could do formatting on May 31. My sense is that it's a bit less urgent.

nikomatsakis (May 10 2019 at 14:42, on Zulip):

That said, I was...pretty vague about "maintenance/triage"...

nikomatsakis (May 10 2019 at 14:42, on Zulip):

i.e., I dashed it out in a few minutes ;)

nikomatsakis (May 10 2019 at 14:42, on Zulip):

we could just do nothing and try to flesh out a plan independently

nikomatsakis (May 10 2019 at 14:43, on Zulip):

I also think plausibly we don't need a meeting for that.

can we discuss this? Do we think we need a meeting around https://github.com/rust-lang/compiler-team/issues/80?

nikomatsakis (May 10 2019 at 14:43, on Zulip):

I guess it's a fairly complex plan; my main concern is whether we have folks who want to do the work it implies (bots etc), but that's probably not a t-compiler thing in the end

mw (May 10 2019 at 14:44, on Zulip):

it sounds like t-infra

davidtwco (May 10 2019 at 14:44, on Zulip):

I'm happy to spend some time on stuff for that if we really want it and it's (understandably) low on the list of priorities for T-infra.

nikomatsakis (May 10 2019 at 14:45, on Zulip):

I'm pretty excited about seeing this happen, to be clear.

nikomatsakis (May 10 2019 at 14:45, on Zulip):

I think it'll be a big (if subtle) improvement for contributors

Wesley Wiser (May 10 2019 at 14:45, on Zulip):

There may also be interested parties from T-libs since we share the rust-lang/rust repo with them as well

nikomatsakis (May 10 2019 at 14:45, on Zulip):

But also us :) (I love hacking with "Rustfmt-on-save")

mw (May 10 2019 at 14:45, on Zulip):

I think it would be "nice to have" :)

simulacrum (May 10 2019 at 14:46, on Zulip):

It might be good to have a list of blockers (i.e., people) -- if there are any, so we can build consensus before implementation

simulacrum (May 10 2019 at 14:46, on Zulip):

but I don't personally think this needs a meeting

Zoxc (May 10 2019 at 14:46, on Zulip):

I just want a tool to preformat a branch before rebasing on to a rustfmt-ed master before doing it

nikomatsakis (May 10 2019 at 14:47, on Zulip):

but I don't personally think this needs a meeting

I think we should not do a meeting, but we should try to advertise the plan as much in advance. I think there's already been a fair amount of discussion and most of the "rustfmt skeptics" I know of have given approval (notably @Vadim Petrochenkov).

I agree with @Zoxc that having a tool to help out with branches would be a win, not sure if that was in the RFC or how hard it is.

simulacrum (May 10 2019 at 14:47, on Zulip):

I believe that's planned or semi-planned

nikomatsakis (May 10 2019 at 14:48, on Zulip):

we could just "fcp merge" on the meeting proposal ;)

nikomatsakis (May 10 2019 at 14:48, on Zulip):

but it feels like -- let's just open the damn RFC instead, if we're going to do that

nikomatsakis (May 10 2019 at 14:48, on Zulip):

I guess I will fcp poll

nikomatsakis (May 10 2019 at 14:48, on Zulip):

all right, we decided not to do this as a meeting (feel free to object, but I'm not hearing any), so that leaves the "maintenance" thing

mw (May 10 2019 at 14:49, on Zulip):

re: maintenance - I'd love to have more structured support for bisecting regressions

nikomatsakis (May 10 2019 at 14:50, on Zulip):

I see a few options:

mw (May 10 2019 at 14:51, on Zulip):

May 31 would be a non-technical meeting anyway, right? seems like a good topic

nikomatsakis (May 10 2019 at 14:51, on Zulip):

correct

nikomatsakis (May 10 2019 at 14:51, on Zulip):

ok, let's schedule it

mw (May 10 2019 at 14:51, on Zulip):

I'm not entirely sure what we'd discuss concretely though :)

simulacrum (May 10 2019 at 14:52, on Zulip):

I would personally not be able to attend a meeting on the 31st (traveling) but I don't think that's necessarily a blocker -- triage-related discussion is a bit nebulous in terms of stakeholders :)

nikomatsakis (May 10 2019 at 14:52, on Zulip):

I don't see you as a blocker, though participation from the release triage folks might be nice

simulacrum (May 10 2019 at 14:53, on Zulip):

if we get a scheduled slot I can bring it up next release team meeting and try to rope in some folks

nikomatsakis (May 10 2019 at 14:53, on Zulip):

I'm not entirely sure what we'd discuss concretely though :)

I think it would be good to have a proposal on the table

nikomatsakis (May 10 2019 at 14:53, on Zulip):

but I guess I suspect I can get some more structured notes before then

nikomatsakis (May 10 2019 at 14:53, on Zulip):

I figured that the bar for "details" is lower for non-technical meetings

nikomatsakis (May 10 2019 at 14:53, on Zulip):

but I can't decide if it should be this low ;)

nikomatsakis (May 10 2019 at 14:54, on Zulip):

(still, we've often had some pretty productive steering meetings without any kind of "pre-plan")

mw (May 10 2019 at 14:54, on Zulip):

let's definitely make @pnkfelix aware of the topic

nikomatsakis (May 10 2019 at 14:54, on Zulip):

Let's say this

nikomatsakis (May 10 2019 at 14:55, on Zulip):

We can tentatively schedule but revisit next week, to see if we've had time to make a kind of pre-plan?

nikomatsakis (May 10 2019 at 14:56, on Zulip):

I'll open a topic in #t-compiler/wg-meta

nikomatsakis (May 10 2019 at 14:56, on Zulip):

to discuss a bit

mw (May 10 2019 at 14:56, on Zulip):

@nagisa also might have opinions on the topic

nikomatsakis (May 10 2019 at 14:56, on Zulip):

ah, yeah

nikomatsakis (May 10 2019 at 14:56, on Zulip):

ok, thanks everyone for participating <3

nikomatsakis (May 10 2019 at 14:56, on Zulip):

Everybody happy with these results? If so, let's close the meeting I think :)

nikomatsakis (May 10 2019 at 14:57, on Zulip):

That is:

mw (May 10 2019 at 14:57, on Zulip):

thanks for driving!

mw (May 10 2019 at 14:57, on Zulip):

:wave:

lwshang (May 10 2019 at 16:52, on Zulip):

I'm very glad to be included in the discussion here. I can make it on 05/17 at the similar time (14:00 UTC time). Shall we continue the following discussion about parallel-rustc on its own stream #t-compiler/wg-parallel-rustc ? That will make it easier to find relative information in the future.

Last update: Nov 20 2019 at 01:45UTC