Stream: t-compiler/meetings

Topic: [steering meeting] 2020-12-04 -- Performance Goals for 2020


pnkfelix (Dec 04 2020 at 15:03, on Zulip):

hi @T-compiler/meeting , we're starting today's steering meeting now

pnkfelix (Dec 04 2020 at 15:04, on Zulip):

We have an agenda, or at least some notes, prepared for us thanks to @simulacrum : https://docs.google.com/document/d/1DzQtKpudHqCiYPuRrFPMkhMCaQ884uG_8dh3Vufv9Vc/edit

pnkfelix (Dec 04 2020 at 15:04, on Zulip):

I don't remember: Did we establish that we would have this meeting over Zulip?

pnkfelix (Dec 04 2020 at 15:04, on Zulip):

(vs zoom?)

simulacrum (Dec 04 2020 at 15:05, on Zulip):

I think Zulip is fine

pnkfelix (Dec 04 2020 at 15:05, on Zulip):

Okay, lets go with zulip then

pnkfelix (Dec 04 2020 at 15:06, on Zulip):

so yeah, lets all maybe take five minutes right now to read the doc

pnkfelix (Dec 04 2020 at 15:06, on Zulip):

and then we can talk about it

simulacrum (Dec 04 2020 at 15:06, on Zulip):

You all should be able to edit / leave comments btw

pnkfelix (Dec 04 2020 at 15:08, on Zulip):

@simulacrum seems like this "A cold-cache (fresh checkout) should produce a [...]" was an unfinished sentence

pnkfelix (Dec 04 2020 at 15:08, on Zulip):

ha! Nice fix!

simulacrum (Dec 04 2020 at 15:08, on Zulip):

yeah not sure what I was thinking

simulacrum (Dec 04 2020 at 15:08, on Zulip):

probably some other metric

pnkfelix (Dec 04 2020 at 15:10, on Zulip):

I think its worth clarifying something up front: You would prefer the improvements here to be "general" in this sense that they would apply to other projects of rustc's size/structure, and/or would apply to a snapshot of rustc taken today, right?

simulacrum (Dec 04 2020 at 15:10, on Zulip):

I think sort of

simulacrum (Dec 04 2020 at 15:10, on Zulip):

I would focus 100% on rustc itself

pnkfelix (Dec 04 2020 at 15:10, on Zulip):

when I first read this, I was thinking it might be more narrow, as in a solution of restructuring the rustc repo (e.g. fine-grain librarification) could address at least some of these issues

simulacrum (Dec 04 2020 at 15:11, on Zulip):

e.g., if it means we have some hacks that we "know" are true in rustc, I would be fine with that

pnkfelix (Dec 04 2020 at 15:11, on Zulip):

but restructuring the rustc repo doesn't fix the problem for other projects

simulacrum (Dec 04 2020 at 15:11, on Zulip):

but I would expect the majority of the work to yield steps, at least, that others could follow

pnkfelix (Dec 04 2020 at 15:11, on Zulip):

Oh I see

simulacrum (Dec 04 2020 at 15:11, on Zulip):

e.g. restructuring is something others can do, and we can say that's recommended

pnkfelix (Dec 04 2020 at 15:11, on Zulip):

So restructuring can be part of the plan, okay.

pnkfelix (Dec 04 2020 at 15:12, on Zulip):

that helps me understand, thanks.

mw (Dec 04 2020 at 15:12, on Zulip):

I suspect that there could be wins from making some things around queries non-generic

mw (Dec 04 2020 at 15:13, on Zulip):

and rustc_middle would profit from better CGU partitioning

nikomatsakis (Dec 04 2020 at 15:13, on Zulip):

(hey all, sorry, I'm in a meeting that's running over)

simulacrum (Dec 04 2020 at 15:13, on Zulip):

Yep, both of those seem like specific steps that we can also use to document some general steps

pnkfelix (Dec 04 2020 at 15:13, on Zulip):

Okay, put a :thumbs_up: on this comment if you are finshed reading, and put a :thumbs_down: if you need more time. (and I'll assume non-responses are :thumbs_down: )

davidtwco (Dec 04 2020 at 15:14, on Zulip):

We should make an effort to document things that we change that aren't just making rustc itself faster in njn's perf book.

davidtwco (Dec 04 2020 at 15:15, on Zulip):

e.g. changing how we structure our crate graph, etc.

pnkfelix (Dec 04 2020 at 15:15, on Zulip):

njn's book is focused on object code performance, not build times, right?

pnkfelix (Dec 04 2020 at 15:15, on Zulip):

(or is it both?)

mw (Dec 04 2020 at 15:15, on Zulip):

it has a chapter on compile times

mw (Dec 04 2020 at 15:15, on Zulip):

https://nnethercote.github.io/perf-book/compile-times.html

pnkfelix (Dec 04 2020 at 15:16, on Zulip):

okay so I think we're only waiting for @Léo Lanteri Thauvin and @nikomatsakis to finish reading at this point

nikomatsakis (Dec 04 2020 at 15:16, on Zulip):

sorry don't block on me

nikomatsakis (Dec 04 2020 at 15:16, on Zulip):

I will catch up

nikomatsakis (Dec 04 2020 at 15:17, on Zulip):

I can't focus right now

pnkfelix (Dec 04 2020 at 15:17, on Zulip):

Lets go ahead and get started on discussion

mw (Dec 04 2020 at 15:17, on Zulip):

if the PGO work I've been discussing with @simulacrum and others bares fruit, the bootstrap compiler would also become faster (~15%)

pnkfelix (Dec 04 2020 at 15:18, on Zulip):

wait, I had thought PGO wasn't showing so much of a win in many cases?

pnkfelix (Dec 04 2020 at 15:18, on Zulip):

what happened?

mw (Dec 04 2020 at 15:18, on Zulip):

wall-time improvements look pretty promising: https://perf.rust-lang.org/compare.html?start=pgo-2020-10-30-none-20&end=pgo-2020-10-30-both-20&stat=wall-time

pnkfelix (Dec 04 2020 at 15:19, on Zulip):

(am I misremembering, about PGO not showing a benefit in the past?)

pnkfelix (Dec 04 2020 at 15:19, on Zulip):

well anyway that is promising

mw (Dec 04 2020 at 15:19, on Zulip):

pure instruction counts look less good because PGO often improves caching and branch prediction

pnkfelix (Dec 04 2020 at 15:20, on Zulip):

maybe I'm thinking of PGO not meeting expectations on Firefox or something

pnkfelix (Dec 04 2020 at 15:20, on Zulip):

anyway

mw (Dec 04 2020 at 15:20, on Zulip):

pnkfelix said:

maybe I'm thinking of PGO not meeting expectations on Firefox or something

Yes, I think that improved after they adapted their training sets

pnkfelix (Dec 04 2020 at 15:21, on Zulip):

Okay. @simulacrum , am I right that the main actions proposed by the doc are the two bullets below the "We should aim for: ..." ?

simulacrum (Dec 04 2020 at 15:22, on Zulip):

Kind of -- not sure I'd call them actions, more goals

pnkfelix (Dec 04 2020 at 15:22, on Zulip):

namely: 1. turn on incr-comp for contributors, and 2. make it easier/possible to build just individual components (like rustdoc, test suite, std lib)

simulacrum (Dec 04 2020 at 15:22, on Zulip):

but yes, in some sense

pnkfelix (Dec 04 2020 at 15:23, on Zulip):

I guess I see them as strategies (or tactics? Still need to make that distinction clear in my head) for acheiving the end goal of better build times.

simulacrum (Dec 04 2020 at 15:23, on Zulip):

yes that's accurate

pnkfelix (Dec 04 2020 at 15:23, on Zulip):

What are blockers for incremental? One big one is that we still don't have a great story around fixing bugs there

pnkfelix (Dec 04 2020 at 15:23, on Zulip):

i.e. we count ourselves lucky when we get a bug report that we can actually reproduce

pnkfelix (Dec 04 2020 at 15:24, on Zulip):

that isn't an objection per se to turning it on by default

simulacrum (Dec 04 2020 at 15:24, on Zulip):

yeah -- I think digging in here might be useful -- I've not heard a lot from incr WG though I know y'all have been meeting a bunch lately

simulacrum (Dec 04 2020 at 15:24, on Zulip):

the main thing I have personally experienced is that incremental kinda feels lackluster a lot of the time

pnkfelix (Dec 04 2020 at 15:25, on Zulip):

but we don't want to be in a situation where we are repeatedly telling new people "oh, that ICE? yeah, you need to clean your build dir first"

mw (Dec 04 2020 at 15:25, on Zulip):

I've usually turned incremental on when building rustc and don't run into many bugs

pnkfelix (Dec 04 2020 at 15:25, on Zulip):

as in, turning incremental on doesn't actually yield the "feel" of an incremental environment ?

mw (Dec 04 2020 at 15:25, on Zulip):

but I'm not building rustc that often these days

Wesley Wiser (Dec 04 2020 at 15:26, on Zulip):

I've pretty much only used -i when building rustc for the last year or two.

pnkfelix (Dec 04 2020 at 15:26, on Zulip):

simulacrum said:

the main thing I have personally experienced is that incremental kinda feels lackluster a lot of the time

@simulacrum what did "lackluster" here mean?

simulacrum (Dec 04 2020 at 15:26, on Zulip):

Yes, it doesn't feel like a big win

pnkfelix (Dec 04 2020 at 15:26, on Zulip):

okay

simulacrum (Dec 04 2020 at 15:26, on Zulip):

Maybe I just have good hardware -- I've heard differently

simulacrum (Dec 04 2020 at 15:26, on Zulip):

but I never felt a marked difference with it on

pnkfelix (Dec 04 2020 at 15:26, on Zulip):

no I think the wg-incr-comp agrees that it doesn't seem to yield the benefits you would expect

simulacrum (Dec 04 2020 at 15:27, on Zulip):

and the clean build is slower by our benchmarks, so I'm always kinda "meh" on turning it on

mw (Dec 04 2020 at 15:27, on Zulip):

yeah, I think it performs worse on rustc than on "regular" crates

mw (Dec 04 2020 at 15:27, on Zulip):

for me the felt difference is a lot bigger when I work on crates that don't require rustc_middle to be rebuilt

simulacrum (Dec 04 2020 at 15:28, on Zulip):

that might be my problem -- I usually end up touching middle or even parser in some of my work :)

nikomatsakis (Dec 04 2020 at 15:28, on Zulip):

I've definitely found it to be very effective sometimes -- but also sometimes completely ineffective

simulacrum (Dec 04 2020 at 15:28, on Zulip):

but, I guess, my concrete question for incr WG is: can we pull off these subminute recompiles?

simulacrum (Dec 04 2020 at 15:28, on Zulip):

like, is that a feasible goal?

simulacrum (Dec 04 2020 at 15:29, on Zulip):

one of the reasons I ask is because it's a key part -- imo -- of delivering e.g. documentation building quickly after edits to docs to users

simulacrum (Dec 04 2020 at 15:30, on Zulip):

I know I'm always super frustrated to type up some comments on a rustc_middle function for example

pnkfelix (Dec 04 2020 at 15:31, on Zulip):

so if I recall correctly, one of the issues there is that the spans all change when you change comments, right?

pnkfelix (Dec 04 2020 at 15:31, on Zulip):

I know there's been some work in that space

pnkfelix (Dec 04 2020 at 15:31, on Zulip):

to make span changes not cause the rest of the stuff to have to be rebuilt

pnkfelix (Dec 04 2020 at 15:32, on Zulip):

or at least ideas thrown around

pnkfelix (Dec 04 2020 at 15:32, on Zulip):

but I don't remember what's actually happened at this point

pnkfelix (Dec 04 2020 at 15:32, on Zulip):

@Wesley Wiser do you remember?

lcnr (Dec 04 2020 at 15:32, on Zulip):

@cjgillot is currently working on this (#72015 and it's sub prs)

lcnr (Dec 04 2020 at 15:32, on Zulip):

or at least has previously

lcnr (Dec 04 2020 at 15:33, on Zulip):

the big issue here is that only changing some of the spans caused some ~5% perf regressions or so without any incremental benefits iirc

lcnr (Dec 04 2020 at 15:33, on Zulip):

so they pretty much had to change all in one go which was both enormous and still had a small perf impact i think

simulacrum (Dec 04 2020 at 15:34, on Zulip):

The concrete proposal being made here, I guess, is that we try to focus compiler team implementation/design bandwidth on this goal of incremental performance / rustc build times. I'm not quite sure what that means -- maybe e.g. we'd focus in on prioritizing incremental bugs (both ICE and perf)

pnkfelix (Dec 04 2020 at 15:34, on Zulip):

its a good idea

pnkfelix (Dec 04 2020 at 15:34, on Zulip):

one thing I do know from the recent wg-incr-comp meetings

pnkfelix (Dec 04 2020 at 15:34, on Zulip):

is that all the members are swamped

Wesley Wiser (Dec 04 2020 at 15:34, on Zulip):

@pnkfelix I know there's tracking issues for that but I wasn't aware there was progress being made on them.

pnkfelix (Dec 04 2020 at 15:35, on Zulip):

with things not related to incr-comp

nikomatsakis (Dec 04 2020 at 15:35, on Zulip):

I am very in favor of this as a focus

pnkfelix (Dec 04 2020 at 15:35, on Zulip):

so progress on the tasks there themselves is also lackluster

simulacrum (Dec 04 2020 at 15:35, on Zulip):

is it other compiler tasks, or just other things?

simulacrum (Dec 04 2020 at 15:35, on Zulip):

basically "can we meaningfully prioritize"

nikomatsakis (Dec 04 2020 at 15:35, on Zulip):

Beyond what's in the doc, it seems to pay multiple dividends (e.g., there is a "positive feedback" where making rustc compile faster will yield more people to help make it compile faster)

nikomatsakis (Dec 04 2020 at 15:35, on Zulip):

But yeah the tactics is the big question, how do we organize to be successful at this goal?

Wesley Wiser (Dec 04 2020 at 15:36, on Zulip):

simulacrum said:

is it other compiler tasks, or just other things?

At least for myself, it's a bit of both. But I'm starting to find more time for incr comp things.

nikomatsakis (Dec 04 2020 at 15:36, on Zulip):

at least the metrics are relatively clear -- but actually it'd be good to define specific workloads I think

nikomatsakis (Dec 04 2020 at 15:36, on Zulip):

editing a comment in rustc_middle seems very plausible

nikomatsakis (Dec 04 2020 at 15:36, on Zulip):

I guess my experience is that perf wins don't come until you can profile and stare at the actual code path

davidtwco (Dec 04 2020 at 15:36, on Zulip):

simulacrum said:

is it other compiler tasks, or just other things?

Likewise to Wesley Wiser , but for me, it's mostly life getting in the way but occasionally other PRs.

nikomatsakis (Dec 04 2020 at 15:37, on Zulip):

but ok I see we're talking more about distractions...

nikomatsakis (Dec 04 2020 at 15:37, on Zulip):

I feel like having a dedicated working group is certainly a start

nikomatsakis (Dec 04 2020 at 15:38, on Zulip):

I've not been following along but how much of a "roadmap" does the working group have?

nikomatsakis (Dec 04 2020 at 15:38, on Zulip):

(no judgement any which way)

pnkfelix (Dec 04 2020 at 15:38, on Zulip):

mmm, I don't think we have a roadmap

pnkfelix (Dec 04 2020 at 15:38, on Zulip):

more a set of tasks that we're all working on

pnkfelix (Dec 04 2020 at 15:38, on Zulip):

and checking in with eachother about

nikomatsakis (Dec 04 2020 at 15:38, on Zulip):

what sort of tasks?

pnkfelix (Dec 04 2020 at 15:38, on Zulip):

and triaging the archive of issues

nikomatsakis (Dec 04 2020 at 15:39, on Zulip):

(like, specific issues?)

pnkfelix (Dec 04 2020 at 15:39, on Zulip):

/me pulls up notes from last meeting

pnkfelix (Dec 04 2020 at 15:39, on Zulip):

specific features

davidtwco (Dec 04 2020 at 15:40, on Zulip):

nikomatsakis said:

what sort of tasks?

I suppose this is the time to post this since that's what I'm working on:

Don't want to derail the discussion too much - not sure if the discussion is intended to be focused on "should this be the goal" or "how can we achieve this goal", but as an intermediate goal:

We could consider turning on Split DWARF for rustc by default too - might help cut down on link times. I've been intending to do this once Split DWARF lands and gets into the bootstrap compiler.

pnkfelix (Dec 04 2020 at 15:40, on Zulip):

(like, @Santiago Pastorino has been working on PR #76896 for a while, whch avoids making local copies fo inline fns in debug mode)

pnkfelix (Dec 04 2020 at 15:41, on Zulip):

pnkfelix said:

/me pulls up notes from last meeting

Those notes, for those who are interested: https://hackmd.io/WCuUWLCAQ2WTjUQyZeSCgg

Santiago Pastorino (Dec 04 2020 at 15:41, on Zulip):

pnkfelix said:

(like, Santiago Pastorino has been working on PR #76896 for a while, whch avoids making local copies fo inline fns in debug mode)

well I'm not really working on that, just did the PR and failed on CI and didn't have time to repro it

pnkfelix (Dec 04 2020 at 15:41, on Zulip):

:)

simulacrum (Dec 04 2020 at 15:41, on Zulip):

Yep, I think split dwarf and other measures that we may not be able to stabilize but can definitely turn on for local compiles is absolutely in scope

pnkfelix (Dec 04 2020 at 15:41, on Zulip):

You were the lucky one with the first item on those notes

pnkfelix (Dec 04 2020 at 15:42, on Zulip):

But the point is

pnkfelix (Dec 04 2020 at 15:42, on Zulip):

I think its safe to say we don't have a roadmap

nikomatsakis (Dec 04 2020 at 15:42, on Zulip):

OK. I guess I'm thinking that there are two basic discussions we could have

nikomatsakis (Dec 04 2020 at 15:43, on Zulip):

the former seems kind of hard to do, unless it's a matter of rustc-related tasks that we can "reapportion"

nikomatsakis (Dec 04 2020 at 15:43, on Zulip):

it's necessarily going to be kind of personal :)

simulacrum (Dec 04 2020 at 15:43, on Zulip):

maybe s/working group/team, in some sense

nikomatsakis (Dec 04 2020 at 15:43, on Zulip):

(though I think that it'd be interesting to try and do some 1:1 conversations around this, I've wanted us to get better at mentoring people in these respects, or e.g. in working with employers to get paid time...)

simulacrum (Dec 04 2020 at 15:43, on Zulip):

e.g. I think there's a big question of "Can compiler team as a whole roadmap productively?"

nikomatsakis (Dec 04 2020 at 15:44, on Zulip):

mm, well, I debated the choice of words though

simulacrum (Dec 04 2020 at 15:44, on Zulip):

(I don't know)

pnkfelix (Dec 04 2020 at 15:44, on Zulip):

we're at the 45 min mark

pnkfelix (Dec 04 2020 at 15:44, on Zulip):

and

nikomatsakis (Dec 04 2020 at 15:44, on Zulip):

my hunch is that it's better to talk about wg but there is a good question of managing the set of wgs and relative priorities

tm (Dec 04 2020 at 15:44, on Zulip):

Changes from #77307 also looked quite promising in terms of bootstrap times.

pnkfelix (Dec 04 2020 at 15:44, on Zulip):

WhileI think there's useful stuff to talk about w.r.t. incr-comp

pnkfelix (Dec 04 2020 at 15:44, on Zulip):

there is also the other bullet that @simulacrum put on here

simulacrum (Dec 04 2020 at 15:45, on Zulip):

Contributors aiming to edit specific components (rustdoc, test suite, standard library) can build just those components, at the cost of some network usage.

simulacrum (Dec 04 2020 at 15:45, on Zulip):

This is sort of "just" rustbuild

nikomatsakis (Dec 04 2020 at 15:45, on Zulip):

/me contemplates a rebranding frm "incremental wg" to "project fast bootstrap"

simulacrum (Dec 04 2020 at 15:45, on Zulip):

but it kinda gets into our guarantees around ABI etc, I've long wanted to sit down with someone and work out exactly what we need for reliable linkage and such

nikomatsakis (Dec 04 2020 at 15:46, on Zulip):

(I would like that very much)

nikomatsakis (Dec 04 2020 at 15:46, on Zulip):

to the point, I'd like us to have a way to signal when an ABI-breaking change occurs and track that

nikomatsakis (Dec 04 2020 at 15:46, on Zulip):

but it is tricky

nikomatsakis (Dec 04 2020 at 15:46, on Zulip):

it's not something the compiler can do alone, changes to stdlib etc apply too

simulacrum (Dec 04 2020 at 15:46, on Zulip):

yep

pnkfelix (Dec 04 2020 at 15:47, on Zulip):

but such changes are still rare, right?

nikomatsakis (Dec 04 2020 at 15:47, on Zulip):

I don't know

nikomatsakis (Dec 04 2020 at 15:47, on Zulip):

e.g., adding a field to some stdlib struct?

simulacrum (Dec 04 2020 at 15:47, on Zulip):

I would like to also consider e.g. matklad's musings on whether std and rustc need such a tight coupling

nikomatsakis (Dec 04 2020 at 15:47, on Zulip):

probably occurs in every release right now, though not every nightly

pnkfelix (Dec 04 2020 at 15:47, on Zulip):

I would think the problem is tracking them. But your point is that maybe they are also happening too often?

pnkfelix (Dec 04 2020 at 15:47, on Zulip):

but yeah, also: I wouldn't be surprised if its every release.

pnkfelix (Dec 04 2020 at 15:48, on Zulip):

but I also would think that from nightly-to-nightly its often fine

nikomatsakis (Dec 04 2020 at 15:48, on Zulip):

simulacrum said:

e.g. I think there's a big question of "Can compiler team as a whole roadmap productively?"

so to this point, one thing we could be talking about is whether we can recruit more people (and broader mindset) to "incremental wg / project fast bootstrap", I guess

simulacrum (Dec 04 2020 at 15:48, on Zulip):

right, yes, that would seem potentially beneficial

nikomatsakis (Dec 04 2020 at 15:49, on Zulip):

specifically more compiler team contributors, probably, though I think it's good to think about new folks

simulacrum (Dec 04 2020 at 15:49, on Zulip):

e.g. I could imagine that we say that work on non-incr/bootstrap speed is going to be deprioritized in review queues, though that may be stronger than we want

pnkfelix (Dec 04 2020 at 15:49, on Zulip):

a mix would be good

nikomatsakis (Dec 04 2020 at 15:50, on Zulip):

I would very much like it if we were able to fulfill the vision of having the compiler team manage its set of active projects (with perhaps the expectation that compiler team contributors get involved in some active project etc)

pnkfelix (Dec 04 2020 at 15:50, on Zulip):

simulacrum said:

e.g. I could imagine that we say that work on non-incr/bootstrap speed is going to be deprioritized in review queues, though that may be stronger than we want

this is interesting. I wouldn't want to deprioritize for a whole year

pnkfelix (Dec 04 2020 at 15:50, on Zulip):

but for a month?

pnkfelix (Dec 04 2020 at 15:50, on Zulip):

could be very interesting

nikomatsakis (Dec 04 2020 at 15:50, on Zulip):

I don't think that this should be the only project (e.g., I intend to keep pushing on RFC 2229, chalk integration), but I do think it is an important one

pnkfelix (Dec 04 2020 at 15:51, on Zulip):

I'm mostly thinking that if we dropped a bunch of other stuff for a month, how much could we improve the cycle time for compiler dev

pnkfelix (Dec 04 2020 at 15:51, on Zulip):

in a month

nikomatsakis (Dec 04 2020 at 15:51, on Zulip):

simulacrum said:

e.g. I could imagine that we say that work on non-incr/bootstrap speed is going to be deprioritized in review queues, though that may be stronger than we want

I would .. hmm. I wouldn't put it like this, but I do think that thinking about review speed and tying to active projects etc is a good idea.

nikomatsakis (Dec 04 2020 at 15:51, on Zulip):

@pnkfelix I do think that sprints are a great way to build momentum

nikomatsakis (Dec 04 2020 at 15:51, on Zulip):

though people mean many different things when they say sprints

nikomatsakis (Dec 04 2020 at 15:51, on Zulip):

one thing we've done in polonius wg from time to time that I found super helpful was

nikomatsakis (Dec 04 2020 at 15:51, on Zulip):

scheduling a week

simulacrum (Dec 04 2020 at 15:52, on Zulip):

right -- in some sense, e.g., if we can expect say a 25% improvement from a month's time of "all hands on deck" then we absolutely should do that in jan/feb, right?

nikomatsakis (Dec 04 2020 at 15:52, on Zulip):

where we all cleared our calendars

nikomatsakis (Dec 04 2020 at 15:52, on Zulip):

and put in like 4 hours each morning

mw (Dec 04 2020 at 15:52, on Zulip):

that sounds nice

nikomatsakis (Dec 04 2020 at 15:52, on Zulip):

I suspect that a full month may be too tall of an ask, but that we might be able to manage a push for 1 week (which generates follow-up items that are then prioritized somewhat less so over the next 3 weeks)

simulacrum (Dec 04 2020 at 15:52, on Zulip):

yeah I think that makes sense too

pnkfelix (Dec 04 2020 at 15:53, on Zulip):

if we synchronzied it right

pnkfelix (Dec 04 2020 at 15:53, on Zulip):

so that its a week of hacking

simulacrum (Dec 04 2020 at 15:53, on Zulip):

I think it'd be great to try to figure out those week sprints perhaps at our planning meetings?

pnkfelix (Dec 04 2020 at 15:53, on Zulip):

followed by one or more weeks of review/polish

pnkfelix (Dec 04 2020 at 15:54, on Zulip):

so lets try to work out what the concrete outcomes here are

Wesley Wiser (Dec 04 2020 at 15:54, on Zulip):

I think that's a really great idea and I would definitely be willing to block off time for that. A whole month would probably be difficult but I could certainly do a week.

pnkfelix (Dec 04 2020 at 15:55, on Zulip):

I want to first double-check about the linkage stuff for just building individual components

pnkfelix (Dec 04 2020 at 15:55, on Zulip):

@simulacrum it sounds like there's more research that would need to be done here, right?

simulacrum (Dec 04 2020 at 15:55, on Zulip):

sort of -- I think probably a 1:1 session to talk through some things might even be enough

pnkfelix (Dec 04 2020 at 15:56, on Zulip):

okay. I'd like to have it in scope for 2021.

simulacrum (Dec 04 2020 at 15:56, on Zulip):

mostly again needs a block out of time :)

pnkfelix (Dec 04 2020 at 15:56, on Zulip):

but I'd also like to figure out when that discussion should happen, roughly

pnkfelix (Dec 04 2020 at 15:56, on Zulip):

e.g. do we table it until later, like March or April

simulacrum (Dec 04 2020 at 15:56, on Zulip):

I think I cannot commit to more until January

pnkfelix (Dec 04 2020 at 15:56, on Zulip):

so that we can focus on the stuff for first bullet until then

simulacrum (Dec 04 2020 at 15:56, on Zulip):

I expect this to happen in parallel, roughly

pnkfelix (Dec 04 2020 at 15:56, on Zulip):

In my head, this whole conversation is about January

simulacrum (Dec 04 2020 at 15:57, on Zulip):

incremental is sort of "in component"

pnkfelix (Dec 04 2020 at 15:57, on Zulip):

mmm

pnkfelix (Dec 04 2020 at 15:57, on Zulip):

I don't know if there's a benefit to doing them in parallel?

simulacrum (Dec 04 2020 at 15:57, on Zulip):

hm not sure I follow

Wesley Wiser (Dec 04 2020 at 15:57, on Zulip):

If we want to see a payoff for the time invested within next year, it would be better to do it earlier in the year. So I would advocate for January or possibly February instead of March or April.

pnkfelix (Dec 04 2020 at 15:57, on Zulip):

Well, if we had infinite dev resources

simulacrum (Dec 04 2020 at 15:57, on Zulip):

Yeah, I think earlier is better than later

pnkfelix (Dec 04 2020 at 15:57, on Zulip):

then doing them in parallel would make perfect sense

nikomatsakis (Dec 04 2020 at 15:57, on Zulip):

pnkfelix said:

In my head, this whole conversation is about January

definitely not December :)

pnkfelix (Dec 04 2020 at 15:57, on Zulip):

but we don't

simulacrum (Dec 04 2020 at 15:57, on Zulip):

but I agree that they don't need to happen in parallel

pnkfelix (Dec 04 2020 at 15:57, on Zulip):

and given the resources we do have

nikomatsakis (Dec 04 2020 at 15:58, on Zulip):

Wesley Wiser said:

I think that's a really great idea and I would definitely be willing to block off time for that. A whole month would probably be difficult but I could certainly do a week.

Ps, this is why in polonius we do a week, much easier for folks who are not full time :)

pnkfelix (Dec 04 2020 at 15:58, on Zulip):

I think it will be better for groups to focus on one bullet or the other

simulacrum (Dec 04 2020 at 15:58, on Zulip):

I could see e.g. focusing on the separate components in January and incr in Feb

pnkfelix (Dec 04 2020 at 15:58, on Zulip):

interesting

nikomatsakis (Dec 04 2020 at 15:58, on Zulip):

(Idea:

nikomatsakis (Dec 04 2020 at 15:58, on Zulip):

We could make this a regular compiler team thing?

pnkfelix (Dec 04 2020 at 15:58, on Zulip):

you think we'll get more payoff from doing separate components first?

simulacrum (Dec 04 2020 at 15:59, on Zulip):

I think so yes

nikomatsakis (Dec 04 2020 at 15:59, on Zulip):

I'm contemplating like "first week of every month is hack week, where the team as a whole picks a project"

pnkfelix (Dec 04 2020 at 15:59, on Zulip):

@nikomatsakis you mean sprints?

nikomatsakis (Dec 04 2020 at 15:59, on Zulip):

the other 3 weeks are your own ;)

pnkfelix (Dec 04 2020 at 15:59, on Zulip):

that's interesting

nikomatsakis (Dec 04 2020 at 15:59, on Zulip):

that's probably too ambitious but maybe a mildly scaled back version

nikomatsakis (Dec 04 2020 at 15:59, on Zulip):

e.g., we have some such weeks

simulacrum (Dec 04 2020 at 15:59, on Zulip):

separate components already cuts build times strongly for a good portion of people (e.g., a rustdoc dev only changing rustdoc)

pnkfelix (Dec 04 2020 at 15:59, on Zulip):

this is all intermixed with our schedules for design meetings and what not

simulacrum (Dec 04 2020 at 15:59, on Zulip):

and incremental sort of only matters for compiler devs (obviously important)

simulacrum (Dec 04 2020 at 16:00, on Zulip):

e.g. if you change a crate that has a <1 minute rebuild time from clean, then incremental isn't going to be a huge win I imagine

pnkfelix (Dec 04 2020 at 16:00, on Zulip):

I don't know

pnkfelix (Dec 04 2020 at 16:00, on Zulip):

7 seconds vs 14 is big to me

simulacrum (Dec 04 2020 at 16:00, on Zulip):

whereas if we can get a compiler rebuild from 5 minutes to 1 minute, that seems big

pnkfelix (Dec 04 2020 at 16:01, on Zulip):

anyway

pnkfelix (Dec 04 2020 at 16:01, on Zulip):

we're over time

pnkfelix (Dec 04 2020 at 16:01, on Zulip):

and we haven't actually made a concrete plan

simulacrum (Dec 04 2020 at 16:01, on Zulip):

I like Niko's idea of sprint weeks, maybe scheduled at planning meetings

pnkfelix (Dec 04 2020 at 16:01, on Zulip):

but it sounds like @simulacrum is optimistic that we could see real progress in January on the indivdual component builds

pnkfelix (Dec 04 2020 at 16:01, on Zulip):

if we focus on it.

pnkfelix (Dec 04 2020 at 16:01, on Zulip):

right?

simulacrum (Dec 04 2020 at 16:01, on Zulip):

yes

pnkfelix (Dec 04 2020 at 16:02, on Zulip):

okay.

nikomatsakis (Dec 04 2020 at 16:02, on Zulip):

simulacrum said:

I like Niko's idea of sprint weeks, maybe scheduled at planning meetings

yeah i'm into this idea, I think monthly is too much, but maybe every 2 months, or quarterly, or something

pnkfelix (Dec 04 2020 at 16:02, on Zulip):

as in 1 sprint/month is too frequent?

nikomatsakis (Dec 04 2020 at 16:02, on Zulip):

if we're going to do it, I think they should be scheduled at fixed times, like our meeting slots, so people can play way ahead

pnkfelix (Dec 04 2020 at 16:02, on Zulip):

or planning them each month is too frequent?

nikomatsakis (Dec 04 2020 at 16:02, on Zulip):

maybe, maybe not

nikomatsakis (Dec 04 2020 at 16:02, on Zulip):

I was thinking holding them would be too much but I could be persuaded otherwise

nikomatsakis (Dec 04 2020 at 16:03, on Zulip):

one thing I will add:

pnkfelix (Dec 04 2020 at 16:03, on Zulip):

I think good outcome right now is to plan to plan at the next planning meeting

nikomatsakis (Dec 04 2020 at 16:03, on Zulip):

I know that many employers give 20% time and they are happy for people to 'save it' and use it in a block

nikomatsakis (Dec 04 2020 at 16:03, on Zulip):

so e.g. I've talked to employers who say that they'd be happy to have people spend 1 week / month on open source

pnkfelix (Dec 04 2020 at 16:03, on Zulip):

or no, wait

pnkfelix (Dec 04 2020 at 16:03, on Zulip):

we don't have a planning meeting in decebmer

pnkfelix (Dec 04 2020 at 16:03, on Zulip):

we cancelled it yesterday

pnkfelix (Dec 04 2020 at 16:03, on Zulip):

:lol:

nikomatsakis (Dec 04 2020 at 16:03, on Zulip):

lol

pnkfelix (Dec 04 2020 at 16:04, on Zulip):

okay

pnkfelix (Dec 04 2020 at 16:04, on Zulip):

um

nikomatsakis (Dec 04 2020 at 16:04, on Zulip):

nikomatsakis said:

so e.g. I've talked to employers who say that they'd be happy to have people spend 1 week / month on open source

some kind of sprint-like structure could really optimize for this :)

pnkfelix (Dec 04 2020 at 16:04, on Zulip):

we'll make a concrete plan at some point this month

simulacrum (Dec 04 2020 at 16:04, on Zulip):

I think we can maybe just, uh, schedule last week of Jan or something

pnkfelix (Dec 04 2020 at 16:04, on Zulip):

for when we will focus on each of these two bullets

nikomatsakis (Dec 04 2020 at 16:04, on Zulip):

yeah

pnkfelix (Dec 04 2020 at 16:04, on Zulip):

it won't take long to decide

nikomatsakis (Dec 04 2020 at 16:04, on Zulip):

I think it should be a schedule like "last week of every month" except Dec

pnkfelix (Dec 04 2020 at 16:04, on Zulip):

either at a triage meeting or at one of the friday meetings

nikomatsakis (Dec 04 2020 at 16:04, on Zulip):

but we can start with Jan :)

nikomatsakis (Dec 04 2020 at 16:05, on Zulip):

ideally we'd have people sign up, and we would obviously not expect 100% participation

nikomatsakis (Dec 04 2020 at 16:05, on Zulip):

anyway I'm getting ahead of myself but I'm excited :)

pnkfelix (Dec 04 2020 at 16:05, on Zulip):

/me wonders why "last week of month" is better than "first week of month"

nikomatsakis (Dec 04 2020 at 16:05, on Zulip):

it doesn't matter, but first week of jan is bad :)

nikomatsakis (Dec 04 2020 at 16:05, on Zulip):

well, this Jan anyway

pnkfelix (Dec 04 2020 at 16:05, on Zulip):

clearly "second week of month" is the ideal compromise

pnkfelix (Dec 04 2020 at 16:05, on Zulip):

sure, but you've already said we'd cancel last week of december

nikomatsakis (Dec 04 2020 at 16:05, on Zulip):

last week may be bad because of how the human brain works

nikomatsakis (Dec 04 2020 at 16:06, on Zulip):

in particular:

Wesley Wiser (Dec 04 2020 at 16:06, on Zulip):

I would like to suggest that we try to do the planning for the sprint at least 2 weeks in advance. That way, if need be, we can use up to two Friday meetings to finalize the details. I think going into the week with things still undecided would likely hurt productivity a lot.

pnkfelix (Dec 04 2020 at 16:06, on Zulip):

actually i'm now wondering if 2nd week of month is ideal

nikomatsakis (Dec 04 2020 at 16:06, on Zulip):

I can imagine people have something due in February (or whatever)

nikomatsakis (Dec 04 2020 at 16:06, on Zulip):

and they have to cancel sprint to finish it

nikomatsakis (Dec 04 2020 at 16:06, on Zulip):

Wesley Wiser said:

I would like to suggest that we try to do the planning for the sprint at least 2 weeks in advance. That way, if need be, we can use up to two Friday meetings to finalize the details. I think going into the week with things still undecided would likely hurt productivity a lot.

yes, this was a thought I had about last week

nikomatsakis (Dec 04 2020 at 16:06, on Zulip):

i.e., that gives you time in that month to plan

nikomatsakis (Dec 04 2020 at 16:06, on Zulip):

clearly 3rd week is ideal :)

pnkfelix (Dec 04 2020 at 16:06, on Zulip):

I see

pnkfelix (Dec 04 2020 at 16:06, on Zulip):

Okay then maybe 3rd week of month

pnkfelix (Dec 04 2020 at 16:06, on Zulip):

heh

nikomatsakis (Dec 04 2020 at 16:06, on Zulip):

you start planning or the Jan sprint in Jan

nikomatsakis (Dec 04 2020 at 16:06, on Zulip):

people will definitely fail to plan to Feb sprint in Jan

nikomatsakis (Dec 04 2020 at 16:07, on Zulip):

etc

nikomatsakis (Dec 04 2020 at 16:07, on Zulip):

s/people/Niko/

nikomatsakis (Dec 04 2020 at 16:07, on Zulip):

ok I have to go :)

pnkfelix (Dec 04 2020 at 16:07, on Zulip):

alright. we'll finalize all this later

pnkfelix (Dec 04 2020 at 16:07, on Zulip):

thanks to everyone in @T-compiler/meeting for attending!!!

Joshua Nelson (Dec 04 2020 at 16:27, on Zulip):

simulacrum said:

This is sort of "just" rustbuild

well, for tools specifically I think it would be a giant boost not to have to require building rustc first, cc https://github.com/rust-lang/rust/pull/79540

Joshua Nelson (Dec 04 2020 at 16:30, on Zulip):

simulacrum said:

separate components already cuts build times strongly for a good portion of people (e.g., a rustdoc dev only changing rustdoc)

oh oops I didn't read far enough :laughing:

Joshua Nelson (Dec 04 2020 at 16:33, on Zulip):

another thing is that the conversation has focused really strongly on incremental builds - which definitely do matter - but not talked much about full/first-time builds

Joshua Nelson (Dec 04 2020 at 16:34, on Zulip):

for rustdoc specifically I guess that will matter less once rustc is downloaded from CI

Joshua Nelson (Dec 04 2020 at 16:34, on Zulip):

but in general I think it's an area we should work on, and it would help incremental builds too

Joshua Nelson (Dec 04 2020 at 16:43, on Zulip):

nikomatsakis said:

to the point, I'd like us to have a way to signal when an ABI-breaking change occurs and track that

this would be amazing :heart: I really hate having to tell people "sometimes incremental just breaks when bootstrapping"

Last update: Jan 22 2021 at 13:30UTC