Stream: t-compiler/wg-nll

Topic: profiling-analysis


nikomatsakis (Jun 26 2018 at 11:36, on Zulip):

I've been poring over the profiling results from https://github.com/rust-lang/rust/pull/51538 and taking notes in this gist. I've identified a number of areas for improvement (one of which was so simple I already pushed it into the PR as the final commit; I'm doing a new build now)

nikomatsakis (Jun 26 2018 at 11:36, on Zulip):

I'll try to file issues on those before today's meeting and advertise them then

nikomatsakis (Jun 26 2018 at 11:36, on Zulip):

but if anybody is looking for something to do, let me know, there are a number of immediately actionable things on that list (a few of which are already being examined)

lqd (Jun 26 2018 at 11:48, on Zulip):

wow great results!

nikomatsakis (Jun 26 2018 at 14:14, on Zulip):

Graph showing percentages relative to do_mir_borrowck

nikomatsakis (Jun 26 2018 at 14:14, on Zulip):

that is pretty interesting

nikomatsakis (Jun 26 2018 at 14:15, on Zulip):

I was previously looking at the graph with "Relative to total execution" — but really we are focused on MIR borrowck, so making them relative to that makes sense, and it highlights some of the areas that "seemed small" before

nikomatsakis (Jun 26 2018 at 14:15, on Zulip):

e.g., the "lazy causal tracking" that @Santiago Pastorino is looking at accounts for 13%

nikomatsakis (Jun 26 2018 at 14:16, on Zulip):

it seems clear that one major area of improvement is improving the various "dataflow-like" things we do

nikomatsakis (Jun 26 2018 at 14:17, on Zulip):

there are two of those: do_dataflow (18% of MIR borrowck) and liveness_of_locals (9%) — both are quite naive

nikomatsakis (Jun 26 2018 at 14:17, on Zulip):

@lqd :point_up: seems like something that may interest you

nikomatsakis (Jun 26 2018 at 14:17, on Zulip):

not sure how much we can readily gain there

nikomatsakis (Jun 26 2018 at 14:18, on Zulip):

but e.g. right now both of them walk the entire CFG each round, versus keeping a dirty list

lqd (Jun 26 2018 at 14:24, on Zulip):

oh this sounds interesting indeed :) I assume we'll talk about all these tasks spanning out of 51538 at tonight's meeting

nikomatsakis (Jun 26 2018 at 14:30, on Zulip):

yeah

nikomatsakis (Jun 28 2018 at 16:16, on Zulip):

I'm thinking I'll create a local build that includes not only master but also all the pending performance PRs (e.g., I have one, @Eh2406 has one, @Santiago Pastorino almost has one :) ) and try to re-run the profiling

nikomatsakis (Jun 28 2018 at 17:24, on Zulip):

https://gist.github.com/nikomatsakis/fa27738b52aa06bda9448553242e7404

nikomatsakis (Jun 28 2018 at 17:24, on Zulip):

current ratio is 1.55x (clap-rs) for me

lqd (Jun 28 2018 at 17:38, on Zulip):

njn has a couple

nikomatsakis (Jun 28 2018 at 17:46, on Zulip):

oh yeah I should snag those

nikomatsakis (Jun 28 2018 at 17:46, on Zulip):

the stuff that @Santiago Pastorino is working on looks increasingly relevant

nikomatsakis (Jun 28 2018 at 17:46, on Zulip):

as well as the dataflow improvements that @Pramod Bisht is hacking on

nikomatsakis (Jun 28 2018 at 17:47, on Zulip):

but beyond those things... I guess improving canonicalization...

nikomatsakis (Jun 28 2018 at 17:47, on Zulip):

it's starting to get tricky after that :)

nikomatsakis (Jun 28 2018 at 17:47, on Zulip):

maybe rewrite analysis in :frog: :stuck_out_tongue:

nikomatsakis (Jun 28 2018 at 17:47, on Zulip):

not entirely sure how I would do that

nikomatsakis (Jun 28 2018 at 17:48, on Zulip):

also not entirely sure it'd be a win :) but .. quite possible

nikomatsakis (Jun 28 2018 at 17:49, on Zulip):

one other possibility is that we are spending 13% in bitwise -- basically merging bit vectors

nikomatsakis (Jun 28 2018 at 17:49, on Zulip):

those particular bit vectors are not stored in a sparse way

nikomatsakis (Jun 28 2018 at 17:49, on Zulip):

not sure how dense/sparse they are in practice

lqd (Jun 28 2018 at 18:05, on Zulip):

could be interesting to instrument this to see how dense they are

nikomatsakis (Jun 28 2018 at 18:07, on Zulip):

yes

nikomatsakis (Jun 29 2018 at 14:07, on Zulip):

ok, in a bit I'm going to setup a build with all the latest PRs that are in a working state and redo the profiling

nikomatsakis (Jun 29 2018 at 14:08, on Zulip):

if anybody wants, i'll drop in to https://appear.in/i-heart-rust

nikomatsakis (Jun 29 2018 at 14:08, on Zulip):

and we can do it live :)

nikomatsakis (Jun 29 2018 at 14:08, on Zulip):

but I gotta to the build first

Santiago Pastorino (Jun 29 2018 at 14:10, on Zulip):

@nikomatsakis I'd like, but unsure if I can, when are you starting exactly?

Santiago Pastorino (Jun 29 2018 at 14:10, on Zulip):

do you plan to record it?

nikomatsakis (Jun 29 2018 at 14:10, on Zulip):

I could, sure ...

nikomatsakis (Jun 29 2018 at 14:10, on Zulip):

I don't know, maybe 30 minutes?

nikomatsakis (Jun 29 2018 at 14:10, on Zulip):

it's not gonna be that exciting I imagine ;)

nikomatsakis (Jun 29 2018 at 14:11, on Zulip):

mostly be staring at percentages going "uh..." :)

Santiago Pastorino (Jun 29 2018 at 14:13, on Zulip):

but I mean, when? you starting now?

lqd (Jun 29 2018 at 14:13, on Zulip):

my guess is ... in 30 minutes :p

nikomatsakis (Jun 29 2018 at 14:15, on Zulip):

@Santiago Pastorino what time works for you, if any? :)

Santiago Pastorino (Jun 29 2018 at 14:16, on Zulip):

after the Rust Reach thing would be great

Santiago Pastorino (Jun 29 2018 at 14:17, on Zulip):

here it's 11.16am, at 2pm would be perfect

Santiago Pastorino (Jun 29 2018 at 14:17, on Zulip):

otherwise now may work

Santiago Pastorino (Jun 29 2018 at 14:18, on Zulip):

if you prefer to do it now could be fine, I may need to stop before finishing

nikomatsakis (Jun 29 2018 at 14:18, on Zulip):

first question, are there more PRs pending that affect performance than these?

nikomatsakis (Jun 29 2018 at 14:18, on Zulip):

https://gist.github.com/nikomatsakis/b3d3e879ae2f0c34d911bb62561de7e1

nikomatsakis (Jun 29 2018 at 14:18, on Zulip):

that are in a buildable state, I mean

Santiago Pastorino (Jun 29 2018 at 14:19, on Zulip):

If you're asking me, I have no idea :), but I guess you're asking in general

nikomatsakis (Jun 29 2018 at 14:22, on Zulip):

well right now i'm still doing the merges

nikomatsakis (Jun 29 2018 at 14:25, on Zulip):

doing a build now, I expect it to be done in ~10 min or so

nikomatsakis (Jun 29 2018 at 14:25, on Zulip):

I'll ping then

nikomatsakis (Jun 29 2018 at 14:25, on Zulip):

it has all these branches in it:

https://gist.github.com/nikomatsakis/b3d3e879ae2f0c34d911bb62561de7e1

nikomatsakis (Jun 29 2018 at 14:26, on Zulip):

hopefully i'm not missing anything

lqd (Jun 29 2018 at 14:26, on Zulip):

I was thinking maybe the "improve canonicalization with hashing scheme" PR but we're not sure if it's in a buildable state (it failed tidy)

nikomatsakis (Jun 29 2018 at 14:26, on Zulip):

no it's not close yet

lqd (Jun 29 2018 at 14:27, on Zulip):

then all the ones I remember are there :)

nikomatsakis (Jun 29 2018 at 14:27, on Zulip):

I should probably have merged the latest rust-lang/master too but oh well

nikomatsakis (Jun 29 2018 at 14:27, on Zulip):

it ought to be pretty close with all those branches merged in :)

nikomatsakis (Jun 29 2018 at 14:34, on Zulip):

@Santiago Pastorino ok, build is done =)

lqd (Jun 29 2018 at 14:50, on Zulip):

(for others, niko is already streaming this work at https://appear.in/i-heart-rust)

nikomatsakis (Jun 29 2018 at 15:15, on Zulip):

hm, I'm looking at a detailed view of the type_check_internal

nikomatsakis (Jun 29 2018 at 15:15, on Zulip):

with some #[inline(never)] annotations

nikomatsakis (Jun 29 2018 at 15:15, on Zulip):

kind of intereting

nikomatsakis (Jun 29 2018 at 15:16, on Zulip):

not sure what it tells me yet ;)

nikomatsakis (Jun 29 2018 at 15:16, on Zulip):

one thing it says @Santiago Pastorino is that the time spent in simulate_block probably is at least 50% in the callback

nikomatsakis (Jun 29 2018 at 15:16, on Zulip):

so re-using that buffer may or may be worth it

nikomatsakis (Jun 29 2018 at 15:17, on Zulip):

although I might still be inclined to do it :)

Santiago Pastorino (Jun 29 2018 at 17:36, on Zulip):

what was that buffer?

nikomatsakis (Jun 29 2018 at 17:37, on Zulip):

heh :) I am referring to https://github.com/rust-lang/rust/issues/51818

nikomatsakis (Jun 29 2018 at 17:37, on Zulip):

also, the people next to me in the lunchroom are talking about the world cup in Spanish. I feel like you're hear with me in spirit ;) except I think they are Colombianos

Santiago Pastorino (Jun 29 2018 at 17:40, on Zulip):

hehehe

Santiago Pastorino (Jun 29 2018 at 17:40, on Zulip):

Colombianos :)

Santiago Pastorino (Jun 29 2018 at 17:40, on Zulip):

they are probably not from Uruguay

Santiago Pastorino (Jun 29 2018 at 17:41, on Zulip):

it's very unlikely to find somebody from Uruguay

Santiago Pastorino (Jun 29 2018 at 17:41, on Zulip):

:P

Last update: Nov 21 2019 at 13:05UTC