Stream: t-compiler/wg-self-profile

Topic: aggregating across multiple crates

nikomatsakis (Dec 02 2019 at 23:05, on Zulip):

So @Wesley Wiser, wycats was asking me about aggregating self-profile data across multiple crates.

nikomatsakis (Dec 02 2019 at 23:05, on Zulip):

They're creating a Zulip account now.

nikomatsakis (Dec 02 2019 at 23:06, on Zulip):

I know that if you do RUSTFLAGS=-Zself-profile you'll get a separate data file for each crate you build

nikomatsakis (Dec 02 2019 at 23:06, on Zulip):

( cc @Yehuda Katz )

Yehuda Katz (Dec 02 2019 at 23:06, on Zulip):

hello!

nikomatsakis (Dec 02 2019 at 23:06, on Zulip):

but is there a tool that combines the output from multiple crates to give you an overall view? At least I think that is what @Yehuda Katz is asking for

Yehuda Katz (Dec 02 2019 at 23:06, on Zulip):

what I'm trying to do is figure out what parts of nu are contributing to build slowness

simulacrum (Dec 02 2019 at 23:07, on Zulip):

You can take a look at -Ztimings in Cargo which gives a super high level view (timings per crate, split into 'metadata' and 'codegen')

nikomatsakis (Dec 02 2019 at 23:07, on Zulip):

There is also this tool that @Eric Huss wrote, I think?

nikomatsakis (Dec 02 2019 at 23:07, on Zulip):

Ah, maybe that's it

simulacrum (Dec 02 2019 at 23:08, on Zulip):

but that won't help at all if you're interested in "which functions in this crate are causing slowness"

nikomatsakis (Dec 02 2019 at 23:08, on Zulip):

I will just reiterate that man do we need some docs to point people at on this

simulacrum (Dec 02 2019 at 23:08, on Zulip):

that we can't answer ~at all today

Yehuda Katz (Dec 02 2019 at 23:08, on Zulip):

I'm first interested in which deps

Yehuda Katz (Dec 02 2019 at 23:08, on Zulip):

that's good enough for now

simulacrum (Dec 02 2019 at 23:08, on Zulip):

okay, then -Ztimings is your best bet for deps I think today

Yehuda Katz (Dec 02 2019 at 23:08, on Zulip):

will try it out

simulacrum (Dec 02 2019 at 23:08, on Zulip):

should be a matter of cargo build -Ztimings (or any other cargo command that builds things, in theory)

simulacrum (Dec 02 2019 at 23:09, on Zulip):

that'll print out a message at the end pointing at an HTML file which gives you a couple graphs and is generally helpful for this question

Yehuda Katz (Dec 02 2019 at 23:09, on Zulip):

sadly there's currently a nightly regression that's making my build not work but I can wait for that to be reverted

nikomatsakis (Dec 02 2019 at 23:10, on Zulip):

you could also pop back to an earlier nightly, @Yehuda Katz

simulacrum (Dec 02 2019 at 23:10, on Zulip):

(or use an earlier nightly -- this functionality has been in and pretty stable for a month or two)

Yehuda Katz (Dec 02 2019 at 23:10, on Zulip):

I'm not totally sure which one works

simulacrum (Dec 02 2019 at 23:10, on Zulip):

cargo-bisect-rustc can help there, in theory :)

Yehuda Katz (Dec 02 2019 at 23:10, on Zulip):

yeah fair

Yehuda Katz (Dec 02 2019 at 23:11, on Zulip):

/me installs

nikomatsakis (Dec 02 2019 at 23:11, on Zulip):

cargo-bisect-rustc can help there, in theory :)

heh interesting application

simulacrum (Dec 02 2019 at 23:11, on Zulip):

https://github.com/rust-lang/cargo-bisect-rustc

Yehuda Katz (Dec 02 2019 at 23:11, on Zulip):

yah

Yehuda Katz (Dec 02 2019 at 23:11, on Zulip):

installing!

Yehuda Katz (Dec 02 2019 at 23:12, on Zulip):

I never quite have any idea what I'm doing :P

Wesley Wiser (Dec 02 2019 at 23:13, on Zulip):

Very soon, you'll be able to do something similar with the measureme data. @mw just released measureme 0.5 this morning and we need to update it in rustc.

Yehuda Katz (Dec 02 2019 at 23:14, on Zulip):

how does measureme work?

Wesley Wiser (Dec 02 2019 at 23:14, on Zulip):

At that point, the crox tool can generate the same kind of visualization as cargo timings but more detailed.

Wesley Wiser (Dec 02 2019 at 23:15, on Zulip):

We have hooks in the compiler (mostly around the query system but also a few other areas) which record "interesting" events. The data is dumped in a binary format to some files in your working directory. Then after the compilation session, we have set of tools you can use to inspect the files.

Wesley Wiser (Dec 02 2019 at 23:17, on Zulip):

Once measureme@0.5 lands in rustc, you'll be able to generate performance traces like this https://github.com/rust-lang/measureme/pull/84#issuecomment-552688721

Wesley Wiser (Dec 02 2019 at 23:18, on Zulip):

I think @mw is opening a PR tomorrow so we should be able to do that in nightlies starting later this week.

Last update: Dec 12 2019 at 00:50UTC