Stream: t-compiler/wg-self-profile

Topic: postprocessing costs and file sizes


mw (May 22 2019 at 14:01, on Zulip):

I did some testing with the latest versions of compiler and summarize

mw (May 22 2019 at 14:02, on Zulip):

for style-servo, the events file is ~140 megabytes

mw (May 22 2019 at 14:02, on Zulip):

running summarize on it takes 0.65 seconds on my machine

Wesley Wiser (May 22 2019 at 14:07, on Zulip):

Cool!

Wesley Wiser (May 22 2019 at 14:07, on Zulip):

I wonder if there's any value in compressing the events file?

mw (May 22 2019 at 14:07, on Zulip):

yeah, seems pretty reasonable

mw (May 22 2019 at 14:08, on Zulip):

depends on whether you want to store it somewhere, I'd say

mw (May 22 2019 at 14:08, on Zulip):

at the moment, the plan is not to store it, right?

Wesley Wiser (May 22 2019 at 14:08, on Zulip):

For perf.rlo?

mw (May 22 2019 at 14:08, on Zulip):

yes

Wesley Wiser (May 22 2019 at 14:08, on Zulip):

No, I don't believe we're going to store it beyond the time required to process it

mw (May 22 2019 at 14:09, on Zulip):

another question would be when to compress it

mw (May 22 2019 at 14:09, on Zulip):

i.e. we don't want to do that in the rustc process

mw (May 22 2019 at 14:09, on Zulip):

because we are going to great lengths to keep the overhead low already

Wesley Wiser (May 22 2019 at 14:10, on Zulip):

Yeah, that's true

Wesley Wiser (May 22 2019 at 14:11, on Zulip):

I seem to recall reading somewhere about a Linux filesystem that always compresses data using a really simple algorithm (zlib maybe?). The reason being that the CPU cost to compress the data was less than the cost of reading/writing more data off disk

Wesley Wiser (May 22 2019 at 14:11, on Zulip):

But since we're writing the data async with mmap, it probably doesn't pay off for us

mw (May 22 2019 at 14:13, on Zulip):

I think ZFS has the option of lz4 compressing data natively

Wesley Wiser (May 22 2019 at 14:15, on Zulip):

Ah, yes that's probably what I'm thinking of https://blogs.oracle.com/solaris/zfs-compression-a-win-win-v2

Wesley Wiser (May 22 2019 at 14:16, on Zulip):

But since we don't wait for the writes to happen or read the data back in process, it probably won't pay off for us

mw (May 22 2019 at 14:16, on Zulip):

yeah

mw (May 22 2019 at 14:17, on Zulip):

we would need to experiment but I'd like to keep it simple if possible

mw (May 22 2019 at 14:18, on Zulip):

so stackcollapse is even faster than summarize: 0.48 secs

Wesley Wiser (May 22 2019 at 14:18, on Zulip):

If we did want to keep the files around, we could definitely compress them after the perf run. I've seen pretty significant space reductions just from the native compression tool in macOS

mw (May 22 2019 at 14:18, on Zulip):

crox is slower but still acceptable: 3.14 secs

mw (May 22 2019 at 14:18, on Zulip):

yes, compressing the file gives me:

mw (May 22 2019 at 14:19, on Zulip):

lz4: 41 MB

mw (May 22 2019 at 14:19, on Zulip):

gzip: 22.4 MB

Wesley Wiser (May 22 2019 at 14:19, on Zulip):

I think this will likely improve crox speed https://github.com/rust-lang/measureme/issues/47

mw (May 22 2019 at 14:19, on Zulip):

xz: 15.4 MB

Wesley Wiser (May 22 2019 at 14:19, on Zulip):

And cut the output file size by half

mw (May 22 2019 at 14:20, on Zulip):

yes, I also suspect that most of the time is spent generating the string data

mw (May 22 2019 at 14:20, on Zulip):

since summarize also looks at all the data, but is much faster

mw (May 22 2019 at 14:22, on Zulip):

one thing I noticed here: having only the PID, but not the crate name in the file name is annoying

mw (May 22 2019 at 14:22, on Zulip):

I've been generating events for all crates in the crate graph

Wesley Wiser (May 22 2019 at 14:22, on Zulip):

Yeah, there's likely some low hanging fruit in optimizing most of the processing tools

Wesley Wiser (May 22 2019 at 14:22, on Zulip):

I haven't done any profiling of any of them

mw (May 22 2019 at 14:23, on Zulip):

wow, the crox output is 550 MB :)

Wesley Wiser (May 22 2019 at 14:23, on Zulip):

That's probably going to break Chrome lol

Wesley Wiser (May 22 2019 at 14:23, on Zulip):

At a certain point, the Chrome dev tool just gives up and doesn't display anything.

Wesley Wiser (May 22 2019 at 14:24, on Zulip):

There's an internal Chrome tool which handles large files better but the UI is a bit clunky

mw (May 22 2019 at 14:27, on Zulip):

yeah...

mw (May 22 2019 at 14:27, on Zulip):

I'd say it is unlikely that we generate the Chrome profiling data eagerly for each perf run

mw (May 22 2019 at 14:28, on Zulip):

given how big the files are

Wesley Wiser (May 22 2019 at 14:29, on Zulip):

Yeah even relatively small crates like regex generate large files

Wesley Wiser (May 22 2019 at 14:30, on Zulip):

(I believe chrome://tracing is the internal tool I'm thinking of but I don't have a profiler file handy to test)

Wesley Wiser (May 22 2019 at 14:31, on Zulip):

The Firefox format looks like it scales much better to me but I haven't had a chance to play around with it

mw (May 22 2019 at 14:32, on Zulip):

yeah

mw (May 22 2019 at 14:32, on Zulip):

well, not a priority for the MVP anyway, I'd say

mw (May 22 2019 at 14:32, on Zulip):

are you working on something?

Wesley Wiser (May 22 2019 at 14:33, on Zulip):

It's been a busy week for me but I think I'll have time over the next few days to work on getting perf.rlo to run -Z self-profile as we discussed

mw (May 22 2019 at 14:33, on Zulip):

cool, I'll look into adding a path argument to -Zself-profile

mw (May 22 2019 at 14:33, on Zulip):

if that doesn't clash with what you're doing

Wesley Wiser (May 22 2019 at 14:33, on Zulip):

Nope!

mw (May 22 2019 at 14:34, on Zulip):

alright

Wesley Wiser (May 22 2019 at 14:34, on Zulip):

That should be easy to tweak later

Wesley Wiser (May 25 2019 at 02:00, on Zulip):

Ok, so I've modified rustc-perf to run with -Z self-profile and dump all of the files in a folder. Executing a full perf run results in ~1400 files totaling 4.4 gb. Note: these are the raw profile files, not processed results file.

cc @simulacrum

simulacrum (May 27 2019 at 01:27, on Zulip):

seems.. reasonable?

simulacrum (May 27 2019 at 01:29, on Zulip):

@Wesley Wiser We'll probably want to process the data -- if you get the bit of code which correctly finds/loads the selected crate's into measureme that'd be a good start

simulacrum (May 27 2019 at 01:29, on Zulip):

I'm not sure how that should be done though

Wesley Wiser (May 27 2019 at 01:30, on Zulip):

Did we decide which results we should keep? Should I just keep the first iteration's results for now?

simulacrum (May 27 2019 at 01:30, on Zulip):

yeah

simulacrum (May 27 2019 at 01:31, on Zulip):

we wanted some form of "aggregation" but that's not really the MVP

Wesley Wiser (May 27 2019 at 01:31, on Zulip):

Yeah

Wesley Wiser (May 27 2019 at 01:32, on Zulip):

I'll just take the first one for now unless there's any objections

simulacrum (May 27 2019 at 01:33, on Zulip):

sure seems fine -- I imagine it's just a call to first or something like that at a relatively top-level place

simulacrum (May 27 2019 at 01:33, on Zulip):

I would try to avoid making that decision in a lowlevel place if possible to make it easier to aggregate later (i.e., keep all the data around for now, right up until the end)

simulacrum (May 27 2019 at 01:33, on Zulip):

but it's not critical that happens so don't worry about it if its hard

Wesley Wiser (May 27 2019 at 01:34, on Zulip):

Got it

Wesley Wiser (May 27 2019 at 01:34, on Zulip):

Thanks!

Wesley Wiser (May 31 2019 at 11:45, on Zulip):

Can I assume that the measureme tools are going to be installed in a specific folder? Or available on the path?

Wesley Wiser (May 31 2019 at 11:45, on Zulip):

Basically, how do I know where to look to call the processing tools?

Wesley Wiser (Jun 04 2019 at 19:38, on Zulip):

@simulacrum Do you have any suggestions about how to call the tools?

simulacrum (Jun 04 2019 at 19:39, on Zulip):

hm, so there was some discussion about inclusion into the sysroot, but that seems far off

simulacrum (Jun 04 2019 at 19:40, on Zulip):

do we expect to need multiple versions or will latest version pretty much work?

Wesley Wiser (Jun 04 2019 at 19:40, on Zulip):

I think at this point, we should just expect the latest version to work

Wesley Wiser (Jun 04 2019 at 19:40, on Zulip):

Of course, how we get the latest version is another question

Wesley Wiser (Jun 04 2019 at 19:41, on Zulip):

But a recent-ish version should be fine

simulacrum (Jun 04 2019 at 19:41, on Zulip):

let's just assume it's installed somewhere in PATH

Wesley Wiser (Jun 04 2019 at 19:41, on Zulip):

Ok

Wesley Wiser (Jun 04 2019 at 19:41, on Zulip):

:+1:

simulacrum (Jun 04 2019 at 19:41, on Zulip):

(Same as we do for perf, valgrind, other tools)

Wesley Wiser (Jun 04 2019 at 19:41, on Zulip):

I guess there's a server somewhere you ssh into and install stuff as needed?

simulacrum (Jun 04 2019 at 19:41, on Zulip):

and then add to the benchmarking readme (IIRC, collector/README, or collector/benchmarks/README) some text about installing measureme

simulacrum (Jun 04 2019 at 19:42, on Zulip):

Yeah, we have two -- collection and site -- I presume the tools are needed on collection?

Wesley Wiser (Jun 04 2019 at 19:42, on Zulip):

Yeah

simulacrum (Jun 04 2019 at 19:43, on Zulip):

yeah, I'll be able to install them (can even stick a cargo install -f measureme in the script so that it reinstalls latest version every runthrough)

Wesley Wiser (Jun 04 2019 at 19:44, on Zulip):

Perfect

Wesley Wiser (Jun 04 2019 at 19:44, on Zulip):

Thanks!

Last update: Nov 17 2019 at 07:00UTC