Stream: t-compiler/shrinkmem-sprint

Topic: sprint day 4 chat


view this post on Zulip pnkfelix (Mar 04 2021 at 03:47):

I forgot to set up a casual chat channel Wednesday, so here's an early one for Thursday

view this post on Zulip rylev (Mar 04 2021 at 10:17):

FYI: I've attempted to add similar stats to rusage on Windows: https://github.com/rust-lang/rust/pull/82754

view this post on Zulip Santiago Pastorino (Mar 04 2021 at 14:24):

was doing the mir_opt_level work, #82736 is up and already r+

view this post on Zulip pnkfelix (Mar 04 2021 at 14:27):

rylev said:

FYI: I've attempted to add similar stats to rusage on Windows: https://github.com/rust-lang/rust/pull/82754

Hey @rylev, do you have any inkling as to how accurate the values look that you get out? Do they seem plausible?

view this post on Zulip Santiago Pastorino (Mar 04 2021 at 14:27):

moved the rest of the messages to #t-compiler/shrinkmem-sprint > mir_opt_level changes

view this post on Zulip rylev (Mar 04 2021 at 17:13):

@pnkfelix sorry didn't see this message. Yes they seem accurate. Linux reports roughly double the memory usage, but that could plausibly be explained by the linker invocations not being in the Windows stats

view this post on Zulip pnkfelix (Mar 04 2021 at 17:45):

Or it could be chalked up to differences in the underlying allocators. When you say “Linux reports …”, is that from perf? or local builds? (I’m mostly curious whether part of this could be chalked up to jemalloc …)

view this post on Zulip simulacrum (Mar 04 2021 at 17:46):

well, glibc or musl or whatever vs. windows system allocator is likely pretty different too

view this post on Zulip pnkfelix (Mar 04 2021 at 17:46):

right.

view this post on Zulip pnkfelix (Mar 04 2021 at 17:47):

arguably it would be good to do all these measurements with jemalloc linked in. But wait, do we even support jemalloc on Windows anyway?

view this post on Zulip simulacrum (Mar 04 2021 at 17:47):

no, not really, just linux

view this post on Zulip pnkfelix (Mar 04 2021 at 17:47):

(that is, I’m suggesting it would be useful as a way to eliminate one source of variation)

view this post on Zulip pnkfelix (Mar 04 2021 at 17:47):

Linux and Mac OS X, no?

view this post on Zulip simulacrum (Mar 04 2021 at 17:47):

(and really probably just x86_64-unknown-linux-gnu)

view this post on Zulip simulacrum (Mar 04 2021 at 17:48):

I think macOS turned out to not actually enable it? I forget, there's a recent thread

view this post on Zulip pnkfelix (Mar 04 2021 at 17:48):

Hmm I had thought the builders linked in jemalloc on Mac OS X.

view this post on Zulip pnkfelix (Mar 04 2021 at 17:48):

Oh that’s right, that recent thread, I forgot.

view this post on Zulip simulacrum (Mar 04 2021 at 17:48):

they try for sure

view this post on Zulip nagisa (Mar 04 2021 at 17:50):

There's a PR to actually enable jemalloc on macs (https://github.com/rust-lang/rust/pull/82642)

view this post on Zulip oli (Mar 04 2021 at 18:08):

I made a PR which shrinks max memory usage a bit: https://github.com/rust-lang/rust/pull/82727 any takers?

view this post on Zulip nagisa (Mar 04 2021 at 18:15):

how are you telling its coming from LLVM optimizing things?

view this post on Zulip nagisa (Mar 04 2021 at 18:15):

(it could be within LLVM if you're looking at bootsrap timings, but not otherwise)

view this post on Zulip pnkfelix (Mar 04 2021 at 18:32):

oli said:

I made a PR which shrinks max memory usage a bit: https://github.com/rust-lang/rust/pull/82727 any takers?

you may want to separate out the tidy changes. At least, github "hide whitespace changes" is missing a lot of cases I would have expected to be treated as pure-whitespace

view this post on Zulip Santiago Pastorino (Mar 04 2021 at 19:25):

oli said:

I made a PR which shrinks max memory usage a bit: https://github.com/rust-lang/rust/pull/82727 any takers?

looks great to me

view this post on Zulip Santiago Pastorino (Mar 04 2021 at 19:26):

agreed with Felix, there's a lot of rustfmt changes that may be better on a separate commit for easier reading

view this post on Zulip oli (Mar 05 2021 at 09:11):

Oh... I didn't see those... I did the cranelift thing afterwards and forgot to check the PR diff again

view this post on Zulip oli (Mar 05 2021 at 09:36):

nagisa said:

how are you telling its coming from LLVM optimizing things?

I looked at the detailed log of inflate-opt full: https://perf.rust-lang.org/detailed-query.html?commit=fe1d496e550e5902514bf91cb314f7ffa8813ec6&base_commit=7f32f62aa5ceba1b795f3702e502d8473238be6b&benchmark=inflate-opt&run_name=full and it looks like typeck got faster, but LLVM balances LTO speed against LLVM passes speed


Last updated: Oct 21 2021 at 20:47 UTC