Stream: general

Topic: Rustc compilation times


Matthias Prechtl (Feb 09 2020 at 16:44, on Zulip):

Out of curiosity, how long does iterating on rustc take on your machine? I'm currently running time ./x.py test src/test/ui and i'm about 10 minutes in

Matthias Prechtl (Feb 09 2020 at 16:45, on Zulip):

I'm on a laptop though, so i'm not expecting anything exceptional

simulacrum (Feb 09 2020 at 17:00, on Zulip):

15-20 minutes on a ryzen 1800x (8 core/16 thread) for a build to stage 1 std

simulacrum (Feb 09 2020 at 17:01, on Zulip):

and that's with cached llvm

Matthias Prechtl (Feb 09 2020 at 18:43, on Zulip):

That's rough, on a desktop with a pretty powerful cpu... what do you do in the meantime? Context switch to another copy of rustc?

ecstatic-morse (Feb 09 2020 at 18:51, on Zulip):

@Matthias Prechtl Make sure you pass -i for incremental and --stage 1 to only build the compiler once (one or both of these might be the default for x.py test nowadays). Also, I often don't run the entire test suite locally, just directories that are likely to change. I usually rely on CI to run the full suite.

ecstatic-morse (Feb 09 2020 at 18:57, on Zulip):

Since I work mostly in rustc_mir, which is pretty late in the compilation pipeline, it only takes a few minutes for a build/test cycle.

ecstatic-morse (Feb 09 2020 at 18:58, on Zulip):

If you are working on the AST, things will be slower.

matprec (Feb 09 2020 at 19:00, on Zulip):

@ecstatic-morse thanks for the pointer, i'll try that one :) Curious why that's not the default?

ecstatic-morse (Feb 09 2020 at 19:02, on Zulip):

Not sure, maybe just intertia. You modify config.toml to ensure incremental is on by default (like I said, that may already be the case).

matprec (Feb 09 2020 at 19:02, on Zulip):
./x.py clean --llvm-skip-rebuild TRUE
time ./x.py build

Takes 26min 24s

matprec (Feb 09 2020 at 19:03, on Zulip):

On an i7-8550U

matprec (Feb 09 2020 at 19:04, on Zulip):

I'm actually kind of impressed, given that the r7 1800X from @simulacrum takes 15-20mins

matprec (Feb 09 2020 at 19:05, on Zulip):

I'll try the incremental version and post an update :)

ecstatic-morse (Feb 09 2020 at 19:05, on Zulip):

@matprec incremental will only be faster when you're making changes. The initial build will be slower

ecstatic-morse (Feb 09 2020 at 19:06, on Zulip):

Do make sure you're always using stage 1 though.

ecstatic-morse (Feb 09 2020 at 19:06, on Zulip):

For almost all changes you don't need a stage 2 build

matprec (Feb 09 2020 at 19:07, on Zulip):

Yeah that's expected :) is a newline change enough to make the incremental checker think it needs to rebuild?

matprec (Feb 09 2020 at 19:07, on Zulip):

Because then i'll likely try a "worst case" incremental build with a change in the parser

ecstatic-morse (Feb 09 2020 at 19:09, on Zulip):

Not sure :slight_smile: Feel free to try

ecstatic-morse (Feb 09 2020 at 19:11, on Zulip):

(I don't work on incremental, and x.py is a bit different than cargo so it might be relying on file timestamps)

ecstatic-morse (Feb 09 2020 at 19:13, on Zulip):

Oh, one more thing. Use x.py check, not x.py build when you're still actively developing something. It's quite a bit faster than building.

simulacrum (Feb 09 2020 at 19:15, on Zulip):

10min sometimes. Maybe I'm remembering wrong too :)

ecstatic-morse (Feb 09 2020 at 19:17, on Zulip):

@matprec One more one more thing. To build just the stage1 compiler with just the stage1 standard library artifacts, it's ./x.py build --stage 1 src/libstd

ecstatic-morse (Feb 09 2020 at 19:21, on Zulip):

If you add a custom toolchain to rustup (I call mine stage1), which points to the build directory containing the stage 1 compiler, you can run rustc +stage1 file.rs to test your compiler without worrying about x.py

ecstatic-morse (Feb 09 2020 at 19:22, on Zulip):

This means I can alternate between editing files and running individual tests without having to rebuild every time.

matprec (Feb 09 2020 at 19:33, on Zulip):
  1. Build incremental rustc
  2. Touch src/librustc_lexer/src/cursor.rs
  3. time ./x.py build -i --stage 1

Results in

real    3m15,065s
user    13m27,142s
sys     0m42,841s

:tada:

matprec (Feb 09 2020 at 19:33, on Zulip):

Thanks so much, your input has been extremely valuable @ecstatic-morse !

matprec (Feb 09 2020 at 19:36, on Zulip):

I just found https://rust-lang.github.io/rustc-guide/building/suggested.html :) Still crawling through the docs, there's just so much to absorb :D

Jonas Schievink (Feb 09 2020 at 19:37, on Zulip):

Yeah that guide is really helpful

Chris Simpkins (Feb 10 2020 at 18:46, on Zulip):

Is it acceptable to run tests with the -i flag? For instance, ./x.py test -i --stage 1 [path to test module]. I am 10 mins into the compile to run one test without -i... :)

Chris Simpkins (Feb 10 2020 at 18:56, on Zulip):

Significantly faster to run tests with -i. Cut my time down to ~1/3 of that without it

matprec (Feb 10 2020 at 19:41, on Zulip):

I'd say go for it, afaik incremental shouldn't affect the behaviour of the compiled rustc

matprec (Feb 10 2020 at 19:42, on Zulip):

@Chris Simpkins ^

Chris Simpkins (Feb 10 2020 at 19:47, on Zulip):

ty!

RalfJ (Feb 13 2020 at 08:55, on Zulip):

I have set incremental=true in my config.toml so I don't need to pass that flag manually all the time any more ;)

RalfJ (Feb 13 2020 at 08:55, on Zulip):

so, yes, definitely acceptable to set that flag for local testing

Chris Simpkins (Feb 13 2020 at 12:50, on Zulip):

Ah very nice! Thank you!

Last update: Feb 25 2020 at 04:20UTC