FYI I've started some work on upgrading on LLVM 10 in https://github.com/rust-lang/rust/pull/67759. Unfortunately things look quite bleak where compile-time is concerned.
should we rebase on 9.0.1 in the meantime?
As a heads up, it looks like the wasm dwarf patches have landed in LLVM 10, so we won't need to rebase those
@Nikita Popov can we get a PR with the latest LLVM 10 compatibility changes? I guess performance is still a concern for updating the submodule, but I will probably want to move rustc to LLVM 10 in Fedora soon anyway.
I can cherry-pick into a new PR myself, if you like
@cuviper I've opened https://github.com/rust-lang/rust/pull/70163 with everything but the submodule update.
And yeah, I didn't have much luck figuring out what the compile-time issue is. If anyone else could take a poke at that, it would be great.
On the positive side, I got somewhat pissed about the the criminal negligence that LLVM has towards compile-time and have set up http://llvm-compile-time-tracker.com/ over the weekend. Maybe it will help to identify issues early.
Thanks! I'll give that a spin locally and then
so how hard would it be to retroactively fill that website with data from the LLVM 10 dev cycle to maybe narrow down the regression?
$ git log --oneline release/9.x..release/10.x | wc -l 17808
seems hard to retroactively do all of that
maybe it could be done like a bisection?
but there's probably not one thing to zero in on -- more likely a lot of performance paper cuts
Could it be bisected?
Looks like we are not the only ones suffering from LLVM getting slower:
Is it a viable path to just skip LLVM 10 altogether and hope that LLVM 11 has better perf?
i.e. is that something we could actually consider, or does it have other technical problems?
sure, I see no technical problem with that
the rust repo supports LLVM 10 now
Well there's the fact that debuginfo for RISC-V is pretty broken in LLVM 9... It's fixed in LLVM 10.
LLVM tends to optimise more and more. It's more likely LLVM 11 will be even slower than LLVM 10 than the opposite.
There is fairly recent comparison from Clang 6 to Clang 10(pre-release): https://openbenchmarking.org/result/1912190-HU-COREI759690 just hit ctrl+f and search for "Time To Compile".
More detailed Clang 9 vs Clang 10 (final): https://www.phoronix.com/scan.php?page=article&item=clang-10-benchmarks&num=4
Hmm... well that's troubling trend.
Is there a way to turn off llvm optimizations selectively? e.g. only keep optimizations that were enabled in llvm 9? then we could incrementally turn on the optimizations we want
Things are looking pretty good for LLVM 11 actually. We're down about 7-8% since I started measuring. Of course, no idea how well that will translate to Rust, which generates significantly different IR than clang.
@Nikita Popov congrats on being almost at the top of HN!
… and top now.
@Nikita Popov Your blog post is excellent! And thanks for all your hard work. This is excellent :)
@Nikita Popov Hi Nikita, would like to ask that how can i help to incorporate Aarch64 TME extensions. Or is there a plan? I've checked out locally but I don't see any incorporations at the https://github.com/rust-lang/llvm-project/tree/rustc/10.0-2020-05-05
From what I see, TME support was added in https://github.com/llvm/llvm-project/commit/a36d31478c182903523e04eb271bbf102bfab2cc, which is part of LLVM 10.
Yes, after the upgrade I will open a pr against stdarch I presume.
@Nikita Popov FWIW I updated the container to 20.04 locally and it did indeed still fail, the error was a giant wall of text though but it did seem somewhat related to the lack of
@Alex Crichton Thanks for testing that. I've now worked around the issue by passing CMAKE_CXX_STANDARD from our side, though LLD should really be setting this itself. Fingers crossed that this is not going to be a long whack-a-mole again...
@Nikita Popov btw once this lands, mind gc'ing the old 10.0 branches which ended up getting unused?
@Nikita Popov hey wanna try to help debug this issue here?
may be faster than going through comments on a PR
the raw log of a build after updating to 20.04 indicates that this is the host build of LLVM, so not related to cross-compiles yet
I'm manually going into the container and reexecuting the
c++ commands rather than updating the source
I threw on
-H which apparently prints out header paths
and nothing comes from the host LLVM (if any), so all #include is coming from the build directory for LLVM
/checkout/obj/build/x86_64-unknown-linux-gnu/llvm/build/include/llvm/IR/IntrinsicEnums.inc included as well yeah
oh dear, I think I mixed up my build directories
ok I'm blowing things away and restarting
By the way, how did you manage to run the image in the first place? For me it gets stuck at "Configuring tzdata", because it expects user input.
ENV DEBIAN_FRONTEND noninteractive
right above the apt-get line
it's a bad hack :(
Thanks, that did it
-std=c++11 flag was coming from
but I'm pretty sure I built an older LLVM then updated the submodule and didn't rebuild
basically I got the submodules mixed up
Ah, so just updating the image might be sufficient after all?
it may be yeah
I'm re-running the build to confirm/deny that
I'm not certain I mixed up the submodules, but my submodule was definitely at not the commit I expected
I am now certain, however, it is correct
ok freebsd build of llvm is making progres
@Nikita Popov ok so I was wrong, updating to ubuntu 20.04 does not fix the llvm build
I must have accidentally switched back
it's the same error as before
I think this has to do with the freebsd toolchain itself
and how we assemble it
so I think we should just disable it and move on
cc'ing folks who might know how to rebuild it
@Alex Crichton I managed to get it to build with this patch: https://gist.github.com/nikic/1200fed33696d1622a4ec1f523d2e651
This uses LLVMs own vector implementation instead of the libc++ one, which is probably too old for that target.
I can apply this to our LLVM fork for now and open an issue for updating the freebsd toolchain. Though also happy to just disable the freebsd builds, if that's preferred.
@Nikita Popov nice! That seems like a fine patch to drop in
it's not even something we use
or, well maybe we do ship that nowadays
if that fixes the build, it's probably best to land that patch but also open an issue and cc freebsd folks
it seems harmless enough to carry for now but if we can fix the underlying problem that'd be best since this will likely crop up again
Yeah. Old distros versions used in CI are a recurring problem with LLVM upgrades. I've opened https://github.com/rust-lang/rust/issues/72390 but not sure who to ping for FreeBSD. Do you happen to know anyone offhand?
Ah sorry I don't know anyone offhand, I'd just git blame the toolchain script and cc the folks there
It should be possible to update the distros at any time, it just requires an explicit opt-in which requires some work
on FreeBSD side, I think that Alan Somers (@asomers on github) is still active on rust work. Else, he should be able to ping the right person.
Alex Crichton said:
Nikita Popov btw once this lands, mind gc'ing the old 10.0 branches which ended up getting unused?
There's one unused branch at https://github.com/rust-lang/llvm-project/tree/rustc/10.0-2020-02-05 (all the other ones were on my fork). I tried deleting it, but the repo uses branch protection, so it was rejected.
@Nikita Popov oh oops good point, should be deleted now!