I am trying to get a sense of how much pain / time / effort is required each time Rust goes through an upgrade (not just a cherry picking a bug fix) of LLVM. Along with that, I am curious about the kinds of changes typically required to adapt to the new version of LLVM.
Looking over the docs in the rustc guide, it feels like quite an involved process that someone might be focused on full time for a while and seems to occur a few times per year for specific features and also to stay current with each LLVM release. Is most of the time spent adapting the Rust <-> LLVM interface for API changes, or is it more dominated by fighting surprise behavior changes in LLVM and finding workarounds?
I have several ideas for tooling to help with these LLVM upgrade tasks, but I want to check whether my assumptions match the experience of those who've worked through an upgrade in past for Rust.
I think we've not historically experienced much pain from LLVM upgrades, beyond breakage -- but that's generally on tier 2 or tier 3 platforms -- @cuviper and @Alex Crichton may be able to say more, I think they've been involved with the last couple upgrades
From what I remember 99% of the pain of an upgrade is fixing regressions in the ci builds we see through bors
The API changes are mostly quite smooth
Sometimes it's taken forever to narrow down issues
Other times it's been super smooth
I haven't been too too involved in the last one or two upgrades though, and I think they were relatively painless
(Don't know how relevant it is, but generally speaking, having frequent LLVM submodule bumps is undesirable because all the rust-lang/rust devs have to rebuild LLVM, which is painful)
It sounds like it's actually easier and less painful than I assumed then. :happy: (The text "Unlike bugfixes, updating to pick up a new feature of LLVM typically requires a lot more work." in the rustc guide led me to assume it was usually a long / arduous adventure.)
Good to know about the additional complexity of LLVM changes forcing rebuilds for rustc devs... I hadn't realised that particular impact of more frequent LLVM upgrades.
It's not really an impact that I think is too important, or at least I barely notice it. I do have decent hardware though :)
It's also something that we can (and should) fix, it's just that it requires some design work and the exact solution isn't completely clear.
I think we've not historically experienced much pain from LLVM upgrades, beyond breakage -- but that's generally on tier 2 or tier 3 platforms
FYI LLVM 9 upgrade broke tier1 targets when using non default options: #64609 #65618
For hunting down LLVM-related regressions, does that typically mean compiling many random revisions of LLVM locally to try to bisect the issue? (If so, it sounds slow since I imagine it takes a while to build at each bisection step.)
For me I typically extract ir first to get a standalone test I can reproduce with just llvm tools, then I reduce (sometimes also with llvm tools)
And then yeah bisection is typically next
Ah right, makes sense to reproduce in LLVM standalone where possible to simplify. LLVM has also added there own llvm-reduce tool for IR reduction recently, maybe that will be of some help here.
For LLVM standalone, there's also llvmlab bisect for accelerated bisection via a shared cache.
It's not really an impact that I think is too important, or at least I barely notice it. I do have decent hardware though :)
It's also something that we can (and should) fix, it's just that it requires some design work and the exact solution isn't completely clear.
LLVM builds take 30-60min on my hardware, which is not totally shabby either. Multiply this by 3 for 3 rustc checkouts (so that I can work on more than one PR at a time). It's a big pain.
in comparison, building one stage (so that I can run stage 1 tests) without having to build LLVM usually takes <20min.
in theory e.g. ccache (not sccache) would solve the multiply by 3