I'd like to consider shipping
rustc as a single statically linked executable. This would mean that compiler plugins and compiler backends (as they exist today) would no longer be supported without a custom built compiler. I'm wondering who the users of these features are.
I think we could still make plugins works by creating proxy methods that take TLS state from the main executable and set it on the plugin.
cc @bjorn3 since you might care about this =P
Thanks for pinging me. What is the benifit of completely statically linking rustc? Removing support for hotplug codegen backends would mean that I would have to either build a rustc with support, or I have to integrate cg_clif into rustc, in which case I would have to somehow hack bootstrap to only build a libstd with the resulting compiler, not a whole rustc. In either case the development cycle for cg_cluf gets increased significantly. Also removing librustc_driver.so would significantly increase the size of rustdoc, miri, clippy and other projects using rustc as library.
Pretty much just performance. It could use ThinLTO, a executable-only TLS model, it would lower startup costs by not having a dylib filled with symbols.
But I do think we could keep hotplug codegen stuff working by just transferring TLS state when calling into the dylib. I haven't come up with a case that wouldn't work with yet
@Zoxc I think it would be worthwhile to resolve how much of a performance benefit it'd be. If it's 1-2% I think the costs are probably not worth it.
ThinLTO alone gives -4% on
syntex_syntax (probably more if LLVM used it too). I'm waiting on https://github.com/rust-lang/rust/pull/67759 to do a perf run.
I'm confused -- can't we already do ThinLTO across most of the compiler? We only ship 1-2 shared libraries today for the compiler, so those could participate in ThinLTO today -- we don't need static linking for that?
I feel like I was talking about this with somebody recently, probably @Alex Crichton? I think we came to the conclusion that @simulacrum suggests -- that we could apply ThinLTO (and PGO) to rustc without going all the way to static linking by having a large library.
Plugins are used by at least servo for their custom GC lint. I think we should devise an alternative for that at some point, but it's not been high on the priority list.
Yes, and we already have that today, at least as far as I can tell. Presumably enabling thinlto and pgo is a matter of ci time and some legwork in bootstrap
In fact I feel like we already do it in some cases
Yes I agree that we do not need a statically linked compiler it the only purpose is to get PGO/LTO/etc benefits, we should start out by applying those techniques to librustc_driver.so and go from there
so maybe we can just open an issue to track the work that needs to be done here? I did a brief search and it doesn't look like any existing issue covers the desiderata here?
We don't support ThinLTO for dylibs. Not sure if there's any technical limitations or if it's just not implemented. cc @mw
I don't know either. The check to not support ThinLTO for Rust dylibs was in there from before I got to work with the code. IIRC, compiling the LLVM backend dylib with xLTO worked fine though.
I removed the check for dylibs + LTO and added
each_linked_rlib and it seems to just work. I'll do a perf run when LLVM 10 lands.