Stream: t-compiler

Topic: compiling the compiler statically


Zoxc (Feb 07 2020 at 03:56, on Zulip):

I'd like to consider shipping rustc as a single statically linked executable. This would mean that compiler plugins and compiler backends (as they exist today) would no longer be supported without a custom built compiler. I'm wondering who the users of these features are.

Zoxc (Feb 07 2020 at 05:07, on Zulip):

I think we could still make plugins works by creating proxy methods that take TLS state from the main executable and set it on the plugin.

Zoxc (Feb 07 2020 at 05:38, on Zulip):

cc @bjorn3 since you might care about this =P

bjorn3 (Feb 07 2020 at 05:46, on Zulip):

Thanks for pinging me. What is the benifit of completely statically linking rustc? Removing support for hotplug codegen backends would mean that I would have to either build a rustc with support, or I have to integrate cg_clif into rustc, in which case I would have to somehow hack bootstrap to only build a libstd with the resulting compiler, not a whole rustc. In either case the development cycle for cg_cluf gets increased significantly. Also removing librustc_driver.so would significantly increase the size of rustdoc, miri, clippy and other projects using rustc as library.

Zoxc (Feb 07 2020 at 05:55, on Zulip):

Pretty much just performance. It could use ThinLTO, a executable-only TLS model, it would lower startup costs by not having a dylib filled with symbols.

But I do think we could keep hotplug codegen stuff working by just transferring TLS state when calling into the dylib. I haven't come up with a case that wouldn't work with yet

simulacrum (Feb 07 2020 at 12:58, on Zulip):

@Zoxc I think it would be worthwhile to resolve how much of a performance benefit it'd be. If it's 1-2% I think the costs are probably not worth it.

Zoxc (Feb 07 2020 at 13:06, on Zulip):

ThinLTO alone gives -4% on syntex_syntax (probably more if LLVM used it too). I'm waiting on https://github.com/rust-lang/rust/pull/67759 to do a perf run.

simulacrum (Feb 07 2020 at 13:12, on Zulip):

I'm confused -- can't we already do ThinLTO across most of the compiler? We only ship 1-2 shared libraries today for the compiler, so those could participate in ThinLTO today -- we don't need static linking for that?

nikomatsakis (Feb 07 2020 at 13:36, on Zulip):

I feel like I was talking about this with somebody recently, probably @Alex Crichton? I think we came to the conclusion that @simulacrum suggests -- that we could apply ThinLTO (and PGO) to rustc without going all the way to static linking by having a large library.

nikomatsakis (Feb 07 2020 at 13:36, on Zulip):

Plugins are used by at least servo for their custom GC lint. I think we should devise an alternative for that at some point, but it's not been high on the priority list.

simulacrum (Feb 07 2020 at 13:37, on Zulip):

Yes, and we already have that today, at least as far as I can tell. Presumably enabling thinlto and pgo is a matter of ci time and some legwork in bootstrap

simulacrum (Feb 07 2020 at 13:38, on Zulip):

In fact I feel like we already do it in some cases

Alex Crichton (Feb 07 2020 at 13:49, on Zulip):

Yes I agree that we do not need a statically linked compiler it the only purpose is to get PGO/LTO/etc benefits, we should start out by applying those techniques to librustc_driver.so and go from there

pnkfelix (Feb 10 2020 at 19:41, on Zulip):

so maybe we can just open an issue to track the work that needs to be done here? I did a brief search and it doesn't look like any existing issue covers the desiderata here?

Zoxc (Feb 22 2020 at 17:48, on Zulip):

We don't support ThinLTO for dylibs. Not sure if there's any technical limitations or if it's just not implemented. cc @mw

mw (Feb 24 2020 at 10:40, on Zulip):

I don't know either. The check to not support ThinLTO for Rust dylibs was in there from before I got to work with the code. IIRC, compiling the LLVM backend dylib with xLTO worked fine though.

Zoxc (Feb 24 2020 at 23:42, on Zulip):

I removed the check for dylibs + LTO and added Dylib to each_linked_rlib and it seems to just work. I'll do a perf run when LLVM 10 lands.

Last update: Feb 25 2020 at 04:10UTC