Stream: general

Topic: rustc vs llvm


Gabriel Górski (Dec 31 2019 at 00:15, on Zulip):

Hey, I'm not sure whether it is right place to ask (I've seen that there's a wg-llvm channel but it's more of a discussion channel for a WG than a question one). So there it goes:

As far as I see, rustc uses libLLVM shared library as a compiler backend provider to compile to every target architecture allowed by LLVM. Thus, x86-64-unknown-linux allows me to (after downloading appropriate target std crate and so on via rustup target add ...) to compile for specific target. Where does this shared library live in *-windows-* toolchains? Both msvc and gnu toolchains have DLLs in bin directory for e.g. rustc-drivers (which I'd gladly love to learn what does it actually do) but there's no libLLVM library. It cannot be baked into rustc binary because it is too small. So where is it? Unless I got it wrong and it works completely in a different way.

Laurențiu Nicola (Jan 01 2020 at 09:25, on Zulip):

Hmm, maybe it's statically-linked into .rustup/toolchains/nightly-x86_64-pc-windows-msvc/lib/rustlib/x86_64-pc-windows-msvc/codegen-backends/rustc_codegen_llvm-llvm.dll?

Laurențiu Nicola (Jan 01 2020 at 09:26, on Zulip):

On a related note, why do I have both toolchains/nightly-x86_64-unknown-linux-gnu/lib/libLLVM-9-rust-1.41.0-nightly.so and toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libLLVM-9-rust-1.41.0-nightly.so?

Gabriel Górski (Jan 01 2020 at 14:06, on Zulip):

@Laurențiu Nicola

Hmm, maybe it's statically-linked into rustc_codegen_llvm-llvm.dll

That's possible. But firstly rustc_codegen_llvm-llvm.dll seems to be quite small in size - just 42 MB, where libLLVM.so weights almost 80 MB. Secondly, it is placed in a pretty weird directory - .rustup/toolchains/<toolchain>/lib/rustlib/<specific-target>/codegen-backends/ where specific target is x86_64-pc-windows-msvc. If it was backend for generating code for all targets, then why is it placed in a host target directory?

On a related note, why do I have both

That's what I noticed as well. Duplication of libraries is only partial, though. Weird.

Is there some kind of document explaining internal architecture of toolchains and their dependencies?

Laurențiu Nicola (Jan 01 2020 at 14:10, on Zulip):

Static linking might explain the size. Re docs, I suppose the place to look into would be https://rust-lang.github.io/rustc-guide/codegen/backend-agnostic.html, but it's not very helpful in this case. Maybe someone who's really familiar with how codegen works will chime in.

Gabriel Górski (Jan 01 2020 at 14:26, on Zulip):

Thanks for link. I agree that someone actively developing those bits would be invaluable.

Amanieu (Jan 01 2020 at 18:25, on Zulip):

@Gabriel Górski Since LLVM is statically linked, the linker can eliminate a lot of unused code from the final binary. On the other hand libLLVM.so needs to export the full LLVM API.

Gabriel Górski (Jan 01 2020 at 18:48, on Zulip):

That makes sense. So is in fact LLVM statically linked into rustc_codegen_llvm-llvm.dll? Why does linux toolchain differ so much from msvc one? Doesn't it make further maintaining of toolchains a bit harder? I mean I suppose there's valid reasoning behind decisions made or at least I have no expertise to decide there isn't.

Still, it would be really cool to know what are the exact responsibilities of included libraries/files and reasoning behind file hierarchy within .rustup directories. Is https://rust-lang.github.io/rustc-guide/ enough to find out? (had no time yet to dig into it)

simulacrum (Jan 01 2020 at 18:49, on Zulip):

to my knowledge rustc guide does not document the directory layout, no

simulacrum (Jan 01 2020 at 18:49, on Zulip):

(indeed I'm not sure there's any documentation beyond people's heads, in this area)

Gabriel Górski (Jan 01 2020 at 18:52, on Zulip):

Hmm I see.

beyond people's heads

I can imagine toolchains become a bit messy over time. I thought there's some kind of documented-ish convention :P

simulacrum (Jan 01 2020 at 18:54, on Zulip):

hm, not sure I follow

simulacrum (Jan 01 2020 at 18:55, on Zulip):

the layout of the rust distribution is pretty ad-hoc, but has remained relatively stable over time

Gabriel Górski (Jan 01 2020 at 19:01, on Zulip):

I'm not insinuating anything. If it works, it works. I'm just exploring insides of the ecosystem, that's all. I'd like to learn more about it. Rustc guide seems like a good starting point I guess

simulacrum (Jan 01 2020 at 19:01, on Zulip):

Sure! I think documenting the layout (not to stabilize it, but just for rustc devs) would be great.

simulacrum (Jan 01 2020 at 19:02, on Zulip):

Just no one has done so yet :)

Santiago Pastorino (Jan 02 2020 at 03:36, on Zulip):

to my knowledge rustc guide does not document the directory layout, no

I think it would be great to do so :)

Last update: Jan 21 2020 at 09:40UTC