Stream: t-compiler

Topic: cranelift backend work


nikomatsakis (Feb 07 2020 at 13:38, on Zulip):

Hey @bjorn3 -- so I keep hearing about all the great work you've been doing on a cranelift backend off in a side branch. I'm curious to hear more about the status and benefits -- i.e., we've thought that cranelift might give us "very fast" debug builds, is that true? It might be worth thinking about bringing that work in tree, no? I was pondering whether we ought to suggest a design meeting to talk it over.

bjorn3 (Feb 07 2020 at 13:55, on Zulip):

Currently a fair amount of programs compile and run. Unfortunately multithreaded program's don't yet work. (bjorn3/rustc_codegen_cranelift#388) It is currently blocked on getting my Cranelift PR for TLS support (bytecodealliance/cranelift#1174) merged.

As for the benefits: The goal is indeed to give much faster debug builds. I recently ran the debug builds of the rustc-perf suite: bjorn3/rustc_codegen_cranelift#878. For clean builds the improvements can be up to 60%, though it is sometimes slower (dylib loading time?) For incremental builds the build times are almost always slower, as no codegen units are stored in the incremental cache yet.

I am definitively interested in bringing it in tree, though I think the Cranelift PR should be merged first. A design meeting would be nice. I don't have a clear path forward apart from getting multithreading support though.

bjorn3 (Feb 07 2020 at 13:59, on Zulip):

work you've been doing on a cranelift backend off in a side branch.

I am actually developing it in a separate repo without any relation to rust-lang/rust (bjorn3/rustc_codegen_cranelift). I use the hotplug codegen backend api to load it into an unmodified rustc. This saves me a lot of compilation time and rebase work.

nikomatsakis (Feb 07 2020 at 14:20, on Zulip):

That's awesome

matklad (Feb 07 2020 at 14:27, on Zulip):

I wonder if bjorn3 and me could one day accidentally create an alternative Rust universe that doesn't use rustc...

Florian Diebold (Feb 07 2020 at 14:57, on Zulip):

one that lazily JIT-compiles each function separately as it's being run? :dream:

bjorn3 (Feb 07 2020 at 15:27, on Zulip):

one that lazily JIT-compiles each function separately as it's being run? :dream:

That shouldn't be too hard. cg_clif already has JIT support. It does currently compile all functions at the same time, but it should be possible to compile everything lazy. Maybe I will implement it today :)

Florian Diebold (Feb 07 2020 at 15:30, on Zulip):

I think the rest of the compiler isn't quite lazy enough yet, but that's already pretty awesome :)

bjorn3 (Feb 12 2020 at 16:28, on Zulip):

/me is trying to implement this lazy jit compilation, but gdb got a SIGSEGV :(

bjorn3 (Feb 12 2020 at 16:34, on Zulip):

I know at least something that I did wrong, but still gdb should not crash.

bjorn3 (Feb 12 2020 at 16:34, on Zulip):

I tried to find a debug package for gdb to get a useful backtrace, but the latest is for debian jessie.

bjorn3 (Feb 12 2020 at 16:35, on Zulip):

This is all I got:

#0  0x000055555584f857 in ?? ()
#1  0x0000555555851d52 in ?? ()
#2  0x000055555585170f in ?? ()
#3  0x000055555585170f in ?? ()
#4  0x000055555570ca2d in ?? ()
#5  0x000055555570cabe in ?? ()
#6  0x000055555561d8d3 in ?? ()
#7  0x000055555574c563 in ?? ()
#8  0x000055555568b029 in linux_ptrace_test_ret_to_nx_instr ()
#9  0x00005555557ac8f5 in ?? ()
#10 0x000055555574cc42 in ?? ()
#11 0x00005555557569a5 in ?? ()
#12 0x000055555576ce4d in ?? ()
#13 0x000055555576d04f in ?? ()
#14 0x000055555576d0f5 in ?? ()
#15 0x000055555576698b in ?? ()
#16 0x00005555557635ff in ?? ()
#17 0x0000555555767b16 in ?? ()
#18 0x00005555555ec898 in ?? ()
#19 0x00007ffff735509b in __libc_start_main () from /lib/x86_64-linux-gnu/libc.so.6
#20 0x00005555555ed48a in ?? ()
bjorn3 (Feb 12 2020 at 17:34, on Zulip):

I now got it working a bit. It now _only_ crashes after a few calls.

bjorn3 (Feb 12 2020 at 17:50, on Zulip):

I think I found the problem: I didn't implement returning from the function shim yet.

bjorn3 (Feb 12 2020 at 17:53, on Zulip):

:tada: :tada: :tada:

Rustc codegen cranelift will JIT run the executable, because the SHOULD_RUN env var is set
New Instance { def: Item(DefId(0:34 ~ mini_core_hello_world[317d]::start[0])), substs: [()] } @ 0x7f110d0a6000
New Instance { def: Item(DefId(0:51 ~ mini_core_hello_world[317d]::main[0])), substs: [] } @ 0x7f110d190000
New Instance { def: Item(DefId(0:48 ~ mini_core_hello_world[317d]::take_unique[0])), substs: [] } @ 0x7f110d17e000
New Instance { def: Item(DefId(0:47 ~ mini_core_hello_world[317d]::take_f32[0])), substs: [] } @ 0x7f110d183000
New Instance { def: Item(DefId(0:50 ~ mini_core_hello_world[317d]::call_return_u128_pair[0])), substs: [] } @ 0x7f110d195000
New Instance { def: Item(DefId(0:49 ~ mini_core_hello_world[317d]::return_u128_pair[0])), substs: [] } @ 0x7f110d19a000
Hello printf
Hello
World!
New Instance { def: DropGlue(DefId(1:228 ~ mini_core[8787]::drop_in_place[0]), Some(mini_core::Box<dyn SomeTrait>)), substs: [mini_core::Box<dyn SomeTrait>] } @ 0x7f110d19f000
New Instance { def: DropGlue(DefId(1:228 ~ mini_core[8787]::drop_in_place[0]), None), substs: [&str] } @ 0x7f110d1a4000
New Instance { def: Item(DefId(1:248 ~ mini_core[8787]::box_free[0])), substs: [dyn SomeTrait] } @ 0x7f110d1a9000
New Instance { def: Item(DefId(0:12 ~ mini_core_hello_world[317d]::{{impl}}[1]::object_safe[0])), substs: [] } @ 0x7f110d1ae000
abc
New Instance { def: Item(DefId(0:61 ~ mini_core_hello_world[317d]::main[0]::zeroed[0])), substs: [(u8, u8)] } @ 0x7f110d1b3000
Existing @ 0x7f110d19f000
New Instance { def: DropGlue(DefId(1:228 ~ mini_core[8787]::drop_in_place[0]), Some(NoisyDrop)), substs: [NoisyDrop] } @ 0x7f110cc3c000
New Instance { def: Item(DefId(0:19 ~ mini_core_hello_world[317d]::{{impl}}[2]::drop[0])), substs: [] } @ 0x7f110cc41000
Boxed outer got dropped!
New Instance { def: DropGlue(DefId(1:228 ~ mini_core[8787]::drop_in_place[0]), Some(NoisyDropInner)), substs: [NoisyDropInner] } @ 0x7f110cc46000
New Instance { def: Item(DefId(0:21 ~ mini_core_hello_world[317d]::{{impl}}[3]::drop[0])), substs: [] } @ 0x7f110cc4b000
Inner got dropped!
Existing @ 0x7f110d1a9000
New Instance { def: DropGlue(DefId(1:228 ~ mini_core[8787]::drop_in_place[0]), Some([NoisyDropInner; 2])), substs: [[NoisyDropInner; 2]] } @ 0x7f110cc55000
Existing @ 0x7f110cc4b000
Inner got dropped!
Existing @ 0x7f110cc4b000
Inner got dropped!
New Instance { def: ClosureOnceShim { call_once: DefId(1:222 ~ mini_core[8787]::FnOnce[0]::call_once[0]) }, substs: [[closure@example/mini_core_hello_world.rs:237:17: 237:28], ((),)] } @ 0x7f110cc5a000
New Instance { def: Item(DefId(0:860 ~ mini_core_hello_world[317d]::main[0]::{{closure}}[1])), substs: [i8, extern "rust-call" fn(((),)) -> u8] } @ 0x7f110cc5f000
New Instance { def: Item(DefId(0:859 ~ mini_core_hello_world[317d]::check_niche_behavior[0])), substs: [] } @ 0x7f110d09a000
New Instance { def: Item(DefId(0:8 ~ mini_core_hello_world[317d]::{{impl}}[0]::report[0])), substs: [] } @ 0x7f110cc64000
bjorn3 (Feb 12 2020 at 17:54, on Zulip):

You can find it at the wip_lazy_jit branch.

bjorn3 (Feb 12 2020 at 18:09, on Zulip):

I hope to do some clean-up tomorrow to then create a PR.

bjorn3 (Feb 26 2020 at 17:54, on Zulip):

I hope to do some clean-up tomorrow to then create a PR.

Haven't done this yet, but the Cranelift TLS PR got merged today, so I was able to merge bjorn3/rustc_codegen_cranelift#784, which makes it possible to run multi-threaded programs.

Yerkebulan Tulibergenov (Feb 26 2020 at 17:58, on Zulip):

@bjorn3 Thank you for your great work! I am so excited about the progress.
Do you have a guesstimate when it will be ready to compile stuff? Or pass rustc test suite if it’s easier for you to answer?

bjorn3 (Feb 26 2020 at 18:24, on Zulip):

Many things already work. The rustc test suite is a hard one to fully pass, as it checks the exact error message for errors happening during codegen, it tests many unstable features not yet implemented in cg_clif and there are also tests for the exact llvm ir and asm generated. I don't know how long it will take for most programs to compile.

bjorn3 (Feb 26 2020 at 18:26, on Zulip):

If you have any idea for an executable to test, commenting on bjorn3/rustc_codegen_cranelift#247 is appreciated. You don't have to test if it compiles and runs.

centril (Feb 26 2020 at 18:33, on Zulip):

@bjorn3 what if we moved the cg_clif backend into tree and then ignored certain UI tests that cranelift cannot handle yet?

centril (Feb 26 2020 at 18:33, on Zulip):

and then you can work on them one by one

bjorn3 (Feb 26 2020 at 18:49, on Zulip):

I rather like the fact that cg_clif is out of tree. It saves much disk space (no LLVM checkout and no rustc build artifacts) and it makes compilation much faster, as I can use a rustup installed nightly, instead of building it myself.

I do have a branch of cg_clif with the necessary commands to run the rustc test suite. It does removes the most tests depending on unimplemented features. I haven't updated it in a while, but I think I will rebass it soon.

Zoxc (Feb 26 2020 at 18:51, on Zulip):

I'm sure we can mark tests as LLVM specific without moving cg_clif in-tree

bjorn3 (Feb 26 2020 at 18:52, on Zulip):

https://github.com/bjorn3/rustc_codegen_cranelift/blob/2142b538d605b81a0176c9d11b7c77c79c4aa739/test.sh#L98

centril (Feb 26 2020 at 18:57, on Zulip):

@bjorn3 oh but don't you want to like have CI ensure no regressions in cg_clif at some point?

bjorn3 (Feb 26 2020 at 18:58, on Zulip):

Marking tests which can not possibly work on non-LLVM backends, like checking llvm ir or asm output, as such makes sense to me. Marking tests that require a feature not yet implemented in cg_clif though would mean that I have to make a PR every time I implement something new.

bjorn3 (Feb 26 2020 at 18:59, on Zulip):

centril said:

bjorn3 oh but don't you want to like have CI ensure no regressions in cg_clif at some point?

Yes, I run ./test.sh on Travis CI.

bjorn3 (Feb 26 2020 at 19:00, on Zulip):

Did you mean changes on the rustc side?

bjorn3 (Feb 26 2020 at 19:02, on Zulip):

If so, almost all breakages were either a changed api or a patch to libstd and friends not applying anymore. Any miscompilations were I believe my fault.

bjorn3 (Feb 26 2020 at 19:03, on Zulip):

There is just one case that a breakage was rustc's fault: printing every mir statement panicked, or rather still panics.

bjorn3 (Feb 26 2020 at 19:04, on Zulip):

#67558

bjorn3 (Feb 26 2020 at 19:07, on Zulip):

Found another: #64872, likely wouldn't have failed if cg_clif was in-tree though

centril (Feb 26 2020 at 19:11, on Zulip):

@bjorn3 yeah rustc side

centril (Feb 26 2020 at 19:11, on Zulip):

maybe it's too soon though

bjorn3 (Mar 12 2020 at 11:06, on Zulip):

I implemented incremental caching of object files yesterday. This drastically improved the results on the rustc-perf suite: https://github.com/bjorn3/rustc_codegen_cranelift/issues/878#issuecomment-597871730 There are still cases where cg_clif is a ~30-60% regression over cg_llvm though.

Wesley Wiser (Mar 12 2020 at 12:57, on Zulip):

This is really incredible work @bjorn3!

bjorn3 (Mar 12 2020 at 13:06, on Zulip):

Next step: do a run with -Zself-profile to find the cause of the remaining reds.

Wesley Wiser (Mar 12 2020 at 13:11, on Zulip):

If there's anything you need on the self-profile side, feel free to ping me or post in #t-compiler/wg-self-profile

bjorn3 (Mar 12 2020 at 13:18, on Zulip):

Just knowing how much time is spent during analysis vs codegen vs linking is probably enough in this case. I think the reds are partially caused by the linker having to process a larger libstd than the optimized libstd of cg_llvm. If I need anything more than that that is missing, I will post in #t-compiler/wg-self-profile.

centril (Mar 12 2020 at 13:20, on Zulip):

@bjorn3 I went ahead and added A-cranelift, https://github.com/rust-lang/rust/labels/A-cranelift

bjorn3 (Mar 12 2020 at 13:28, on Zulip):

Thanks!

centril (Mar 12 2020 at 13:29, on Zulip):

Working on sifting through the issues mentioning "cranelift" to see what to add to it :slight_smile:

bjorn3 (Mar 12 2020 at 13:33, on Zulip):

I have added the label to two issues.

bjorn3 (Mar 12 2020 at 13:38, on Zulip):

I don't think #55993 deserves the label. It isn't necessary for cg_clif, but is just deduplicating some code between rustc and Cranelift.

centril (Mar 12 2020 at 13:40, on Zulip):

removed that one; I've added the label to the ones I thought were relevant; have a look and see if some were erroneously added

bjorn3 (Mar 12 2020 at 13:44, on Zulip):

#55931 is an issue while compiling cg_clif itself, while #69924 is an issue while generating a backtrace during compilation with cg_clif. both should be reproducable without cg_clif. in the former case cg_clif is not executing, while in the later case cg_clif doesn't execute any unsafe code.

centril (Mar 12 2020 at 13:46, on Zulip):

ah, removed the label on those

bjorn3 (Mar 12 2020 at 13:55, on Zulip):

I have added the label to one more issue. I think the issue is now applied to all issues it should be applied to and not applied to issues it shouldn't be applied to.

nikomatsakis (Mar 12 2020 at 13:59, on Zulip):

@bjorn3 what wuld you think about doing a design meeting to discuss the work you've been doing on cranelift?

nikomatsakis (Mar 12 2020 at 13:59, on Zulip):

I can think of a number of different possible focuses

nikomatsakis (Mar 12 2020 at 14:00, on Zulip):
nikomatsakis (Mar 12 2020 at 14:00, on Zulip):
bjorn3 (Mar 12 2020 at 14:10, on Zulip):

I think cg_clif now supports enough programs since new multithreading support that working towards upstreaming this now makes some sense. (preferably as a submodule like miri and clippy)

@nikomatsakis I am interested in a design meeting.

bjorn3 (Mar 12 2020 at 17:33, on Zulip):

@nikomatsakis Did you miss my response due to the meeting happening at the same time?

nikomatsakis (Mar 12 2020 at 17:34, on Zulip):

@bjorn3 I did!

nikomatsakis (Mar 12 2020 at 17:34, on Zulip):

@bjorn3 can you open a proposal on compiler-team?

nikomatsakis (Mar 12 2020 at 17:35, on Zulip):

it doesn't need to be super detailed to start

nikomatsakis (Mar 12 2020 at 17:35, on Zulip):

it'd be great to do that before tomorrow though

nikomatsakis (Mar 12 2020 at 17:35, on Zulip):

ideally we'd also create a hackmd with some notes on status and some prompts for what meeting shoudl discuss, but we can hammer that out async too

bjorn3 (Mar 12 2020 at 17:35, on Zulip):

Sure will open a proposal.

centril (Mar 12 2020 at 17:36, on Zulip):

A submodule makes sense to start with; at some point we should consider moving everything in tree though

centril (Mar 12 2020 at 17:37, on Zulip):

A -Z cranelift flag might also make sense, both for rustc and cargo

bjorn3 (Mar 12 2020 at 17:38, on Zulip):

With the current infra that would be -Zcodegen-backend=cranelift

centril (Mar 12 2020 at 17:40, on Zulip):

@bjorn3 ah; that works, maybe we can make it more convenient cargo side

bjorn3 (Mar 12 2020 at 17:41, on Zulip):

Sure, it would also be nice to support the jit mode in cargo

bjorn3 (Mar 12 2020 at 19:41, on Zulip):

nikomatsakis said:

bjorn3 can you open a proposal on compiler-team?

Is https://gist.github.com/bjorn3/d77f1b5b3cc69575295aa3f931cac053 about right?

bjorn3 (Mar 12 2020 at 20:13, on Zulip):

@nikomatsakis I guess the above quote didn't count as a pingable mention?

nikomatsakis (Mar 12 2020 at 21:40, on Zulip):

@bjorn3 looks great :)

nikomatsakis (Mar 12 2020 at 21:40, on Zulip):

and indeed the "quote" doesn't ping

nikomatsakis (Mar 12 2020 at 21:40, on Zulip):

(in general you can do @_**Foo** to mention someone without pinging them)

bjorn3 (Mar 12 2020 at 21:53, on Zulip):

Opened rust-lang/compiler-team#257

simulacrum (Mar 12 2020 at 22:05, on Zulip):

@bjorn3 I think it would be good to get some agenda items for the meeting -- e.g., one question I might want to answer is whether it makes sense to land things in-tree, so that it is CI gated and such

bjorn3 (Mar 13 2020 at 11:18, on Zulip):

https://hackmd.io/@bjorn3/HJL5ryFS8

bjorn3 (Mar 13 2020 at 11:26, on Zulip):

Feedback is welcome

Last update: Jun 07 2020 at 09:05UTC