Stream: t-compiler

Topic: cranelift backend work


nikomatsakis (Feb 07 2020 at 13:38, on Zulip):

Hey @bjorn3 -- so I keep hearing about all the great work you've been doing on a cranelift backend off in a side branch. I'm curious to hear more about the status and benefits -- i.e., we've thought that cranelift might give us "very fast" debug builds, is that true? It might be worth thinking about bringing that work in tree, no? I was pondering whether we ought to suggest a design meeting to talk it over.

bjorn3 (Feb 07 2020 at 13:55, on Zulip):

Currently a fair amount of programs compile and run. Unfortunately multithreaded program's don't yet work. (bjorn3/rustc_codegen_cranelift#388) It is currently blocked on getting my Cranelift PR for TLS support (bytecodealliance/cranelift#1174) merged.

As for the benefits: The goal is indeed to give much faster debug builds. I recently ran the debug builds of the rustc-perf suite: bjorn3/rustc_codegen_cranelift#878. For clean builds the improvements can be up to 60%, though it is sometimes slower (dylib loading time?) For incremental builds the build times are almost always slower, as no codegen units are stored in the incremental cache yet.

I am definitively interested in bringing it in tree, though I think the Cranelift PR should be merged first. A design meeting would be nice. I don't have a clear path forward apart from getting multithreading support though.

bjorn3 (Feb 07 2020 at 13:59, on Zulip):

work you've been doing on a cranelift backend off in a side branch.

I am actually developing it in a separate repo without any relation to rust-lang/rust (bjorn3/rustc_codegen_cranelift). I use the hotplug codegen backend api to load it into an unmodified rustc. This saves me a lot of compilation time and rebase work.

nikomatsakis (Feb 07 2020 at 14:20, on Zulip):

That's awesome

matklad (Feb 07 2020 at 14:27, on Zulip):

I wonder if bjorn3 and me could one day accidentally create an alternative Rust universe that doesn't use rustc...

Florian Diebold (Feb 07 2020 at 14:57, on Zulip):

one that lazily JIT-compiles each function separately as it's being run? :dream:

bjorn3 (Feb 07 2020 at 15:27, on Zulip):

one that lazily JIT-compiles each function separately as it's being run? :dream:

That shouldn't be too hard. cg_clif already has JIT support. It does currently compile all functions at the same time, but it should be possible to compile everything lazy. Maybe I will implement it today :)

Florian Diebold (Feb 07 2020 at 15:30, on Zulip):

I think the rest of the compiler isn't quite lazy enough yet, but that's already pretty awesome :)

bjorn3 (Feb 12 2020 at 16:28, on Zulip):

/me is trying to implement this lazy jit compilation, but gdb got a SIGSEGV :(

bjorn3 (Feb 12 2020 at 16:34, on Zulip):

I know at least something that I did wrong, but still gdb should not crash.

bjorn3 (Feb 12 2020 at 16:34, on Zulip):

I tried to find a debug package for gdb to get a useful backtrace, but the latest is for debian jessie.

bjorn3 (Feb 12 2020 at 16:35, on Zulip):

This is all I got:

#0  0x000055555584f857 in ?? ()
#1  0x0000555555851d52 in ?? ()
#2  0x000055555585170f in ?? ()
#3  0x000055555585170f in ?? ()
#4  0x000055555570ca2d in ?? ()
#5  0x000055555570cabe in ?? ()
#6  0x000055555561d8d3 in ?? ()
#7  0x000055555574c563 in ?? ()
#8  0x000055555568b029 in linux_ptrace_test_ret_to_nx_instr ()
#9  0x00005555557ac8f5 in ?? ()
#10 0x000055555574cc42 in ?? ()
#11 0x00005555557569a5 in ?? ()
#12 0x000055555576ce4d in ?? ()
#13 0x000055555576d04f in ?? ()
#14 0x000055555576d0f5 in ?? ()
#15 0x000055555576698b in ?? ()
#16 0x00005555557635ff in ?? ()
#17 0x0000555555767b16 in ?? ()
#18 0x00005555555ec898 in ?? ()
#19 0x00007ffff735509b in __libc_start_main () from /lib/x86_64-linux-gnu/libc.so.6
#20 0x00005555555ed48a in ?? ()
bjorn3 (Feb 12 2020 at 17:34, on Zulip):

I now got it working a bit. It now _only_ crashes after a few calls.

bjorn3 (Feb 12 2020 at 17:50, on Zulip):

I think I found the problem: I didn't implement returning from the function shim yet.

bjorn3 (Feb 12 2020 at 17:53, on Zulip):

:tada: :tada: :tada:

Rustc codegen cranelift will JIT run the executable, because the SHOULD_RUN env var is set
New Instance { def: Item(DefId(0:34 ~ mini_core_hello_world[317d]::start[0])), substs: [()] } @ 0x7f110d0a6000
New Instance { def: Item(DefId(0:51 ~ mini_core_hello_world[317d]::main[0])), substs: [] } @ 0x7f110d190000
New Instance { def: Item(DefId(0:48 ~ mini_core_hello_world[317d]::take_unique[0])), substs: [] } @ 0x7f110d17e000
New Instance { def: Item(DefId(0:47 ~ mini_core_hello_world[317d]::take_f32[0])), substs: [] } @ 0x7f110d183000
New Instance { def: Item(DefId(0:50 ~ mini_core_hello_world[317d]::call_return_u128_pair[0])), substs: [] } @ 0x7f110d195000
New Instance { def: Item(DefId(0:49 ~ mini_core_hello_world[317d]::return_u128_pair[0])), substs: [] } @ 0x7f110d19a000
Hello printf
Hello
World!
New Instance { def: DropGlue(DefId(1:228 ~ mini_core[8787]::drop_in_place[0]), Some(mini_core::Box<dyn SomeTrait>)), substs: [mini_core::Box<dyn SomeTrait>] } @ 0x7f110d19f000
New Instance { def: DropGlue(DefId(1:228 ~ mini_core[8787]::drop_in_place[0]), None), substs: [&str] } @ 0x7f110d1a4000
New Instance { def: Item(DefId(1:248 ~ mini_core[8787]::box_free[0])), substs: [dyn SomeTrait] } @ 0x7f110d1a9000
New Instance { def: Item(DefId(0:12 ~ mini_core_hello_world[317d]::{{impl}}[1]::object_safe[0])), substs: [] } @ 0x7f110d1ae000
abc
New Instance { def: Item(DefId(0:61 ~ mini_core_hello_world[317d]::main[0]::zeroed[0])), substs: [(u8, u8)] } @ 0x7f110d1b3000
Existing @ 0x7f110d19f000
New Instance { def: DropGlue(DefId(1:228 ~ mini_core[8787]::drop_in_place[0]), Some(NoisyDrop)), substs: [NoisyDrop] } @ 0x7f110cc3c000
New Instance { def: Item(DefId(0:19 ~ mini_core_hello_world[317d]::{{impl}}[2]::drop[0])), substs: [] } @ 0x7f110cc41000
Boxed outer got dropped!
New Instance { def: DropGlue(DefId(1:228 ~ mini_core[8787]::drop_in_place[0]), Some(NoisyDropInner)), substs: [NoisyDropInner] } @ 0x7f110cc46000
New Instance { def: Item(DefId(0:21 ~ mini_core_hello_world[317d]::{{impl}}[3]::drop[0])), substs: [] } @ 0x7f110cc4b000
Inner got dropped!
Existing @ 0x7f110d1a9000
New Instance { def: DropGlue(DefId(1:228 ~ mini_core[8787]::drop_in_place[0]), Some([NoisyDropInner; 2])), substs: [[NoisyDropInner; 2]] } @ 0x7f110cc55000
Existing @ 0x7f110cc4b000
Inner got dropped!
Existing @ 0x7f110cc4b000
Inner got dropped!
New Instance { def: ClosureOnceShim { call_once: DefId(1:222 ~ mini_core[8787]::FnOnce[0]::call_once[0]) }, substs: [[closure@example/mini_core_hello_world.rs:237:17: 237:28], ((),)] } @ 0x7f110cc5a000
New Instance { def: Item(DefId(0:860 ~ mini_core_hello_world[317d]::main[0]::{{closure}}[1])), substs: [i8, extern "rust-call" fn(((),)) -> u8] } @ 0x7f110cc5f000
New Instance { def: Item(DefId(0:859 ~ mini_core_hello_world[317d]::check_niche_behavior[0])), substs: [] } @ 0x7f110d09a000
New Instance { def: Item(DefId(0:8 ~ mini_core_hello_world[317d]::{{impl}}[0]::report[0])), substs: [] } @ 0x7f110cc64000
bjorn3 (Feb 12 2020 at 17:54, on Zulip):

You can find it at the wip_lazy_jit branch.

bjorn3 (Feb 12 2020 at 18:09, on Zulip):

I hope to do some clean-up tomorrow to then create a PR.

Last update: Feb 25 2020 at 02:55UTC