Stream: t-compiler/help

Topic: testing wasm target


tmandry (Jul 09 2019 at 21:29, on Zulip):

Are there any instructions for how to run rustc tests on a wasm32 target? I'm trying to replicate a bors failure

simulacrum (Jul 09 2019 at 21:37, on Zulip):

@tmandry Do you have Docker?

tmandry (Jul 09 2019 at 21:37, on Zulip):

yes

simulacrum (Jul 09 2019 at 21:37, on Zulip):

If so, then it should be a ./src/ci/docker/run.sh test-various

simulacrum (Jul 09 2019 at 21:38, on Zulip):

It loosely looks like that's the bors test builder (amongst other things) for wasm

simulacrum (Jul 09 2019 at 21:38, on Zulip):

you can edit the Dockerfile in src/ci/docker/test-various/Dockerfile to make it more targeted for your use case

tmandry (Jul 09 2019 at 21:38, on Zulip):

^ was about to ask :)

simulacrum (Jul 09 2019 at 21:38, on Zulip):

(if another builder failed the same should apply, presuming it's a Linux builder -- the macOS and windows builders don't use container)

simulacrum (Jul 09 2019 at 21:39, on Zulip):

(fwiw, I can try and take a look at the failure too if you have a link handy)

tmandry (Jul 09 2019 at 21:40, on Zulip):

I think it might be an actual compiler bug

tmandry (Jul 09 2019 at 21:40, on Zulip):

but it doesn't manifest on linux targets

simulacrum (Jul 09 2019 at 21:41, on Zulip):

this will cache after the initial build by the way (we don't delete the obj directory, which is where it'll build)

tmandry (Jul 09 2019 at 21:42, on Zulip):

great, running the docker build now

tmandry (Jul 09 2019 at 21:42, on Zulip):

thanks so much :D

simulacrum (Jul 10 2019 at 01:26, on Zulip):

@tmandry FWIW (just got the email notification from your comment) this looks like the standard output if an assert fails in wasm

simulacrum (Jul 10 2019 at 01:26, on Zulip):

and I would suspect that this might simply be because of size differences, so maybe try running on e.g. i686-unknown-linux-gnu? That'll be easier to debug at least

simulacrum (Jul 10 2019 at 01:26, on Zulip):

there's docker containers that'll do it if you want to avoid the hassle of installing libs yourself

simulacrum (Jul 10 2019 at 01:27, on Zulip):

(not sure if this is at all useful, or how far along you are, so feel free to ignore)

tmandry (Jul 10 2019 at 01:30, on Zulip):

@simulacrum here's the error: https://github.com/rust-lang/rust/pull/62270#issuecomment-508938503

tmandry (Jul 10 2019 at 01:31, on Zulip):

it's hitting an unreachable block somehow..

tmandry (Jul 10 2019 at 01:31, on Zulip):

oh I just re-read your chat

simulacrum (Jul 10 2019 at 01:32, on Zulip):

yeah, we essentially have -Cpanic=abort on wasm I believe because there's not really unwinding IIRC

tmandry (Jul 10 2019 at 01:32, on Zulip):

okay, that's really helpful to know, thanks :smiley:

simulacrum (Jul 10 2019 at 01:32, on Zulip):

except it's ... less aborty somehow

simulacrum (Jul 10 2019 at 01:33, on Zulip):

It might be helpful to run this in e.g. chrome

simulacrum (Jul 10 2019 at 01:33, on Zulip):

or firefox, w/e

simulacrum (Jul 10 2019 at 01:33, on Zulip):

and use F12 to debug

simulacrum (Jul 10 2019 at 01:33, on Zulip):

you can probably google for how to load wasm files in JS into memory or (maybe) copy what the NodeJS script is doing

tmandry (Jul 10 2019 at 01:33, on Zulip):

yeah, I started doing that but didn't very far

simulacrum (Jul 10 2019 at 01:34, on Zulip):

if you want to paste the wasm file I can try to do so? I think I've done something similar before

tmandry (Jul 10 2019 at 01:34, on Zulip):

(what I actually used was node's inspect mode)

tmandry (Jul 10 2019 at 01:36, on Zulip):

@simulacrum a.wasm if you want to take a look

simulacrum (Jul 10 2019 at 01:36, on Zulip):

thanks, downloaded

simulacrum (Jul 10 2019 at 01:37, on Zulip):

(FWIW, Zoom natively has a file attachments feature)

simulacrum (Jul 10 2019 at 01:37, on Zulip):

although that might be it... I somehow thought that was an s3 URL

tmandry (Jul 10 2019 at 01:37, on Zulip):

so does zulip, which I used :p

simulacrum (Jul 10 2019 at 01:37, on Zulip):

er, yes, I meant zulip ... typo

tmandry (Jul 10 2019 at 01:38, on Zulip):

otherwise, I can try i686 (are there official containers for doing this with rust?)

simulacrum (Jul 10 2019 at 01:39, on Zulip):

yeah, you can do the same thing as you did with wasm if you'd like

simulacrum (Jul 10 2019 at 01:39, on Zulip):

ah, let me track down the builder for you

simulacrum (Jul 10 2019 at 01:39, on Zulip):

src/ci/docker/i686-gnu/Dockerfile and src/ci/docker/run.sh i686-gnu

tmandry (Jul 10 2019 at 01:40, on Zulip):

that's so great

tmandry (Jul 10 2019 at 01:40, on Zulip):

was it always this easy?

simulacrum (Jul 10 2019 at 01:42, on Zulip):

this was done in ~2017(?) I think? Whenever we moved from buildbot to travis I believe

tmandry (Jul 10 2019 at 01:43, on Zulip):

for some reason I always thought it was more complicated /shrug

simulacrum (Jul 10 2019 at 01:52, on Zulip):

it does look like the wasm has just an unconditional panic

simulacrum (Jul 10 2019 at 01:52, on Zulip):

fwiw

simulacrum (Jul 10 2019 at 01:52, on Zulip):
  (func $_ZN26async_fn_size_moved_locals4main17h2d699e92deec06acE (;4;)
    (local $var0 i32)
    get_global $global0
    i32.const 64
    i32.sub
    tee_local $var0
    set_global $global0
    get_local $var0
    i32.const 1028
    i32.store offset=12
    get_local $var0
    i32.const 1048748
    i32.store offset=56
    get_local $var0
    i32.const 52
    i32.add
    i32.const 4
    i32.store
    get_local $var0
    i32.const 36
    i32.add
    i32.const 2
    i32.store
    get_local $var0
    i32.const 4
    i32.store offset=44
    get_local $var0
    i64.const 3
    i64.store offset=20 align=4
    get_local $var0
    i32.const 1048660
    i32.store offset=16
    get_local $var0
    get_local $var0
    i32.const 60
    i32.add
    i32.store offset=48
    get_local $var0
    get_local $var0
    i32.const 56
    i32.add
    i32.store offset=40
    get_local $var0
    get_local $var0
    i32.const 12
    i32.add
    i32.store offset=60
    get_local $var0
    get_local $var0
    i32.const 40
    i32.add
    i32.store offset=32
    get_local $var0
    i32.const 16
    i32.add
    i32.const 1048752
    call $_ZN3std9panicking15begin_panic_fmt17h52d189e875ddbafaE
    unreachable
simulacrum (Jul 10 2019 at 01:54, on Zulip):

unless I'm misinterpreting of course

simulacrum (Jul 10 2019 at 01:55, on Zulip):

anyway, I need to run, but I suspect that the unconditional panic here (if I understood the wasm correctly) means that this would always panic, which leads me to believe that the test might just fail on i686 too

simulacrum (Jul 10 2019 at 01:56, on Zulip):

I used this in an html file to load the wasm for what it's worth so I could step through in firefox's debugger ```
<script>
var importObject = {
imports: {
imported_func: function(arg) {
console.log(arg);
}
}
};
request = new XMLHttpRequest();
request.open('GET', 'a.wasm');
request.responseType = 'arraybuffer';
request.send();

request.onload = function() {
var bytes = request.response;
WebAssembly.instantiate(bytes, importObject).then(results => {
results.instance.exports.main();
});
};
</script>


simulacrum (Jul 10 2019 at 02:58, on Zulip):

@tmandry

---- async_fn_size::entry_main output ----
    error output:
        panicked at 'assertion failed: `(left == right)`
          left: `1032`,
         right: `1028`', src/lib.rs:101:5

        Stack:

        Error
            at module.exports.__wbg_new_59cb74e423758ede (/home/mark/async-fn-size/target/wasm32-unknown-unknown/wbg-tmp/wasm-bindgen-test.js:726:30)
            at console_error_panic_hook::Error::new::h6ee3a5f0b6230b19 (wasm-function[1044]:34)
            at console_error_panic_hook::hook_impl::hab2142e7c8799646 (wasm-function[311]:100)
            at console_error_panic_hook::hook::h65e11320dbd63f4d (wasm-function[1985]:38)
            at core::ops::function::Fn::call::h81e0e990fcfadc08 (wasm-function[1816]:45)
            at std::panicking::rust_panic_with_hook::h44a386ec93168747 (wasm-function[475]:265)
            at std::panicking::continue_panic_fmt::hf5de305382b63f0d (wasm-function[1187]:116)
            at std::panicking::begin_panic_fmt::hd34a9b77b3de5f46 (wasm-function[1476]:95)
            at async_fn_size::entry_main::h165c5c04e59afc05 (wasm-function[54]:1179)
            at core::ops::function::FnOnce::call_once::hfd971c972cb74b22 (wasm-function[2140]:22)



    JS exception that was thrown:
        RuntimeError: unreachable
            at __rust_start_panic (wasm-function[2313]:33)
            at rust_panic (wasm-function[2243]:31)
            at std::panicking::rust_panic_with_hook::h44a386ec93168747 (wasm-function[475]:304)
            at std::panicking::continue_panic_fmt::hf5de305382b63f0d (wasm-function[1187]:116)
            at std::panicking::begin_panic_fmt::hd34a9b77b3de5f46 (wasm-function[1476]:95)
            at async_fn_size::entry_main::h165c5c04e59afc05 (wasm-function[54]:1179)
            at core::ops::function::FnOnce::call_once::hfd971c972cb74b22 (wasm-function[2140]:22)
            at wasm_bindgen_test::__rt::Context::execute_sync::{{closure}}::h7f716071de880aef (wasm-function[1558]:22)
            at futures::future::lazy::Lazy<F,R>::get::h5a6ee61d14078cdd (wasm-function[116]:397)
            at <futures::future::lazy::Lazy<F,R> as futures::future::Future>::poll::he5890d83765bfd36 (wasm-function[1805]:38)
simulacrum (Jul 10 2019 at 02:59, on Zulip):

runs fine with:

    assert_eq!(1028, std::mem::size_of_val(&single()));
    assert_eq!(1028, std::mem::size_of_val(&single_with_noop()));
    assert_eq!(3080, std::mem::size_of_val(&joined()));
    assert_eq!(3080, std::mem::size_of_val(&joined_with_noop()));
simulacrum (Jul 10 2019 at 03:00, on Zulip):

though those values do fail on x86_64 linux, so maybe just ignore the test on wasm32?

simulacrum (Jul 10 2019 at 03:01, on Zulip):

no idea why they're different on wasm

simulacrum (Jul 10 2019 at 03:09, on Zulip):

https://github.com/rust-lang/rust/pull/62270#issuecomment-509889491

tmandry (Jul 10 2019 at 03:22, on Zulip):

@simulacrum sorry, I stepped away but thanks so much for debugging this!

simulacrum (Jul 10 2019 at 03:23, on Zulip):

No worries! Let me know if you want me to explain process or anything

tmandry (Jul 10 2019 at 03:26, on Zulip):

did you get that error message by loading a page with the script you pasted?

tmandry (Jul 10 2019 at 03:27, on Zulip):

it seems like it should be printed to the console when running node, but it's not

simulacrum (Jul 10 2019 at 03:34, on Zulip):

I added a dependency

simulacrum (Jul 10 2019 at 03:35, on Zulip):

[lib]
crate-type = ["cdylib", "rlib"]

[dependencies]
console_error_panic_hook = "0.1.6"
wasm-bindgen-test = "0.2"

simulacrum (Jul 10 2019 at 03:36, on Zulip):

and used wasm-pack test --node after annotating the main function:

#[wasm_bindgen_test]
#[cfg_attr(test, test)]
pub fn entry_main() {
    std::panic::set_hook(Box::new(console_error_panic_hook::hook));

    assert_eq!(1028, std::mem::size_of_val(&single()));
    assert_eq!(1028, std::mem::size_of_val(&single_with_noop()));
    assert_eq!(3080, std::mem::size_of_val(&joined()));
    assert_eq!(3080, std::mem::size_of_val(&joined_with_noop()));
}
simulacrum (Jul 10 2019 at 03:36, on Zulip):

@tmandry ^

tmandry (Jul 10 2019 at 03:38, on Zulip):

:o

tmandry (Jul 10 2019 at 03:38, on Zulip):

very nice

tmandry (Jul 10 2019 at 03:39, on Zulip):

it would be nice to have that crate in our build. I'll file an issue tomorrow

simulacrum (Jul 10 2019 at 03:42, on Zulip):

@tmandry https://github.com/Mark-Simulacrum/async-fn-size

simulacrum (Jul 10 2019 at 03:42, on Zulip):

(just for reference, but that's probably not the best way to do things)

simulacrum (Jul 10 2019 at 03:43, on Zulip):

feel free to ping me further, but I'm off to bed for now :)

tmandry (Jul 10 2019 at 03:44, on Zulip):

perfect :) I opened #62556 and linked to that from there

simulacrum (Jul 10 2019 at 03:45, on Zulip):

Should definitely be possible. If I'm waiting for things to compile I might take a look :)

Last update: Nov 11 2019 at 23:20UTC