Stream: general

Topic: Miri: no item to reborrow for Unique


Jake Goulding (May 06 2019 at 16:48, on Zulip):

Miri evaluation error: no item to reborrow for Unique from tag 3488 found in borrow stack

@RalfJ is this a "programmer did something bad" error or "Miri doesn't know this" error? I'm thinking the former.

RalfJ (May 06 2019 at 21:11, on Zulip):

yeah that's a "you got UB" error

Jake Goulding (May 07 2019 at 01:39, on Zulip):

@RalfJ would you mind helping me see where?

use std::collections::BTreeSet;

fn uniq_refs<'i, 'd: 'i, T>(
    data: &'d mut [T],
    indices: &'i BTreeSet<usize>,
) -> impl Iterator<Item = &'d mut T> + 'i {
    let in_bounds_indices = indices.range(0..data.len());

    in_bounds_indices.map(move |&i| unsafe {
        let r = data.get_unchecked_mut(i);
        let p: *mut T = r;
        &mut *p
    })
}

use std::iter::FromIterator;

fn main() {
    let mut scores = vec![1, 2, 3];
    let indices = BTreeSet::from_iter(vec![0, 2]);

    let selected_scores: Vec<_> = uniq_refs(&mut scores, &indices).collect();

    for score in selected_scores {
        *score += 1;
    }

    println!("{:?}", scores);
}
error[E0080]: Miri evaluation error: no item to reborrow for Unique from tag 3488 found in borrow stack
    --> /root/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libcore/mem.rs:1328:34
     |
1328 |         ManuallyDrop::into_inner(self.value)
     |                                  ^^^^^^^^^^ Miri evaluation error: no item to reborrow for Unique from tag 3488 found in borrow stack
     |
     = note: inside call to `std::mem::MaybeUninit::<&mut i32>::assume_init` at /root/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libcore/ptr.rs:578:5
     = note: inside call to `std::ptr::read::<&mut i32>` at /root/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/liballoc/vec.rs:2368:26
note: inside call to `<std::vec::IntoIter<&mut i32> as std::iter::Iterator>::next` at src/main.rs:24:18
    --> src/main.rs:24:18
     |
24   |     for score in selected_scores {
     |                  ^^^^^^^^^^^^^^^
     = note: inside call to `main` at /root/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/rt.rs:64:34
     = note: inside call to closure at /root/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/rt.rs:52:53
     = note: inside call to closure at /root/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/panicking.rs:293:40
     = note: inside call to `std::panicking::try::do_call::<[closure@DefId(1/1:1542 ~ std[82ff]::rt[0]::lang_start_internal[0]::{{closure}}[0]) 0:&dyn std::ops::Fn() -> i32 + std::marker::Sync + std::panic::RefUnwindSafe], i32>` at /root/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/panicking.rs:289:5
     = note: inside call to `std::panicking::try::<i32, [closure@DefId(1/1:1542 ~ std[82ff]::rt[0]::lang_start_internal[0]::{{closure}}[0]) 0:&dyn std::ops::Fn() -> i32 + std::marker::Sync + std::panic::RefUnwindSafe]>` at /root/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/panic.rs:388:9
     = note: inside call to `std::panic::catch_unwind::<[closure@DefId(1/1:1542 ~ std[82ff]::rt[0]::lang_start_internal[0]::{{closure}}[0]) 0:&dyn std::ops::Fn() -> i32 + std::marker::Sync + std::panic::RefUnwindSafe], i32>` at /root/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/rt.rs:52:25
     = note: inside call to `std::rt::lang_start_internal` at /root/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/rt.rs:64:5
     = note: inside call to `std::rt::lang_start::<()>`

playground

Jake Goulding (May 07 2019 at 01:41, on Zulip):

If I remove the Vec and iterate directly, Miri stays silent.

RalfJ (May 07 2019 at 08:43, on Zulip):

(Sorry, I am on a workshop and don't have time to dig into this right now -- I'll come back to you eventually.)

Jake Goulding (May 07 2019 at 12:24, on Zulip):

How very rude. /s :wink:

RalfJ (May 08 2019 at 14:49, on Zulip):

uh, you are doing illegal lifetime laundering here

RalfJ (May 08 2019 at 14:49, on Zulip):

casting &mut to *mut and back

RalfJ (May 08 2019 at 14:50, on Zulip):

and the reason it explodes is that every time you use data, you are asserting that this is a unique reference

RalfJ (May 08 2019 at 14:52, on Zulip):

This is basically equivalent to

fn fst(x: &mut (i32, i32)) -> &mut i32 {
    &mut x.0
}
fn snd(x: &mut (i32, i32)) -> &mut i32 {
    &mut x.0
}

fn main() {
    let p = &mut (0, 1);
    let ptr1 = fst(p);
    let ptr1 = unsafe { &mut *(ptr1 as *mut i32) }; // lifetime laundering
    let ptr2 = snd(p);
    let _val = *ptr1;
}
RalfJ (May 08 2019 at 14:52, on Zulip):

Calling snd creates a mutable reference overlapping with ptr1, thus invalidating ptr1

RalfJ (May 08 2019 at 14:56, on Zulip):

@Jake Goulding ^

Jake Goulding (May 08 2019 at 14:57, on Zulip):

You say lifetime laundering — is that precise?

Jake Goulding (May 08 2019 at 14:57, on Zulip):

Conceptually, I'm intending to do the same thing as e.g. split_at_mut

Jake Goulding (May 08 2019 at 14:58, on Zulip):

I want to get two disjoint mutable references to values in a slice.

RalfJ (May 08 2019 at 14:59, on Zulip):

you need to go through raw pointers then

Jake Goulding (May 08 2019 at 14:59, on Zulip):

Do I need to switch to getting a *mut T and use add instead of get_unchecked_mut?

RalfJ (May 08 2019 at 14:59, on Zulip):

and unfortunately we have no get on a *mut [T], so you need to use ptr arithmetic

RalfJ (May 08 2019 at 15:00, on Zulip):

Do I need to switch to getting a *mut T and use add instead of get_unchecked_mut?

yes

Jake Goulding (May 08 2019 at 15:00, on Zulip):

OK

    let start = data.as_mut_ptr();
    let in_bounds_indices = indices.range(0..data.len());

    in_bounds_indices.map(move |&i| unsafe { &mut *start.add(i) })
Jake Goulding (May 08 2019 at 15:00, on Zulip):

is Miri-clean

RalfJ (May 08 2019 at 15:00, on Zulip):

that looks reasonable

RalfJ (May 08 2019 at 15:00, on Zulip):

what does Miri say?

RalfJ (May 08 2019 at 15:01, on Zulip):

:+1:

Jake Goulding (May 08 2019 at 15:01, on Zulip):

nothing

RalfJ (May 08 2019 at 15:01, on Zulip):

Cc @Gankro -- another case of mutable refs being more ergonomic causing trouble

RalfJ (May 08 2019 at 15:01, on Zulip):

well actually the code got IMO nicer with raw ptrs but maybe I am weird^^

Jake Goulding (May 08 2019 at 15:02, on Zulip):

Yeah, it's not worse, certainly

Jake Goulding (May 08 2019 at 15:03, on Zulip):

every time you use data, you are asserting that this is a unique reference

So because the element was a &mut T and still live, and then I tried to call a fn(&mut [T]) that contained that same element, that caused aliasing?

Jake Goulding (May 08 2019 at 15:03, on Zulip):

Very similar to the linked list PR that you submitted and I "reviewed"?

Jake Goulding (May 08 2019 at 15:04, on Zulip):

mutable refs being more ergonomic

I'd say that my main reason was trying to get out of unsafe as fast as possible.

RalfJ (May 08 2019 at 15:04, on Zulip):

every time you use data, you are asserting that this is a unique reference

So because the element was a &mut T and still live, and then I tried to call a fn(&mut [T]) that contained that same element, that caused aliasing?

yes

Jake Goulding (May 08 2019 at 15:06, on Zulip):

Someday I'll get it!

Jake Goulding (May 08 2019 at 15:07, on Zulip):

And the iterator-only version didn't cause UB because the &mut T wasn't live when we made the subsequent call to get_unchecked_mut

RalfJ (May 08 2019 at 15:07, on Zulip):

not sure what that version was, but yes -- if you dont use the invalidated ptr again later, there's no issue

Jake Goulding (May 08 2019 at 15:51, on Zulip):

not sure what that version was

https://play.integer32.com/?version=stable&mode=debug&edition=2018&gist=26ae035045e89cd54b72d62413e4f601

Jake Goulding (May 08 2019 at 16:05, on Zulip):

Is there any way for Miri to point at the two (or more?) offending locations? If it pointed at data.get_unchecked_mut(i); I might have had more luck fixing this without consulting The Great Oracle

RalfJ (May 08 2019 at 18:27, on Zulip):

@Jake Goulding that would be https://github.com/rust-lang/miri/issues/531

RalfJ (May 08 2019 at 18:28, on Zulip):

but I have no idea how to implement that even remotely efficiently. looks like it requires remembering items when they get removed from the stack, and storing a stacktrace of where that happened. for every single item. that's crazy.

Jake Goulding (May 08 2019 at 18:32, on Zulip):

"easy" — just run the program backwards once you find the problem :-)

lqd (May 08 2019 at 19:07, on Zulip):

yay rr in miri, the time travelling interpreter :)

oli (May 08 2019 at 20:04, on Zulip):

priroda can do this

oli (May 08 2019 at 20:05, on Zulip):

(with horrible perf)

oli (May 08 2019 at 20:06, on Zulip):

uh yea or we just build a tagging system where you figure out the memory location relevant to the error and "just" record all accesses to that memory location

oli (May 08 2019 at 20:06, on Zulip):

then show them to the user in reverse order

Last update: Nov 22 2019 at 00:10UTC