Stream: t-lang/wg-unsafe-code-guidelines

Topic: miri "trying to reborrow for Unique"


Simon Sapin (Feb 20 2020 at 06:53, on Zulip):

Hi. When running cargo miri test on my library I get error: Miri evaluation error: trying to reborrow for Unique, but parent tag <160163> does not have an appropriate item in the borrow stack but I don’t see how to find the root cause. The full output is at https://paste.mozilla.org/iVboPkXU, the code at https://github.com/servo/html5ever/tree/304c155bce19f956f0641cce3a1e19bc3b7bdaa1/zbuf.

Simon Sapin (Feb 20 2020 at 06:57, on Zulip):

The error happens when trying to take &mut reference to something inside of a heap allocation made through std::alloc::alloc. The struct that contains the ptr::NonNull is created on the stack with let mut and then used with methods that take &mut self

Simon Sapin (Feb 20 2020 at 06:59, on Zulip):

(Also, “reborrow for Unique” means &mut not std::ptr::Unique, right?)

bjorn3 (Feb 20 2020 at 09:02, on Zulip):

Try passing -Zmiri-track-pointer-tag=160163 to track where it is invalidated.

bjorn3 (Feb 20 2020 at 09:03, on Zulip):

Simon Sapin said:

(Also, “reborrow for Unique” means &mut not std::ptr::Unique, right?)

Yes

Simon Sapin (Feb 20 2020 at 10:12, on Zulip):

The output looks the same with cargo +nightly miri test -- -Zmiri-track-pointer-tag=160163. Or is it a RUSTFLAGS?

bjorn3 (Feb 20 2020 at 12:44, on Zulip):

I don't know. You can try using a RUSTFLAG.

Simon Sapin (Feb 20 2020 at 12:59, on Zulip):

error: unknown debugging option: `miri-track-pointer-tag` with RUSTFLAGS

Simon Sapin (Feb 20 2020 at 13:19, on Zulip):

-Zmiri-track-pointer-tag=<tag> aborts interpretation with a backtrace when the given pointer tag is popped from a borrow stack (which is where the tag becomes invalid and any future use of it will error anyway).”

Simon Sapin (Feb 20 2020 at 13:20, on Zulip):

I’m not sure what that means, but is it possible to get the error above if that tag never was on the borrow stack?

Simon Sapin (Feb 20 2020 at 14:52, on Zulip):

@RalfJ what is the borrow stack stack? How can I build a mental model of how to write code that manually allocates and deals with raw pointers while making miri happy?

RalfJ (Feb 20 2020 at 21:42, on Zulip):

@Simon Sapin a bunch of material about Stacked Borrows is collected at https://github.com/rust-lang/unsafe-code-guidelines/blob/master/wip/stacked-borrows.md

RalfJ (Feb 20 2020 at 21:44, on Zulip):

that track-pointer-tag didnt do anything is odd... it should show a "warning" when the ptr tag gets removed (the "abort" part is not true any more)

RalfJ (Feb 20 2020 at 21:44, on Zulip):

but indeed it could be that the tag never was on the respective borrow stack

RalfJ (Feb 20 2020 at 21:45, on Zulip):

since this is around slices, my first guess would be: did you do &slice[0] as *mut _ instead of slice.as_mut_ptr()? the former doesnt work

RalfJ (Feb 20 2020 at 21:45, on Zulip):

IOW, it sounds like you are taking a pointer that is okay for some part of the relevant memory, and trying to use it for a larger part of memory. That's why the tag is missing.

Simon Sapin (Feb 20 2020 at 21:47, on Zulip):

Yes I’m using as_mut_ptr but on [MaybeUninit<u8>; 0] which is the last field of a #[repr(C)] struct, in a heap allocation larger than the struct

Simon Sapin (Feb 20 2020 at 21:47, on Zulip):

https://github.com/servo/html5ever/blob/304c155bce19f956f0641cce3a1e19bc3b7bdaa1/zbuf/heap_data.rs#L99-L113

RalfJ (Feb 20 2020 at 21:47, on Zulip):

ah... yeah Stacked Borrows wouldnt be happy about that

RalfJ (Feb 20 2020 at 21:48, on Zulip):

as long as its all raw ptrs, it doesnt care

Simon Sapin (Feb 20 2020 at 21:49, on Zulip):

Doesn’t care about what?

RalfJ (Feb 20 2020 at 21:50, on Zulip):

anything, really... if you get a raw ptr from the allocator and only ever use raw ptr ops, I dont think it is even possible to get a Stacked Borrows error

RalfJ (Feb 20 2020 at 21:50, on Zulip):

but the moment you create a reference, it and every ptr derived from it (including raw ptrs derived from it) is confined to the bounds given by the type of the reference

Simon Sapin (Feb 20 2020 at 21:50, on Zulip):

Hmm I see. I guess what I’d need is field projection for raw pointers

RalfJ (Feb 20 2020 at 21:51, on Zulip):

we have that, but only on nightly

Simon Sapin (Feb 20 2020 at 21:51, on Zulip):

Right

RalfJ (Feb 20 2020 at 21:51, on Zulip):

on stable, working with only raw ptrs is not possible :/

Simon Sapin (Feb 20 2020 at 21:51, on Zulip):

Well I’d have to manually compute byte offsets, which doesn’t feel great

RalfJ (Feb 20 2020 at 21:51, on Zulip):

well or you wait until raw ptrs becomes better

RalfJ (Feb 20 2020 at 21:52, on Zulip):

you can still use Miri with -Zmiri-disable-validation (though that disables more than just stacked borrows... we should have a flag just for that)

Simon Sapin (Feb 20 2020 at 21:52, on Zulip):

And size_of::<Header>() is more conservative than needed since it rounds up to a multiple of alignment

RalfJ (Feb 20 2020 at 21:52, on Zulip):

not sure what you mean?

RalfJ (Feb 20 2020 at 21:53, on Zulip):

the issue tracking this problem is https://github.com/rust-lang/unsafe-code-guidelines/issues/134. if you want to add some information there or briefly describe your usecase, that would be great.

Simon Sapin (Feb 20 2020 at 21:55, on Zulip):

My header is { Cell<usize>, u32 }, which on 64-bit has size_of == 16 but only 12 useful bytes, so the data could start at the 13th byte

RalfJ (Feb 20 2020 at 21:56, on Zulip):

ah

RalfJ (Feb 20 2020 at 21:56, on Zulip):

but that would make header references overlap with the data...

Simon Sapin (Feb 20 2020 at 21:57, on Zulip):

Hence ideally having having a DST { Cell<usize>, u32, [u8] }, but with a thin pointer

RalfJ (Feb 20 2020 at 21:58, on Zulip):

is one of the other fields the length of the slice part?

Simon Sapin (Feb 20 2020 at 21:58, on Zulip):

{ Cell<usize>, u32, [u8; 0] } was an attempt to approximate that

Simon Sapin (Feb 20 2020 at 21:58, on Zulip):

The u32 is the allocated capacity

RalfJ (Feb 20 2020 at 21:58, on Zulip):

hm. the interaction of custom DST and Stacked Borrows is... interesting at least ;)

RalfJ (Feb 20 2020 at 21:59, on Zulip):

TBH probably the best bet is to just ignore everything except for the fixed-size prefix

RalfJ (Feb 20 2020 at 21:59, on Zulip):

but that would likely lead to problems similar as yours

RalfJ (Feb 20 2020 at 22:00, on Zulip):

anyway it's getting late, feel free to add some more questions here but I am going to go to sleep for now :D

Simon Sapin (Feb 20 2020 at 22:00, on Zulip):

TBH I’m not counting on custom DSTs in the language having even a serious design proposal in the next few years, if ever

Thom Chiovoloni (Feb 27 2020 at 06:43, on Zulip):

Yeah, I had issues trying to use a [u8; 0] as the start of an allocation offset for a custom-dst-like thing https://github.com/thomcc/thin_str/blob/master/src/lib.rs#L49. Miri got mat about using the [u8; 0] so now I just use it for some offset sanity checks. Kinda annoying but :shrug:

RalfJ (Feb 27 2020 at 09:14, on Zulip):

@Thom Chiovoloni could you write a summary somewhere about this? Ideally, a self-contained snippet demonstrating the pattern that you think should work, but which Miri complains about. Once we got a nice demo, it would be good to add that to https://github.com/rust-lang/unsafe-code-guidelines/issues/134.

RalfJ (Feb 27 2020 at 09:15, on Zulip):

@Simon Sapin -Zmiri-disable-stacked-borrows just landed in Miri; with the next submodule update then one (unsatisfying) option is to at least use Miri to test all the other things besides the aliasing rules.

Last update: Jun 05 2020 at 23:00UTC