Stream: t-lang/wg-unsafe-code-guidelines

Topic: Avoid UB on Vec on shared memory without synchronization


gnzlbg (Nov 08 2019 at 10:34, on Zulip):

@Alan Jeffrey what you are asking for cannot work

gnzlbg (Nov 08 2019 at 10:34, on Zulip):

What you are trying to do is


gnzlbg (Nov 08 2019 at 10:36, on Zulip):

and then have two threads of execution, writing and reading to the vector without any synchronization

gnzlbg (Nov 08 2019 at 10:36, on Zulip):

You are using read_volatile and write_volatile for that, and that's UB.

gnzlbg (Nov 08 2019 at 10:36, on Zulip):

You are just hoping that the compiler is dumb enough not to properly optimize your programs.

gnzlbg (Nov 08 2019 at 10:37, on Zulip):

You'd need something like ucg#212 to avoid that kind of UB by using atomic volatile reads / writes on the memory.

gnzlbg (Nov 08 2019 at 10:37, on Zulip):

But that only removes the UB of the data-races to the individual accesses.

gnzlbg (Nov 08 2019 at 10:38, on Zulip):

If one process calls Vec::resize or Vec::set_len, the (ptr, cap, len) state of the vector will be altered without any synchronization, invalidating the (ptr, cap, len) of the other process.

gnzlbg (Nov 08 2019 at 10:38, on Zulip):

That's a data-race, and the only way to avoid it is via synchronization, e.g., by using a "concurrent vec" instead, or by putting the Vec behind a mutex or similar

gnzlbg (Nov 08 2019 at 10:39, on Zulip):

Different OSes have different primitives for IPC, permissions, etc. - so you could build a safe abstraction by using those (e.g. see what Boost.Interprocess does - its documentation explains this particular case)

gnzlbg (Nov 08 2019 at 10:40, on Zulip):

Without synchronization, you are seeing that a process can call Vec::resize to truncate the vector, invalidating the len field that a different process has, and then that process does a use-after-free because of the invalid len, and you get a SIGBUS.

gnzlbg (Nov 08 2019 at 10:41, on Zulip):

None of that matters, because the moment the len field was invalidated without synchronization, both processes already have UB, that is, Rust does not provide any guarantees about what any of those processes do, independently of the languages they are written in.

gnzlbg (Nov 08 2019 at 10:41, on Zulip):

So you can get a SIGBUS, or the rust process can crash, or format your hard drive, or ... anything is allowed at that point.

gnzlbg (Nov 08 2019 at 10:42, on Zulip):

It also doesn't matter whether the two threads of execution are in the same or different processes, if its UB within the same process, _it is UB if you spawn the threads in different processes_, and it doesn't matter if you implement the other thread in a different programming language.

gnzlbg (Nov 08 2019 at 10:43, on Zulip):

I also can't think of any language tools that anybody can give you to make these particular data-races not be UB.

gnzlbg (Nov 08 2019 at 10:44, on Zulip):

Either never write to the (ptr, len, cap) field, so that all processes just read them, or you need to use synchronization.

gnzlbg (Nov 08 2019 at 10:44, on Zulip):

Depending on your OS, there might be ways you can require other adversarial processes to also use those synchronization primitives.

gnzlbg (Nov 08 2019 at 10:45, on Zulip):

But if you cannot do that on a particular platform, then you can't really write a safe abstraction around that, because it isn't safe to do so.

gnzlbg (Nov 08 2019 at 10:45, on Zulip):

Ask your OS, but from the OS POV, a valid answer might be "don't share shared memory with processes you don't trust", which is kind of fair as well. Best case you get an answer like "when creating the shared memory, pass flag X to forbid truncating it".

gnzlbg (Nov 08 2019 at 10:48, on Zulip):

https://www.boost.org/doc/libs/1_71_0/doc/html/interprocess/synchronization_mechanisms.html

gnzlbg (Nov 08 2019 at 10:50, on Zulip):

What you probably want is something like this (IPC mutexes require all processes to "play nice"): https://www.boost.org/doc/libs/1_71_0/doc/html/interprocess/synchronization_mechanisms.html#interprocess.synchronization_mechanisms.file_lock

gnzlbg (Nov 08 2019 at 10:51, on Zulip):

and use mandatory locking if you want a safe abstraction.... but mandatory locking is kind of broken on most OSes.

Alan Jeffrey (Nov 08 2019 at 21:09, on Zulip):

@gnzlbg Yes, you can't just put a Vec in shared memory and hope that it'll be OK,

Alan Jeffrey (Nov 08 2019 at 21:10, on Zulip):

hence https://github.com/asajeffrey/shared-data/blob/master/src/shared_vec.rs

Alan Jeffrey (Nov 08 2019 at 21:10, on Zulip):

which uses an AtomicUsize for the length.

Alan Jeffrey (Nov 08 2019 at 21:11, on Zulip):

This doesn't quite do everything, since it's using atomic loads and stores rather than atomic volatiles.

Alan Jeffrey (Nov 08 2019 at 21:12, on Zulip):

and it's not supporting growing yet.

Alan Jeffrey (Nov 08 2019 at 21:14, on Zulip):

and truncation of the underlying shared memory file is still an issue, different OSs seem to have different ways round that, sigh.

gnzlbg (Nov 10 2019 at 14:40, on Zulip):

@Alan Jeffrey I don't understand which problem does that AtomicUsize solve

gnzlbg (Nov 10 2019 at 14:41, on Zulip):

The shared memory that you are using (a file descriptor, with a length and a memory block), has a "length" that can be modified concurrently (depending on the OS, and certain options)

gnzlbg (Nov 10 2019 at 14:41, on Zulip):

That's the length that must be synchronized

gnzlbg (Nov 10 2019 at 14:42, on Zulip):

That data-structure is like a vector, but the SharedVec that you are creating there is like a &[T].

gnzlbg (Nov 10 2019 at 14:42, on Zulip):

Making the length field of a &[T] atomic does not solve the problem of multiple threads of execution modifying the Vec that actually owns that memory

Alan Jeffrey (Nov 10 2019 at 23:38, on Zulip):

@gnzlbg Assuming no truncation, the length is never set to more than the SharedVec allocation, and the requirement that T implements SharedMemCast/Ref means that every bitstring is a valid inhabitant of T.

Alan Jeffrey (Nov 10 2019 at 23:39, on Zulip):

So since the Len is atomic, there's no data races, and AFAICT there's no other sources of UB.

gnzlbg (Nov 13 2019 at 07:54, on Zulip):

If the "assuming no truncation" assumption is violated, then you have UB. If the file grows enough, such that no consecutive virtual memory pages can be allocated in your process for the file, you might have a problem as well.

gnzlbg (Nov 13 2019 at 07:55, on Zulip):

So since the Len is atomic, there's no data races,

The len of Vec or &[T] is not atomic, so I don't understand which problem having an atomic length solves for SharedVec. If you assume that the file will not be resized, its length can just be an usize.

gnzlbg (Nov 13 2019 at 07:57, on Zulip):

The only reason I can think of to make the length of SharedVec atomic would be if you want to allow concurrent modifications of it from a single process, but since SharedVec is like a &[T], that is like making a &[T] atomic to try to allow modifying it concurrently. Can be done, but isn't necessary, and often it isn't desirable, since each thread can have a different copy of the &[T] just fine, and its ok for these to have different lengths.

Last update: Nov 19 2019 at 18:00UTC