@Alan Jeffrey what you are asking for cannot work
What you are trying to do is
and then have two threads of execution, writing and reading to the vector without any synchronization
You are using
write_volatile for that, and that's UB.
You are just hoping that the compiler is dumb enough not to properly optimize your programs.
You'd need something like ucg#212 to avoid that kind of UB by using
atomic volatile reads / writes on the memory.
But that only removes the UB of the data-races to the individual accesses.
If one process calls
(ptr, cap, len) state of the vector will be altered without any synchronization, invalidating the
(ptr, cap, len) of the other process.
That's a data-race, and the only way to avoid it is via synchronization, e.g., by using a "concurrent vec" instead, or by putting the
Vec behind a mutex or similar
Different OSes have different primitives for IPC, permissions, etc. - so you could build a safe abstraction by using those (e.g. see what Boost.Interprocess does - its documentation explains this particular case)
Without synchronization, you are seeing that a process can call
Vec::resize to truncate the vector, invalidating the
len field that a different process has, and then that process does a use-after-free because of the invalid
len, and you get a
None of that matters, because the moment the
len field was invalidated without synchronization, both processes already have UB, that is, Rust does not provide any guarantees about what any of those processes do, independently of the languages they are written in.
So you can get a
SIGBUS, or the rust process can crash, or format your hard drive, or ... anything is allowed at that point.
It also doesn't matter whether the two threads of execution are in the same or different processes, if its UB within the same process, _it is UB if you spawn the threads in different processes_, and it doesn't matter if you implement the other thread in a different programming language.
I also can't think of any language tools that anybody can give you to make these particular data-races not be UB.
Either never write to the
(ptr, len, cap) field, so that all processes just read them, or you need to use synchronization.
Depending on your OS, there might be ways you can require other adversarial processes to also use those synchronization primitives.
But if you cannot do that on a particular platform, then you can't really write a safe abstraction around that, because it isn't safe to do so.
Ask your OS, but from the OS POV, a valid answer might be "don't share shared memory with processes you don't trust", which is kind of fair as well. Best case you get an answer like "when creating the shared memory, pass flag X to forbid truncating it".
What you probably want is something like this (IPC mutexes require all processes to "play nice"): https://www.boost.org/doc/libs/1_71_0/doc/html/interprocess/synchronization_mechanisms.html#interprocess.synchronization_mechanisms.file_lock
and use mandatory locking if you want a safe abstraction.... but mandatory locking is kind of broken on most OSes.
@gnzlbg Yes, you can't just put a
Vec in shared memory and hope that it'll be OK,
which uses an
AtomicUsize for the length.
This doesn't quite do everything, since it's using atomic loads and stores rather than atomic volatiles.
and it's not supporting growing yet.
and truncation of the underlying shared memory file is still an issue, different OSs seem to have different ways round that, sigh.
@Alan Jeffrey I don't understand which problem does that
The shared memory that you are using (a file descriptor, with a length and a memory block), has a "length" that can be modified concurrently (depending on the OS, and certain options)
That's the length that must be synchronized
That data-structure is like a vector, but the
SharedVec that you are creating there is like a
Making the length field of a
&[T] atomic does not solve the problem of multiple threads of execution modifying the
Vec that actually owns that memory
@gnzlbg Assuming no truncation, the length is never set to more than the
SharedVec allocation, and the requirement that
Ref means that every bitstring is a valid inhabitant of
So since the Len is atomic, there's no data races, and AFAICT there's no other sources of UB.
If the "assuming no truncation" assumption is violated, then you have UB. If the file grows enough, such that no consecutive virtual memory pages can be allocated in your process for the file, you might have a problem as well.
So since the Len is atomic, there's no data races,
The len of
&[T] is not atomic, so I don't understand which problem having an atomic length solves for
SharedVec. If you assume that the file will not be resized, its length can just be an
The only reason I can think of to make the length of
SharedVec atomic would be if you want to allow concurrent modifications of it from a single process, but since
SharedVec is like a
&[T], that is like making a
&[T] atomic to try to allow modifying it concurrently. Can be done, but isn't necessary, and often it isn't desirable, since each thread can have a different copy of the
&[T] just fine, and its ok for these to have different lengths.