When one process isn't enough
You're building a high-performance data pipeline. Process A generates gigabytes of sensor data. Process B needs to analyze it in real-time. Sending data over a socket means copying bytes from kernel space to user space, then back again. The CPU spends more time moving data than processing it. You want both processes to look at the exact same bytes in RAM. No copies. No serialization. Just one chunk of memory that two separate programs can touch.
That's shared memory. You want zero-copy communication. The OS can give you that, but Rust makes you earn it.
How memory mapping works
Shared memory works by mapping a file or an anonymous region into the virtual address space of multiple processes. The operating system makes the same physical RAM pages appear at different virtual addresses in each process. When one process writes, the hardware updates the RAM. The other process sees the change immediately because it's looking at the same RAM.
Think of a whiteboard in a shared hallway. Office A writes a calculation. Office B walks by and reads the result. No messenger needed. No photocopies. The board is the shared state.
Rust doesn't expose this directly in the standard library. Mapping memory bypasses the borrow checker. The compiler can't track who touches the memory across process boundaries. You have to tell the compiler, "I know what I'm doing," using unsafe. The compiler can't track cross-process aliases. You have the responsibility to bridge the gap.
Minimal example: mapping a file
Rust relies on crates for OS-specific syscalls. The community standard is memmap2. It wraps the platform calls and gives you a safe interface over the raw mapping.
Convention: memmap2 is the active crate. The original memmap is unmaintained. Always pick memmap2 for new projects.
use memmap2::MmapMut;
use std::fs::File;
use std::io::Write;
/// Creates a memory-mapped file and writes a byte.
fn main() {
// Create a file to back the mapping.
let file = File::create("shared.bin").unwrap();
file.write_all(b"Hello").unwrap();
// Map the file into memory.
// This requires unsafe because the OS mapping could fail
// or the file could be truncated by another process.
let mut mmap = unsafe { MmapMut::map_mut(&file).unwrap() };
// MmapMut derefs to &mut [u8].
// We can index into it like a slice.
mmap[0] = b'X';
// The file on disk now contains "Xello".
// Changes are flushed to disk when the MmapMut is dropped,
// or immediately if the OS decides to page out.
}
Add memmap2 = "0.9" to your Cargo.toml. The MmapMut type holds the mapping. When mmap goes out of scope, the mapping is unmapped and the OS cleans up. Keep the unsafe block tight. The moment you exit it, the compiler regains control.
Walking through the mapping
When you call MmapMut::map_mut, the crate asks the OS to map the file. The OS returns a pointer to the mapped region. Rust wraps this pointer in a MmapMut struct. The unsafe block is required because the compiler cannot guarantee the mapping is valid. The file could be truncated, permissions could change, or another process could corrupt the data.
The Deref implementation lets you treat MmapMut as &mut [u8]. You can use slice methods, indexing, and iteration. The compiler enforces borrowing rules on the slice. You can't create two mutable references to the same region within the same process. That protection holds as long as you stay within the MmapMut abstraction.
Once you cross into another process, the borrow checker has no visibility. Process B might map the same file and write to index 0 while Process A is reading it. Rust's type system doesn't stop races across processes. You need synchronization.
Realistic example: shared buffer with synchronization
Real shared memory isn't just writing bytes. You usually have a protocol. A header with metadata, followed by a data buffer. And you need synchronization. If Process A writes while Process B reads, you get torn reads. Rust's AtomicUsize works across processes because the OS guarantees atomicity for these operations on mapped memory.
Convention: Shared memory structs must be aligned. Use repr(align) or ensure the mapping offset is aligned. Misaligned accesses can crash on some architectures or degrade performance.
use memmap2::{MmapMut, MmapOptions};
use std::fs::File;
use std::sync::atomic::{AtomicUsize, Ordering};
use std::mem::size_of;
/// Shared buffer layout.
/// The sequence number acts as a lock-free version counter.
/// Processes check the sequence before reading the data.
#[repr(C)]
struct SharedBuffer {
sequence: AtomicUsize,
data: [u8; 1024],
}
/// Maps a file and interprets it as a SharedBuffer.
fn main() {
let file = File::create("buffer.bin").unwrap();
// Pre-allocate the file size.
// Mapping a file smaller than the struct causes undefined behavior.
file.set_len(size_of::<SharedBuffer>() as u64).unwrap();
// Map the specific region.
// MmapOptions lets you control offset and length.
let mmap = unsafe {
MmapOptions::new()
.offset(0)
.len(size_of::<SharedBuffer>())
.map_mut(&file)
.unwrap()
};
// Cast the slice to a struct pointer.
// SAFETY:
// 1. `mmap` is valid and not unmapped.
// 2. `mmap.as_mut_ptr()` is aligned to `SharedBuffer`.
// File mappings are page-aligned, which satisfies struct alignment.
// 3. `mmap` length is exactly `size_of::<SharedBuffer>()`.
// 4. No other references to this memory exist in this process.
let buffer = unsafe {
&mut *(mmap.as_mut_ptr() as *mut SharedBuffer)
};
// Update sequence and data.
// Relaxed ordering is safe here because the sequence
// acts as a version number, not a synchronization point
// for other memory accesses within this process.
buffer.sequence.store(1, Ordering::Relaxed);
buffer.data[0] = b'A';
// In a real app, Process B would read the sequence,
// check it's even/odd or incremented, then read data.
let seq = buffer.sequence.load(Ordering::Relaxed);
println!("Sequence: {}, Data byte: {}", seq, buffer.data[0]);
}
The SAFETY comment lists the invariants. If you can't prove these, you don't have a safe wrapper. The repr(C) ensures the layout is predictable across processes. The AtomicUsize provides a way to coordinate without heavy locks. Treat the memory layout as a contract. If the header size changes, the offset breaks, and the other process reads garbage.
Pitfalls and compiler errors
The compiler will reject raw pointer dereferences outside unsafe. That's E0133 (dereference of raw pointer requires unsafe). It's doing its job. You can't just cast a *mut u8 to a reference and hope for the best. The borrow checker assumes references are unique or immutable. Shared memory breaks that assumption.
If you create two mutable references to the same mapped region inside unsafe, you get undefined behavior. The compiler trusts you inside unsafe. If you lie, the program crashes or corrupts memory. The borrow checker is your friend. When you step into unsafe, you're the borrow checker. Act like it.
Watch out for file truncation. If Process A truncates the file while Process B has it mapped, Process B gets a SIGBUS signal and dies. The OS kills the process for accessing unmapped memory. Always lock the file size or use anonymous mappings if truncation is a risk.
Anonymous mappings exist for temporary shared memory without a backing file. memmap2 supports this via MmapMut::map_mut_anon. Use this when you don't need persistence. The memory disappears when all processes unmap it.
Convention: Use let _ = result when you intentionally discard a return value from a shared memory operation. It signals to readers that you considered the value and chose to drop it.
Decision: when to use shared memory
Use memmap2 when you need zero-copy communication between separate processes and the data fits in RAM.
Use Arc<Mutex<T>> when you're sharing state between threads within the same process; the overhead of system calls for IPC is unnecessary here.
Use crossbeam::channel when you need a producer-consumer pattern across threads and want to avoid manual synchronization; channels handle the locking for you.
Use tokio::sync::mpsc when you're in an async runtime and want to share messages between tasks without blocking the executor.
Reach for raw sockets or pipes when the data stream is unbounded and doesn't fit in a fixed-size buffer; memory mapping requires a known size upfront.
Shared memory is powerful but dangerous. It gives you speed at the cost of complexity. You manage synchronization, alignment, and lifetime manually. Use it when profiling proves that copying is the bottleneck. Otherwise, stick to channels and messages. The extra safety is worth the overhead.