IPC System Design

Authoritative specification: docs/design/IPC-DESIGN.md

Implementation Status: Complete as of v0.25.1. Fast path IPC measured at <1us latency (79ns syscall_getpid, 44ns ipc_stats_read).

The VeridianOS Inter-Process Communication (IPC) system provides high-performance message passing with integrated capability support. The design emphasizes zero-copy transfers and minimal kernel involvement.

Architecture Overview

Three-Layer Design

┌─────────────────────────────────────────┐
│         POSIX API Layer                 │  fd = socket(); send(fd, buf, len)
├─────────────────────────────────────────┤
│       Translation Layer                 │  POSIX → Native IPC mapping
├─────────────────────────────────────────┤
│        Native IPC Layer                 │  port_send(); channel_receive()
└─────────────────────────────────────────┘

This layered approach provides:

• POSIX compatibility for easy porting
• Zero-overhead native API for performance
• Clean separation of concerns

IPC Primitives

1. Synchronous Message Passing

For small, latency-critical messages:

#![allow(unused)]
fn main() {
pub struct SyncMessage {
    // Message header (16 bytes)
    sender: ProcessId,
    msg_type: MessageType,
    flags: MessageFlags,
    
    // Inline data (up to 64 bytes)
    data: [u8; 64],
    
    // Capability transfer (up to 4)
    capabilities: [Option<Capability>; 4],
}

// Fast path: Register-based transfer
pub fn port_send(port: PortCap, msg: &SyncMessage) -> Result<(), IpcError> {
    // Message fits in registers for fast transfer
    syscall!(SYS_PORT_SEND, port, msg)
}

pub fn port_receive(port: PortCap) -> Result<SyncMessage, IpcError> {
    // Block until message available
    syscall!(SYS_PORT_RECEIVE, port)
}
}

Performance characteristics:

• Latency: <1μs for 64-byte messages
• No allocation: Stack-based transfer
• Direct handoff: Sender to receiver without queuing

2. Asynchronous Channels

For streaming and bulk data:

#![allow(unused)]
fn main() {
pub struct Channel {
    // Ring buffer for messages
    buffer: SharedMemory,
    
    // Producer/consumer indices
    write_idx: AtomicUsize,
    read_idx: AtomicUsize,
    
    // Notification mechanism
    event: EventFd,
}

impl Channel {
    pub async fn send(&self, data: &[u8]) -> Result<(), IpcError> {
        // Wait for space in ring buffer
        while self.is_full() {
            self.event.wait().await?;
        }
        
        // Copy to shared buffer
        let idx = self.write_idx.fetch_add(1, Ordering::Release);
        self.buffer.write_at(idx, data)?;
        
        // Notify receiver
        self.event.signal()?;
        Ok(())
    }
}
}

Features:

• Buffered: Multiple messages in flight
• Non-blocking: Async/await compatible
• Batching: Amortize syscall overhead

3. Zero-Copy Shared Memory

For large data transfers:

#![allow(unused)]
fn main() {
pub struct SharedBuffer {
    // Memory capability
    memory_cap: Capability,
    
    // Virtual address in sender space
    sender_addr: VirtAddr,
    
    // Size of shared region
    size: usize,
}

// Create shared memory region
let buffer = SharedBuffer::create(1024 * 1024)?; // 1MB

// Map into receiver's address space
receiver.map_shared(buffer.memory_cap)?;

// Transfer ownership without copying
sender.transfer_buffer(buffer, receiver)?;
}

Advantages:

• True zero-copy: Data never copied
• Large transfers: Gigabytes without overhead
• DMA compatible: Direct hardware access

Port System

Port Creation and Binding

#![allow(unused)]
fn main() {
pub struct Port {
    // Unique port identifier
    id: PortId,
    
    // Message queue
    messages: VecDeque<SyncMessage>,
    
    // Waiting threads
    waiters: WaitQueue,
    
    // Access control
    capability: Capability,
}

// Create a new port
let port = Port::create()?;

// Bind to well-known name
namespace.bind("com.app.service", port.capability)?;

// Connect from client
let service = namespace.lookup("com.app.service")?;
}

Port Rights

Capabilities control port access:

#![allow(unused)]
fn main() {
bitflags! {
    pub struct PortRights: u16 {
        const SEND = 0x01;      // Can send messages
        const RECEIVE = 0x02;   // Can receive messages
        const MANAGE = 0x04;    // Can modify port
        const GRANT = 0x08;     // Can share capability
    }
}

// Create receive-only capability
let recv_cap = port_cap.derive(PortRights::RECEIVE)?;
}

Performance Optimizations

1. Fast Path for Small Messages

#![allow(unused)]
fn main() {
// Kernel fast path
pub fn handle_port_send_fast(
    port: PortId,
    msg: &SyncMessage,
) -> Result<(), IpcError> {
    // Skip queue if receiver waiting
    if let Some(receiver) = port.waiters.pop() {
        // Direct register transfer
        receiver.transfer_registers(msg);
        receiver.wake();
        return Ok(());
    }
    
    // Fall back to queuing
    port.enqueue(msg)
}
}

2. Batched Operations

#![allow(unused)]
fn main() {
pub struct BatchedChannel {
    messages: Vec<Message>,
    batch_size: usize,
}

impl BatchedChannel {
    pub fn send(&mut self, msg: Message) -> Result<(), IpcError> {
        self.messages.push(msg);
        
        // Flush when batch full
        if self.messages.len() >= self.batch_size {
            self.flush()?;
        }
        Ok(())
    }
    
    pub fn flush(&mut self) -> Result<(), IpcError> {
        // Single syscall for entire batch
        syscall!(SYS_CHANNEL_SEND_BATCH, &self.messages)?;
        self.messages.clear();
        Ok(())
    }
}
}

3. CPU Cache Optimization

#![allow(unused)]
fn main() {
// Align message structures to cache lines
#[repr(C, align(64))]
pub struct CacheAlignedMessage {
    header: MessageHeader,
    data: [u8; 48], // Fit in single cache line
}

// NUMA-aware channel placement
pub fn create_channel_on_node(node: NumaNode) -> Channel {
    let buffer = allocate_on_node(CHANNEL_SIZE, node);
    Channel::new(buffer)
}
}

Security Features

Capability Integration

All IPC operations require capabilities:

#![allow(unused)]
fn main() {
// Type-safe capability requirements
pub fn connect<T: Service>(
    endpoint: &str,
) -> Result<TypedPort<T>, IpcError> {
    let cap = namespace.lookup(endpoint)?;
    
    // Verify capability type matches service
    if cap.service_type() != T::SERVICE_ID {
        return Err(IpcError::TypeMismatch);
    }
    
    Ok(TypedPort::new(cap))
}
}

Message Filtering

#![allow(unused)]
fn main() {
pub struct MessageFilter {
    allowed_types: BitSet,
    max_size: usize,
    rate_limit: RateLimit,
}

impl Port {
    pub fn set_filter(&mut self, filter: MessageFilter) {
        self.filter = Some(filter);
    }
    
    fn accept_message(&self, msg: &Message) -> bool {
        if let Some(filter) = &self.filter {
            filter.allowed_types.contains(msg.msg_type)
                && msg.size() <= filter.max_size
                && filter.rate_limit.check()
        } else {
            true
        }
    }
}
}

Error Handling

IPC Errors

#![allow(unused)]
fn main() {
#[derive(Debug)]
pub enum IpcError {
    // Port errors
    PortNotFound,
    PortClosed,
    PortFull,
    
    // Permission errors
    InsufficientRights,
    InvalidCapability,
    
    // Message errors
    MessageTooLarge,
    InvalidMessage,
    
    // System errors
    OutOfMemory,
    WouldBlock,
}
}

Timeout Support

#![allow(unused)]
fn main() {
pub fn port_receive_timeout(
    port: PortCap,
    timeout: Duration,
) -> Result<SyncMessage, IpcError> {
    let deadline = Instant::now() + timeout;
    
    loop {
        match port_try_receive(port)? {
            Some(msg) => return Ok(msg),
            None if Instant::now() >= deadline => {
                return Err(IpcError::Timeout);
            }
            None => thread::yield_now(),
        }
    }
}
}

POSIX Compatibility Layer

Socket Emulation

#![allow(unused)]
fn main() {
// POSIX socket() -> create port
pub fn socket(domain: i32, type_: i32, protocol: i32) -> Result<Fd, Errno> {
    let port = Port::create()?;
    let fd = process.fd_table.insert(FdType::Port(port));
    Ok(fd)
}

// POSIX send() -> port send
pub fn send(fd: Fd, buf: &[u8], flags: i32) -> Result<usize, Errno> {
    let port = process.fd_table.get_port(fd)?;
    
    // Convert to native IPC
    let msg = SyncMessage {
        data: buf.try_into()?,
        ..Default::default()
    };
    
    port_send(port, &msg)?;
    Ok(buf.len())
}
}

Performance Metrics

Latency Targets

Throughput Targets

Best Practices

1. Use sync for small messages: Lower latency than async
2. Batch when possible: Amortize syscall overhead
3. Prefer zero-copy: For messages >4KB
4. Cache port capabilities: Avoid repeated lookups
5. Set appropriate filters: Prevent DoS attacks