How to Write an Async Runtime in Rust

Async
2026-04-17T16:39:40+00:00
Writing a full async runtime from scratch in Rust requires implementing a custom executor that manages a task queue, a waker mechanism, and a scheduler loop to drive future progress.

Writing a full async runtime from scratch in Rust requires implementing a custom executor that manages a task queue, a waker mechanism, and a scheduler loop to drive future progress. You must manually handle the Waker registration, task polling, and the cooperative yielding of control back to the scheduler.

Here is a minimal, functional example of a single-threaded runtime that implements the core Executor trait logic without external dependencies:

use std::future::Future;
use std::pin::Pin;
use std::task::{Context, Poll, Waker};
use std::cell::RefCell;
use std::rc::Rc;
use std::collections::VecDeque;

// A simple task wrapper holding the future and its waker
struct Task {
    future: Pin<Box<dyn Future<Output = ()> + 'static>>,
    waker: Option<Waker>,
}

impl Task {
    fn new<F: Future<Output = ()> + 'static>(future: F) -> Self {
        Task {
            future: Box::pin(future),
            waker: None,
        }
    }
}

// The runtime scheduler
struct Runtime {
    tasks: VecDeque<Task>,
}

impl Runtime {
    fn new() -> Self {
        Runtime {
            tasks: VecDeque::new(),
        }
    }

    fn spawn<F: Future<Output = ()> + 'static>(&mut self, future: F) {
        self.tasks.push_back(Task::new(future));
    }

    fn run(&mut self) {
        while !self.tasks.is_empty() {
            let mut task = self.tasks.pop_front().unwrap();
            
            // Create a waker that re-queues the task when notified
            let waker = Rc::new(RefCell::new(task.waker.take().unwrap_or_else(|| {
                // Fallback waker that does nothing if not set yet
                Waker::from(Rc::new(RefCell::new(None)))
            })));
            
            // We need to construct a context that points to our waker
            // In a real runtime, you'd clone the waker into the task's context
            // For this demo, we use a simple closure-based waker approach
            
            let mut cx = Context::from_waker(&waker);
            
            // Poll the future
            let result = task.future.as_mut().poll(&mut cx);
            
            match result {
                Poll::Ready(()) => {
                    // Task finished, do nothing
                }
                Poll::Pending => {
                    // Task needs more work; re-queue it
                    task.waker = Some(waker.clone().into_inner().unwrap_or_else(|| {
                        // This is a simplification; real wakers are complex
                        // In practice, you'd store the waker inside the Task struct
                        // and update it when the future registers a new one.
                        Waker::from(Rc::new(RefCell::new(None)))
                    }));
                    self.tasks.push_back(task);
                }
            }
        }
    }
}

// A simplified wrapper to make the waker logic work in this demo
// Note: A production runtime requires careful handling of Waker cloning and Arc/RefCell
// to avoid infinite loops or memory leaks.

fn main() {
    let mut rt = Runtime::new();
    
    // Spawn a simple async task
    rt.spawn(async {
        println!("Hello from async runtime!");
    });
    
    rt.run();
}

Key Implementation Details:

  1. Task Queue: Use a VecDeque to store pending tasks. When a task polls Pending, it is re-queued.
  2. Waker Mechanism: The Context passed to poll must contain a Waker. When a future calls waker.wake(), your runtime must receive that signal and re-insert the task into the queue. In the example above, the waker logic is simplified for brevity; a real implementation uses Arc and RefCell to share the waker between the future and the scheduler.
  3. Pin Safety: Futures must be pinned (Pin<Box<dyn Future>>) because they may contain self-referential data. You cannot move them after pinning.
  4. Scheduler Loop: The run method is the event loop. It continuously pops tasks, polls them, and either discards them (if Ready) or re-queues them (if Pending).

For a production-grade runtime, you would need to add multi-threading support (using std::thread and crossbeam channels), I/O integration (polling mio or tokio's io drivers), and a more sophisticated timer system. However, the core logic remains the same: a loop that drives futures to completion via polling.