The upload that crashes your server
You build a file upload endpoint. It works perfectly for profile pictures. A user drags in a 500MB video file. Your server's memory spikes to 100%, the process gets killed by the OS, and your users get a 502 Bad Gateway error.
The problem isn't the file size. The problem is how you handled the data. You likely tried to read the entire file into a Vec<u8> before saving it. That approach works for text forms. It fails catastrophically for binary uploads.
Rust gives you the tools to handle streams safely. You just need to treat the upload as a flow of water, not a bucket you have to carry.
Multipart forms are streams, not blobs
When a browser sends a file, it doesn't send one giant packet. HTTP uses the multipart/form-data encoding. The browser splits the request into parts separated by a boundary string. Each part contains headers and then the raw bytes of the file.
The network delivers these bytes in chunks. Your Rust server receives them asynchronously. If you wait for all chunks to arrive before doing anything, you block the thread and fill RAM. The correct pattern is to process each chunk as it arrives and write it directly to disk.
Think of it like a conveyor belt in a factory. You don't stop the belt to stack every box in a pile before moving them to the warehouse. You grab each box as it passes and place it on the truck. Streaming works the same way. You read a chunk, write a chunk, discard the chunk. Memory usage stays flat regardless of file size.
Minimal example: streaming to disk
Axum is the most common framework for Rust web apps. It includes a Multipart extractor that handles the parsing for you. The extractor gives you a stream of fields. Each field is a stream of chunks.
This example sets up a server that accepts uploads and writes them to a file. It uses tokio::fs for non-blocking file operations.
use axum::{extract::Multipart, response::IntoResponse, routing::post, Router};
use tokio::fs::File;
use tokio::io::AsyncWriteExt;
#[tokio::main]
async fn main() {
// Set up the router with a single POST route
let app = Router::new().route("/upload", post(upload_handler));
// Bind to port 3000
let listener = tokio::net::TcpListener::bind("0.0.0.0:3000").await.unwrap();
axum::serve(listener, app).await.unwrap();
}
/// Handles the file upload by streaming chunks directly to disk
async fn upload_handler(mut multipart: Multipart) -> impl IntoResponse {
// Open the output file for writing
let mut file = File::create("uploaded.bin").await.unwrap();
// Iterate through each field in the multipart form
while let Some(field) = multipart.next_field().await.unwrap() {
// Iterate through each chunk of bytes in the field
while let Some(chunk) = field.chunk().await.unwrap() {
// Write the chunk to disk immediately
file.write_all(&chunk).await.unwrap();
}
}
"Upload complete"
}
The Multipart extractor consumes the request body. You call next_field() to get the next part of the form. Inside that loop, chunk() yields Bytes objects. These are zero-copy slices of the incoming data. You write them to the file and drop them. The memory is reclaimed instantly.
Never buffer the entire file. Stream it.
How the data flows
When the handler runs, the async runtime coordinates three things: the network socket, the multipart parser, and the file system.
- The browser sends HTTP headers followed by the first chunk of data.
- Axum's
Multipartextractor parses the headers and yields the first field. - Your code calls
chunk(). The runtime reads bytes from the socket until it has a full chunk or hits a timeout. tokio::fs::File::write_allwrites the bytes to the operating system's page cache. The call returns immediately without blocking the thread.- The loop continues until
chunk()returnsNone. - The field ends. The loop checks for the next field.
- When all fields are processed, the handler returns.
If you use std::fs::File instead of tokio::fs::File, step 4 blocks the entire thread. The async runtime cannot run other tasks while the disk write finishes. Under load, this stalls your server. Always use tokio::fs in async handlers.
Convention aside: The Rust community treats std::fs as a blocking operation. If you see std::fs in an async fn, it's a code smell. Use tokio::fs or spawn a blocking task with tokio::task::spawn_blocking.
Realistic example: names, errors, and security
The minimal example overwrites the same file every time. It ignores the filename sent by the browser. It panics on any error. Real code needs to handle these cases.
Browsers send a filename in the field headers. You can access it with field.file_name(). This returns an Option<&str>. You should also handle errors properly using Result types instead of unwrap().
use axum::{
extract::Multipart,
http::StatusCode,
response::IntoResponse,
routing::post,
Router,
};
use tokio::fs::File;
use tokio::io::AsyncWriteExt;
use uuid::Uuid;
#[tokio::main]
async fn main() {
let app = Router::new().route("/upload", post(upload_handler));
let listener = tokio::net::TcpListener::bind("0.0.0.0:3000").await.unwrap();
axum::serve(listener, app).await.unwrap();
}
/// Handles upload with unique filenames and proper error handling
async fn upload_handler(mut multipart: Multipart) -> impl IntoResponse {
// Generate a unique ID to prevent collisions and path traversal
let unique_id = Uuid::new_v4().to_string();
let filename = format!("uploads/{}.dat", unique_id);
// Create the file, returning a 500 error if it fails
let mut file = match File::create(&filename).await {
Ok(f) => f,
Err(e) => return (StatusCode::INTERNAL_SERVER_ERROR, format!("Failed to create file: {}", e)),
};
// Process each field
while let Some(result) = multipart.next_field().await {
let field = match result {
Ok(f) => f,
Err(e) => return (StatusCode::BAD_REQUEST, format!("Multipart error: {}", e)),
};
// Stream chunks to disk
while let Some(result) = field.chunk().await {
let chunk = match result {
Ok(c) => c,
Err(e) => return (StatusCode::BAD_REQUEST, format!("Chunk error: {}", e)),
};
if let Err(e) = file.write_all(&chunk).await {
return (StatusCode::INTERNAL_SERVER_ERROR, format!("Write error: {}", e));
}
}
}
(StatusCode::OK, "File saved successfully")
}
This handler generates a UUID for the filename. This avoids collisions if two users upload at the same time. It also prevents path traversal attacks where a malicious client sends a filename like ../../etc/passwd. By ignoring the client's filename and using a generated ID, you control the file system structure.
The error handling uses early returns. If any step fails, the handler returns an HTTP error response immediately. This keeps the logic linear and readable.
Sanitize the filename before you trust it. Never trust the client.
Pitfalls and compiler errors
File uploads introduce specific failure modes. Knowing them saves debugging time.
Blocking the runtime
Using std::fs inside an async handler blocks the executor. The compiler won't stop you. It compiles fine. But your server performance degrades.
If you accidentally use std::fs::File::create, the code runs. Under high concurrency, the thread pool fills up. Requests queue. Latency spikes.
Use tokio::fs::File. If you must call a blocking library, wrap it in tokio::task::spawn_blocking.
Memory exhaustion
Reading the file into a Vec<u8> is the most common mistake.
// BAD: Loads entire file into RAM
let mut data = Vec::new();
while let Some(chunk) = field.chunk().await.unwrap() {
data.extend_from_slice(&chunk);
}
This works for small files. For large files, it allocates memory until the process crashes. The compiler might warn you about growing vectors, but it won't prevent the allocation.
Stream the data. Write chunks directly to the destination.
Trait bound errors
Axum extractors require specific trait bounds. If you try to extract Multipart from a request that doesn't have the Content-Type: multipart/form-data header, the extraction fails at runtime, not compile time.
If you mess up the handler signature, you'll see compiler errors. For example, if you forget to mark the handler as async, you get a type mismatch.
// Error: E0308 mismatched types
// Expected `impl IntoResponse`, found `fn`
fn upload(multipart: Multipart) -> String { ... }
The compiler expects an async function that returns a type implementing IntoResponse. Fix the signature to async fn upload(...) -> impl IntoResponse.
Borrow checker issues
If you try to move the Multipart stream into a closure or another function, you might hit E0382 (use of moved value). The stream is consumed as you iterate it. You cannot clone the stream.
// Error: E0382 use of moved value `multipart`
let field = multipart.next_field().await;
let field2 = multipart.next_field().await; // multipart is partially moved
The next_field method takes &mut self. You must keep the mutable borrow alive for the duration of the loop. The while let pattern handles this correctly by reborrowing.
Trust the borrow checker. It usually has a point.
Decision: when to use this vs alternatives
Choose the right tool based on your file size and processing needs.
Use Axum's Multipart extractor when you are building a standard web application and need to parse form data. It handles the boundary parsing and header extraction automatically.
Use tokio::fs::File when you are writing uploaded data to disk in an async context. It integrates with the runtime and avoids blocking threads.
Use tempfile::NamedTempFile when you need to process the file before saving it permanently. Write to a temp file, run validation or virus scanning, then rename it to the final location. This prevents partial files from cluttering your storage.
Use Bytes or Vec<u8> only for tiny files under 1MB, like configuration files or small avatars. Loading small files into memory is faster than disk IO. For anything larger, stream the data.
Use actix-web multipart support if you are already using Actix. The API is similar but uses MultipartForm derive macros for structured data.
Stream the data. Never hoard it.