When a JSON file isn't enough
You are building a CLI tool that tracks your daily coffee intake. At first, you dump the data into a coffee.json file. It works fine for a week. Then the file grows to fifty megabytes. Loading the file takes two seconds every time you run the tool. You want to update just one entry, but you have to read the whole file, modify it in memory, and write the whole file back. If the power cuts during the write, you lose everything.
You need a database. But spinning up PostgreSQL or MongoDB is overkill. You do not want a separate server process, a network port, or a configuration file for the database connection. You just want a file on disk that your code talks to directly.
That is what an embedded database does. It is a library, not a service. It lives inside your binary. You call functions, and it manages a file on disk for you. No localhost, no sudo systemctl start postgres, no connection strings. Just storage that scales beyond memory and survives crashes.
Think of a standard database like a restaurant kitchen. You send orders, the chefs cook, and you get food back. There is a clear boundary. You communicate over a network protocol. An embedded database is like a Swiss Army knife attached to your belt. You pull it out, use it, put it back. There is no boundary. Your code calls the database functions directly. The database runs in the same process as your application. If your app crashes, the database is gone too. If your app is fast, the database is fast.
The trade-off is simplicity versus isolation. You get zero-latency access and no deployment complexity. You lose the ability to share the database with other processes easily and the safety net of a separate server that survives app restarts. Choose embedded storage when your app owns the data end-to-end.
What an embedded database actually does
Rust has two main players in this space: sled and redb. sled was the community favorite for years. It stores data in a directory on disk, using a log-structured merge tree under the hood. LSM trees are great for write performance because they append data sequentially. Reads can be slower because the database has to merge multiple layers of data. sled is now in maintenance mode. It works, but it has not seen major updates in years.
redb is the modern successor. It uses a B+ tree structure, which balances read and write performance. It supports transactions, crash safety, and concurrent readers. It stores everything in a single file, making backups and deployment trivial. redb is actively developed and fixes the architectural limitations of sled. Start new projects with redb. Treat sled as a legacy tool.
The modern standard: redb
Here is the smallest working case: opening a database, writing a key-value pair, and reading it back.
use redb::{Database, TableDefinition, ReadableTable, WritableTable};
// Define the table schema once at compile time.
// This tells redb the exact key and value types.
const COFFEE_TABLE: TableDefinition<&[u8], &[u8]> = TableDefinition::new("coffee");
fn main() -> Result<(), Box<dyn std::error::Error>> {
// Creates or opens the single database file.
// redb handles the underlying B+ tree allocation automatically.
let db = Database::create("coffee.redb")?;
// All writes happen inside a transaction.
// This ensures atomicity: all changes save together or none do.
let write_txn = db.begin_write()?;
{
// Open the table within the transaction scope.
// The table handle borrows from the transaction.
let mut table = write_txn.open_table(COFFEE_TABLE)?;
// Insert key-value pairs as byte slices.
// redb copies the bytes into its internal storage.
table.insert(b"beans", b"arabica")?;
}
// Committing flushes the transaction to disk.
// Dropping the handle without this step rolls back everything.
write_txn.commit()?;
// Reading happens in a separate transaction.
// redb supports multiple concurrent readers without blocking.
let read_txn = db.begin_read()?;
let table = read_txn.open_table(COFFEE_TABLE)?;
if let Some(accessor) = table.get(b"beans")? {
println!("Beans: {:?}", accessor.value());
}
Ok(())
}
redb requires transactions for writes. You call begin_write to start a transaction, perform operations, and call commit to persist them. If you drop the transaction handle without committing, redb rolls back all changes. This protects you from partial writes. If your program crashes mid-transaction, the database remains consistent.
TableDefinition is a compile-time schema check. You define the table once with its key and value types. redb uses this to ensure you do not insert the wrong types. It is a community convention to define these as constants at the module level. The convention exists because table definitions are zero-cost compile-time tokens. Defining them inline inside functions triggers unnecessary warnings and hurts readability.
The single-file design is a major advantage. You can copy coffee.redb to back up your data. You can move it to another machine. No directory traversal, no hidden files. Commit your transactions. If you drop the write handle without committing, your data evaporates.
How transactions and lifetimes work
Real applications rarely store raw byte slices. You want to store structs. Both sled and redb store bytes. They do not know about Rust structs. You must serialize and deserialize data manually.
Here is how you handle a custom struct while respecting redb lifetime rules.
use redb::{Database, TableDefinition, ReadableTable, WritableTable};
use serde::{Deserialize, Serialize};
const COFFEE_TABLE: TableDefinition<&str, &[u8]> = TableDefinition::new("coffee");
#[derive(Serialize, Deserialize)]
struct Coffee {
beans: String,
roast: String,
}
fn main() -> Result<(), Box<dyn std::error::Error>> {
let db = Database::create("coffee.redb")?;
let write_txn = db.begin_write()?;
{
let mut table = write_txn.open_table(COFFEE_TABLE)?;
let coffee = Coffee { beans: "arabica".into(), roast: "medium".into() };
// Serialize to bytes before inserting.
// bincode is faster and smaller than JSON for internal storage.
let bytes = bincode::serialize(&coffee)?;
table.insert("favorite", &bytes)?;
}
write_txn.commit()?;
let read_txn = db.begin_read()?;
let table = read_txn.open_table(COFFEE_TABLE)?;
// Clone the bytes to own them outside the transaction scope.
// redb values borrow from the transaction and cannot outlive it.
let stored_bytes = table.get("favorite")?.map(|v| v.value().to_vec());
if let Some(bytes) = stored_bytes {
let coffee: Coffee = bincode::deserialize(&bytes)?;
println!("Loaded: {} {}", coffee.beans, coffee.roast);
}
Ok(())
}
redb enforces strict lifetimes. Values returned from table.get borrow from the transaction. You cannot hold a reference to a value after the transaction ends. If you try to return a value from a function that closes the transaction, the compiler rejects you with E0597 (does not live long enough). You must clone the value or process it inside the transaction scope. The community convention is to call .to_vec() or .to_string() immediately when you need data to outlive the read transaction. This explicit clone signals to future readers that you intentionally broke the borrow chain.
Pitfalls and compiler traps
Embedded databases hide complexity, but they still have sharp edges. Watch for these common issues.
Forgetting to commit in redb
redb transactions are explicit. If you write data and forget to call commit, the data is gone. The compiler will not warn you. The insert method returns Ok, which tricks you into thinking the data is saved. It is not. The data lives only in memory until commit flushes it to disk. Always call commit. If you want to be extra safe, wrap the transaction logic in a helper function that guarantees commit runs on success.
sled creates a directory, redb creates a file
sled creates a directory with multiple files. redb creates a single file. If you pass a path that already exists as a file to sled, it will fail. If you pass a path that exists as a directory to redb, it will fail. Check your paths. Use std::path::Path to inspect what you are opening. This mismatch causes confusion when migrating from sled to redb.
Serialization is your responsibility
Both sled and redb store bytes. They do not know about Rust structs. You must serialize and deserialize data manually. Use serde with bincode for efficient binary serialization. bincode is fast and compact. serde_json works but adds overhead and larger file sizes. Serialization is your responsibility. The database stores bytes; you decide what those bytes mean. Treat your serialization format as a public API. Changing the struct layout breaks old data. Version your formats if your tool will live longer than a weekend.
Choosing your storage
Pick the tool that matches your query complexity and maintenance needs.
Use redb for new embedded storage needs. It offers transactions, B+ tree performance, crash safety, and active development. It is the current standard for key-value storage in Rust.
Use sled only when maintaining legacy code that already depends on it. It is in maintenance mode and lacks recent security or performance updates. Do not start new projects with sled.
Use SQLite via rusqlite when you need SQL queries, joins, or a mature ecosystem of tooling. SQLite is an embedded database too, but it speaks SQL. If your data has relationships or you need complex filtering, SQLite is the better choice.
Use a simple JSON file when your dataset fits comfortably in memory and you only append or replace the whole file. Embedded databases add complexity. If you have less than a thousand records and load time is not an issue, keep it simple.
Pick the tool that matches your query complexity, not just your storage needs.