When a loop isn't enough
You have a binary protocol where packets arrive in chunks. You need to reconstruct full messages from a stream of bytes. A while loop works, but it scatters your logic across index variables, boundary checks, and state flags. You want to write for packet in stream and let the loop handle the plumbing. That requires a custom iterator.
Custom iterators let you package complex traversal logic into a type that behaves exactly like a Vec or a Range. You get access to map, filter, collect, and the ergonomic for loop syntax without fighting the compiler.
The iterator as a state machine
An iterator is a state machine that hands you one item at a time. Think of a ticket dispenser at a deli. You don't see the whole stack of tickets. You press a button, and the machine gives you the next number. It remembers the current number inside. When the roll runs out, it tells you "no more."
In Rust, that button press is the next method. The machine's internal memory is the struct you define. The Iterator trait is the contract: you provide the state and the next logic, and the language gives you the rest.
Minimal example
Start with a struct that holds the state. Implement Iterator by defining type Item and the next method.
struct Counter {
current: u32,
}
impl Iterator for Counter {
// The Item associated type tells the compiler what `next` returns inside `Some`.
type Item = u32;
fn next(&mut self) -> Option<Self::Item> {
// Check the boundary before incrementing to avoid overflow on the last item.
if self.current < 5 {
self.current += 1;
Some(self.current)
} else {
// Returning None signals the iterator is exhausted.
None
}
}
}
fn main() {
let counter = Counter { current: 0 };
// The `for` loop calls `next` until it receives `None`.
for num in counter {
println!("{num}");
}
}
Convention aside: iterator structs usually end in Iter. Naming your type CounterIter signals to readers that this type implements Iterator. It's a small signal that pays off in large codebases.
How the compiler uses your iterator
When you write for num in counter, the compiler rewrites that loop. It creates a mutable borrow of the iterator and calls next in a tight loop. If next returns Some(value), the loop body runs with that value. If next returns None, the loop breaks immediately.
This is why next takes &mut self. The iterator must be able to update its internal state every time you ask for the next item. If you try to call next on an immutable reference, the compiler rejects you with E0596 (cannot borrow as mutable). The iterator owns the progress; the loop just consumes the output.
There is a bridge you should know about. The for loop actually calls IntoIterator::into_iter, not Iterator::next directly. For a custom struct, you implement Iterator, and the compiler provides a blanket implementation of IntoIterator that simply returns self. This means your custom type can be used in for loops and passed to methods expecting IntoIterator, just like a Vec. You don't need to implement IntoIterator manually unless you want to change how the type is consumed, such as returning a different iterator type.
Adding size hints
If your iterator knows how many items are left, tell the compiler. Implement size_hint to return a tuple of (lower_bound, Option<upper_bound>). This helps methods like collect allocate the exact capacity upfront, avoiding reallocations.
impl Iterator for Counter {
type Item = u32;
fn next(&mut self) -> Option<Self::Item> {
if self.current < 5 {
self.current += 1;
Some(self.current)
} else {
None
}
}
fn size_hint(&self) -> (usize, Option<usize>) {
// Calculate remaining items. The lower and upper bounds match because the count is exact.
let remaining = 5 - self.current;
(remaining, Some(remaining))
}
}
If you provide an accurate size_hint, collect can call Vec::with_capacity once. Skipping this forces Vec to grow geometrically, which wastes time and memory on large datasets.
Realistic example: chunking a slice
Real iterators often wrap existing data. You rarely allocate inside next. You usually return references to data you hold. This introduces lifetimes. The iterator must live shorter than the data it points to.
struct ChunkIter<'a, T> {
data: &'a [T],
chunk_size: usize,
offset: usize,
}
impl<'a, T> Iterator for ChunkIter<'a, T> {
// The iterator yields slices of the original data, borrowing the lifetime 'a.
type Item = &'a [T];
fn next(&mut self) -> Option<Self::Item> {
// Stop when we've consumed all data.
if self.offset >= self.data.len() {
return None;
}
// Calculate the end of the current chunk, clamping to the slice length.
let end = std::cmp::min(self.offset + self.chunk_size, self.data.len());
let chunk = &self.data[self.offset..end];
// Advance the offset for the next call.
self.offset = end;
Some(chunk)
}
}
fn main() {
let numbers = [1, 2, 3, 4, 5, 6, 7];
let chunks = ChunkIter {
data: &numbers,
chunk_size: 3,
offset: 0,
};
for chunk in chunks {
println!("{:?}", chunk);
}
}
The lifetime 'a ties the iterator to the slice. The compiler ensures you can't return a ChunkIter that outlives numbers. If you try to store the iterator in a struct that lives longer than the data, you'll hit a lifetime error. Don't fight the lifetime checker here. The iterator must live shorter than the data.
Convention aside: use Self::Item in the return type of next rather than Item. It's clearer and avoids ambiguity if you have nested scopes. The community standard is Option<Self::Item>.
Pitfalls and errors
Custom iterators introduce a few traps. The most common is an infinite loop. If your next logic has a bug and never returns None, your program hangs. There is no compiler check for termination. You must prove None is reachable. Write the termination condition explicitly and test the boundary.
Another issue is E0277 (trait bound not satisfied) when you try to collect the result. The compiler needs to know the target type. If you write let vec = iter.collect();, the compiler might not infer Vec. Add the type annotation: let vec: Vec<_> = iter.collect();. This tells the compiler to look for FromIterator for Vec.
Watch out for borrowing conflicts. If your iterator holds a mutable reference and you try to return a reference from next, you might trigger E0502 (cannot borrow as mutable because it is also borrowed as immutable). The iterator's internal state and the yielded item must not overlap in a way that violates borrowing rules. If you need to yield mutable references, look into DoubleEndedIterator or redesign the state to avoid conflicts.
Prove termination in your head. The compiler won't save you from an infinite loop.
When to use a custom iterator
Rust offers several ways to process sequences. Pick the right tool based on the complexity of the logic and the state you need to maintain.
Use a custom iterator when the generation logic requires state that doesn't fit a simple closure, like parsing a stream, maintaining a sliding window, or tracking complex indices across multiple data structures.
Use std::iter::successors when you can define the next item purely as a function of the current item, avoiding the boilerplate of a struct. It's perfect for sequences like Fibonacci numbers or traversing a linked list where each node points to the next.
Use a chain of map, filter, and take when you are transforming an existing collection and the logic fits functional composition. Adapters are lazy and composable; they often replace the need for a custom struct.
Reach for a while loop when you need to mutate external state inside the loop body or when the iteration pattern breaks the iterator contract, such as yielding multiple items per call or requiring side effects that next cannot express cleanly.
Reach for successors first. Write the struct only when the closure runs out of room.