When loops get in the way
You have a stream of raw data. Maybe network packets, maybe a CSV file, maybe a list of user actions. You need to filter out the noise, transform the shape, and gather the results. The instinct is to write a loop, push to a temporary vector, loop again, push to another vector. That works, but it burns memory and obscures intent. Rust offers a different path: chain small operations together so the data flows through a pipeline without ever stopping. This is the iterator pattern. It turns scattered logic into a single, readable expression that the compiler optimizes into tight machine code.
The assembly line model
Think of an assembly line. A car chassis moves along a conveyor belt. Station one welds the doors. Station two installs the engine. Station three paints the body. The car doesn't stop and get stored in a warehouse between stations. It flows. Each station is a small, focused operation. The whole line produces a finished car without intermediate storage.
Rust iterators work the same way. You define a source of items, then attach adapters like filter and map. The adapters form a chain. When you pull a value from the end, the chain works backward, pulling one item from the source, passing it through each adapter, and yielding the result. No intermediate vectors. No wasted allocations. Just pure flow.
Minimal pipeline
Here is the basic structure. You start with a collection, consume it, chain adapters, and collect the result.
fn main() {
// Start with a concrete collection on the stack/heap.
let numbers = vec![1, 2, 3, 4, 5];
// Chain operations to define the pipeline.
// into_iter() consumes the vec and yields owned i32 values.
// filter() keeps only even numbers.
// map() doubles each surviving number.
// collect() pulls the stream into a new Vec.
let result: Vec<i32> = numbers
.into_iter()
.filter(|&n| n % 2 == 0)
.map(|n| n * 2)
.collect();
assert_eq!(result, vec![4, 8, 12]);
}
Convention aside: the compiler needs to know what collect produces. You can annotate the variable type as shown, or use the turbofish syntax collect::<Vec<i32>>(). The community often prefers the variable annotation for readability, but the turbofish shines when you need the result inline without a binding.
How the chain executes
The key insight is laziness. Calling filter or map doesn't do any work. It just returns a new iterator object that remembers the previous iterator and the closure. Nothing happens until you call collect or loop over the result. This is zero-cost abstraction. The compiler sees the whole chain and inlines the logic. The generated assembly often looks like a single for loop with an if and a multiply. You get the clarity of high-level composition with the performance of hand-written loops.
Under the hood, every iterator implements a next() method. collect calls next repeatedly until it returns None. Each adapter wraps the previous iterator. When collect asks the map adapter for a value, map asks filter for a value. filter asks into_iter for a value. into_iter yields from the vector. The value bubbles up, transformed at each step.
Trust the compiler to inline the chain. Write the pipeline, not the loop.
Realistic data processing
Let's look at a realistic case. You're reading a configuration file where each line is a key-value pair. Some lines are comments. Some are malformed. You need to parse valid pairs into a HashMap.
use std::collections::HashMap;
/// Parses raw config lines into a key-value map, skipping comments and errors.
fn parse_config(raw_lines: &[&str]) -> HashMap<String, String> {
raw_lines
.iter()
// Skip comments and empty lines.
// iter() yields &str references, preserving the slice.
.filter(|line| !line.starts_with('#') && !line.trim().is_empty())
// Split on '=' and handle split errors gracefully.
// filter_map combines filtering and transformation.
.filter_map(|line| {
// splitn(2, '=') is efficient; it stops after the first split.
let parts: Vec<&str> = line.splitn(2, '=').collect();
if parts.len() == 2 {
Some((parts[0].trim().to_string(), parts[1].trim().to_string()))
} else {
None
}
})
// HashMap implements FromIterator<(K, V)>.
.collect()
}
Use filter_map to combine filtering and transformation. It keeps the pipeline tight.
Async iteration and Pin
Async iterators add complexity. The standard library doesn't have a stable AsyncIterator trait yet. You usually rely on crates like futures or tokio-stream. The core idea is the same, but the machinery involves Pin and Poll because async state machines can't move in memory.
Async iterators introduce a structural constraint. The iterator state might hold a reference to itself while waiting for a future to resolve. If the state moves in memory, that reference dangles. Rust prevents this with Pin. The Pin type guarantees the memory address won't change. When you implement poll_next, the first argument is Pin<&mut Self>. You can't just call self.count. You must use self.get_mut() to access fields safely. This boilerplate protects you from subtle memory bugs in async code.
use std::pin::Pin;
use std::task::{Context, Poll};
/// AsyncIterator mirrors sync Iterator but returns Poll<Option<T>>.
/// In practice, use the Stream trait from the futures crate.
trait AsyncIterator {
type Item;
fn poll_next(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Option<Self::Item>>;
}
struct AsyncCounter {
count: usize,
}
impl AsyncIterator for AsyncCounter {
type Item = usize;
fn poll_next(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll<Option<Self::Item>> {
// get_mut() safely accesses fields inside Pin.
let this = self.get_mut();
if this.count < 5 {
this.count += 1;
Poll::Ready(Some(this.count))
} else {
Poll::Ready(None)
}
}
}
Convention aside: don't roll your own async iterator trait. Use futures::Stream. It provides the standard trait and adapters like filter and map that work with await. Implementing Stream is the community standard.
Async iterators are the backbone of streaming data. Master Pin early, or rely on a runtime crate to hide the boilerplate.
Custom collections with FromIterator
You can make your own types work with collect. Implement FromIterator. This trait defines how an iterator builds your type. The trait bound I: IntoIterator<Item = T> allows your implementation to accept slices, arrays, and other iterators, not just Iterator objects.
use std::iter::FromIterator;
/// A wrapper around Vec<i32> that integrates with iterator pipelines.
struct MyCollection(Vec<i32>);
impl FromIterator<i32> for MyCollection {
/// Constructs MyCollection from any iterator yielding i32.
fn from_iter<I: IntoIterator<Item = i32>>(iter: I) -> Self {
// Delegate to Vec's implementation.
// iter.collect() consumes the iterator and builds the inner vec.
MyCollection(iter.collect())
}
}
fn main() {
let data = vec![1, 2, 3];
// collect() uses FromIterator to build MyCollection.
let collection: MyCollection = data.into_iter().collect();
}
Implement FromIterator to make your types first-class citizens in the pipeline ecosystem.
Pitfalls and compiler signals
Iterators are powerful, but ownership rules still apply. The compiler will catch mistakes early.
If you call iter() on a Vec<String>, you get references &String. If you try to map to a type that needs ownership, the compiler rejects you with E0507 (cannot move out of borrowed content). Use into_iter() to take ownership, or iter_mut() to modify in place.
collect is generic. Without a type hint, the compiler doesn't know whether to build a Vec, HashMap, or HashSet. You'll get a "cannot infer type" error. Annotate the variable or use turbofish.
Closures capture by reference by default. If you need ownership inside the closure, use move. Forgetting move in a closure that outlives the scope causes E0373 (closure may outlive the current function).
Read the borrow checker errors as clues about data flow. The compiler tells you exactly where ownership breaks. Fix the flow, don't fight the rules.
Decision matrix
Use iter() when you need to read values without taking ownership. The source collection stays valid for later use.
Use iter_mut() when you need to modify values in place. The source collection remains borrowed mutably.
Use into_iter() when you need owned values or want to consume the collection. The source is moved and no longer accessible.
Use filter_map when you need to both transform and filter in one step. It avoids creating intermediate Option wrappers.
Use enumerate when you need the index alongside the value. It keeps the pipeline functional instead of forcing a manual counter.
Use chunks or windows when you need to process overlapping or grouped subsets. These adapters yield slices of the original data.
Use Stream from the futures crate for async iteration. It provides the standard trait and adapters that work with await.
Pick the iterator method that matches your ownership needs. The wrong choice blocks the pipeline before it starts.