When a function isn't enough
You wrote a function that fetches user data from a database. It works. Now your team wants logging. You wrap the function in a call that prints the request and response. Then they want retries. You add a loop that catches errors and tries again. Then timeouts. You add a tokio::select! block. The code looks like a bowl of spaghetti. The core logic is buried under three layers of match statements and if blocks. Adding a new concern means editing the middle of the function and risking bugs in the retry or timeout logic.
tower solves this by making wrapping first-class. You define your core logic as a Service. You add behaviors like logging, retries, and timeouts as Layers. You stack them with ServiceBuilder. The result is a single service that does everything, but the code stays flat. Each concern lives in its own layer. The core logic never touches middleware code. This pattern powers axum, tonic, and hyper. It is the standard way to compose async services in Rust.
The Service trait: a contract, not a function
A Service looks like a function, but it is more. A function takes input and returns output. A Service takes input and returns a Future of output. That future might do I/O, wait for a database, or sleep. The trait also has a poll_ready method. This is backpressure. The service can say it is busy and ask the caller to wait. This lets the service manage its own load without the caller guessing.
The trait defines four associated types. Request is the input type. Response is the output type. Error is the error type. Future is the type of the future returned by call. The future must be an associated type because it might borrow from self. If call returned impl Future, the future could not easily capture references to the service state. The associated type allows the future to hold &mut self or parts of it.
use tower::Service;
use std::task::{Context, Poll};
use std::future::ready;
/// Echoes the request string back to the caller.
struct EchoService;
impl Service<&str> for EchoService {
// WHY: The response is a String.
type Response = String;
// WHY: This service never fails.
type Error = std::convert::Infallible;
// WHY: The future is ready immediately.
type Future = std::future::Ready<Result<Self::Response, Self::Error>>;
/// Checks if the service is ready to accept a request.
fn poll_ready(&mut self, _cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>> {
// WHY: Always ready for this simple service.
Poll::Ready(Ok(()))
}
/// Processes the request and returns a future.
fn call(&mut self, req: &str) -> Self::Future {
// WHY: Return a ready future with the echoed string.
ready(Ok(format!("Echo: {}", req)))
}
}
The poll_ready method returns Poll<Result<(), Error>>. It returns Poll::Ready(Ok(())) when the service can accept a request. It returns Poll::Pending when the service is busy. It returns Poll::Ready(Err(e)) when the service has failed permanently. The caller must check poll_ready before calling call. If you call call on a service that is not ready, the behavior is undefined. The service might panic, return an error, or produce garbage. The contract requires the check.
Calling a service: the ready pattern
Calling a service directly is painful. You have to poll poll_ready in a loop, handle the result, call call, then poll the future. tower provides ServiceExt to hide this boilerplate. You import ServiceExt and call .ready().await.call(req). The extension handles the polling loop and returns a future that resolves to the response.
use tower::{Service, ServiceExt};
use std::future::ready;
struct EchoService;
impl Service<&str> for EchoService {
type Response = String;
type Error = std::convert::Infallible;
type Future = std::future::Ready<Result<Self::Response, Self::Error>>;
fn poll_ready(&mut self, _cx: &mut std::task::Context<'_>) -> std::task::Poll<Result<(), Self::Error>> {
std::task::Poll::Ready(Ok(()))
}
fn call(&mut self, req: &str) -> Self::Future {
ready(Ok(format!("Echo: {}", req)))
}
}
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let mut service = EchoService;
// WHY: ready() polls poll_ready until the service is ready.
// WHY: call() invokes the service and returns the response.
let response = service.ready().await?.call("Hello").await?;
println!("{}", response);
Ok(())
}
If you forget to import ServiceExt, the compiler rejects you with E0599 (no method named ready found). The method lives in the extension trait, not the core trait. The extension trait is in the prelude of tower, but you still need to bring it into scope if you are not using tower::prelude::*. The convention is to import ServiceExt explicitly or use the prelude.
The ready() method returns a future that polls poll_ready until it returns Poll::Ready. If poll_ready returns Poll::Pending, ready() yields and waits. If poll_ready returns an error, ready() returns the error. This ensures you never call call on a busy service. Trust the extension. It enforces the contract.
Layers: wrapping behavior
A Layer is a function that takes a Service and returns a new Service. The new service wraps the old one. It can intercept requests, modify responses, or add timeouts. A layer does not run code when you create it. It returns a service that runs code when called. This separation lets you compose layers without executing logic until the service is actually used.
Writing layers manually involves boilerplate. You have to implement Layer and return a struct that implements Service. tower provides LayerFn for simple closures. You pass a closure that takes a service and returns a new service. The closure can wrap the service in a struct or return a boxed service.
use tower::{Service, ServiceExt, layer::LayerFn};
use std::future::ready;
use std::pin::Pin;
use std::task::{Context, Poll};
struct LoggingService<S> {
inner: S,
}
impl<S, Req> Service<Req> for LoggingService<S>
where
S: Service<Req>,
S::Future: std::marker::Unpin,
Req: std::fmt::Debug,
S::Response: std::fmt::Debug,
{
type Response = S::Response;
type Error = S::Error;
type Future = Pin<Box<dyn std::future::Future<Output = Result<S::Response, S::Error>>>>;
fn poll_ready(&mut self, cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>> {
self.inner.poll_ready(cx)
}
fn call(&mut self, req: Req) -> Self::Future {
println!("Request: {:?}", req);
let fut = self.inner.call(req);
Box::pin(async move {
let result = fut.await;
println!("Response: {:?}", result);
result
})
}
}
// WHY: LayerFn creates a layer from a closure.
let logging_layer = LayerFn::new(|service| LoggingService { inner: service });
The LoggingService wraps the inner service. It prints the request before calling the inner service. It prints the response after the inner service returns. The call method returns a boxed future because the future captures the async block. The Unpin bound on S::Future is needed to poll the inner future inside the async block. This is a common pattern in tower layers.
Convention aside: keep layers small and focused. A layer should do one thing. Logging, retries, timeouts, metrics. If a layer does too much, it becomes hard to test and reuse. Split it into multiple layers. Stack them with ServiceBuilder.
ServiceBuilder: stacking without the boilerplate
Stacking layers manually means calling layer.layer(service) repeatedly. The types get messy. You have to track the type of the wrapped service at each step. ServiceBuilder collects layers and applies them all at once. You call .layer() multiple times, then .service(). The builder returns the fully wrapped service. The type is inferred. The code stays clean.
use tower::{Service, ServiceExt, ServiceBuilder, layer::LayerFn};
use std::future::ready;
use std::pin::Pin;
use std::task::{Context, Poll};
struct EchoService;
impl Service<&str> for EchoService {
type Response = String;
type Error = std::convert::Infallible;
type Future = std::future::Ready<Result<Self::Response, Self::Error>>;
fn poll_ready(&mut self, _cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>> {
Poll::Ready(Ok(()))
}
fn call(&mut self, req: &str) -> Self::Future {
ready(Ok(format!("Echo: {}", req)))
}
}
struct LoggingService<S> {
inner: S,
}
impl<S, Req> Service<Req> for LoggingService<S>
where
S: Service<Req>,
S::Future: std::marker::Unpin,
Req: std::fmt::Debug,
S::Response: std::fmt::Debug,
{
type Response = S::Response;
type Error = S::Error;
type Future = Pin<Box<dyn std::future::Future<Output = Result<S::Response, S::Error>>>>;
fn poll_ready(&mut self, cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>> {
self.inner.poll_ready(cx)
}
fn call(&mut self, req: Req) -> Self::Future {
println!("Request: {:?}", req);
let fut = self.inner.call(req);
Box::pin(async move {
let result = fut.await;
println!("Response: {:?}", result);
result
})
}
}
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let logging_layer = LayerFn::new(|service| LoggingService { inner: service });
// WHY: ServiceBuilder collects layers and applies them to the service.
// WHY: The first layer added is the outermost wrapper.
let mut service = ServiceBuilder::new()
.layer(logging_layer)
.service(EchoService);
let response = service.ready().await?.call("Hello").await?;
println!("{}", response);
Ok(())
}
Order matters. ServiceBuilder applies layers in the order you add them. The first layer becomes the outermost wrapper. Requests hit the first layer, then the second, then the service. If you add a timeout layer before a retry layer, the timeout applies to the whole retry loop. If you add it after, the timeout applies to each attempt. Think about the order carefully. The first layer you add sees the request first.
Convention aside: use ServiceBuilder for all service composition. Do not manually wrap layers. The builder handles type inference and reduces boilerplate. It is the standard way to build services in the tower ecosystem.
Pitfalls and compiler errors
Services are &mut self. You cannot share a service across threads without Sync. Most tower services are Clone. Clone the service per request or per task. If you try to send a non-Send future across threads, the compiler rejects you with E0277 (trait bound not satisfied). Ensure your service and future types implement Send if you use them in multi-threaded contexts.
If you forget to check poll_ready, you break the contract. The compiler will not stop you. The service might panic or misbehave at runtime. Always use ServiceExt::ready() or implement the polling loop correctly. The extension trait exists to prevent this mistake. Use it.
If you try to call a service with the wrong request type, the compiler rejects you with E0308 (mismatched types). The Service trait is generic over the request type. The type must match exactly. You can use type aliases or generics to make the service more flexible.
If you try to use a service that does not implement Clone where a clone is needed, the compiler rejects you with E0277. Many tower layers require the inner service to be Clone. Implement Clone for your service or wrap it in Arc if it is Sync.
Counter-intuitive but true: the more layers you add, the harder it becomes to debug. Each layer adds a frame to the stack trace and a step in the request flow. Keep the stack shallow. Use layers sparingly. Profile before adding complexity.
Decision: when to use tower
Use tower when you need composable middleware for network services, background workers, or any async request-response pattern. Use tower when you want to share middleware logic across multiple services without duplicating code. Use tower when you are building a library that needs to integrate with the Rust async ecosystem.
Reach for plain async functions when your logic is linear and does not need cross-cutting concerns like retries, timeouts, or logging. Plain functions are simpler and have less overhead. Do not use tower for simple scripts or one-off tasks.
Pick a full framework like axum when you are building a web API and want routing, extraction, and response handling out of the box. axum uses tower under the hood. You get the benefits of tower without writing the boilerplate. Use axum for web servers. Use tower for lower-level composition or non-web services.
Treat tower as the foundation. Build your services on top of it. Stack layers carefully. Trust the contract. The compiler and the ecosystem will reward you.