Relay — Why We Rebuilt Our Email Pipeline

After processing two billion messages, we discovered the bottleneck was never our SMTP servers — it was the queue sitting in front of them.

In January 2024, our delivery pipeline processed its two-billionth email. Instead of celebrating, we found a flaw no amount of horizontal scaling could resolve: the queue had become the product's real bottleneck.

The coordination tax

The original design fanned every request across configured SMTP providers. At low volume, it felt simple. Past 500 million messages per month, provider coordination consumed more CPU than encoding or transmission, and the shared state machine became the dominant cost center.

// Before: sequential, blocking pipeline
await transporter.sendMail(msg);
pipeline.push({ msg, retry: true });
// → ● Delivered in 340ms

Why We Rebuilt Our Email Pipeline From Scratch

The coordination tax