Cost of Cloud Egress: An Honest Accounting for a Mid-Size SaaS
We didn't notice egress was 18% of our cloud bill until a board review forced a line-item audit. The architecture changes that followed were uncomfortable but necessary.
We didn't notice egress was 18% of our cloud bill until a board review forced a line-item audit. The architecture changes that followed were uncomfortable but necessary — mid-size B2B SaaS, ~$85k/month AWS, 14 engineers, no dedicated FinOps hire.
Answer first
Map every gigabyte out by service (CloudFront, S3 GET to internet, cross-AZ, NAT gateway, RDS snapshot export). Egress hides in "data transfer" line items. Fix architecture (region locality, CDN cache keys, privateLink, compress payloads, stop cross-AZ chatty microservices) before negotiating Enterprise Discount — discount on waste is still waste.
How we found 18%
Board deck asked "unit economics improving?" — we showed compute flat, revenue up, margin flat. CFO exported Cost Explorer grouped by Usage Type containing DataTransfer:
| Category | $/month | % of bill |
|---|---|---|
| EC2 compute | 31k | 36% |
| RDS | 22k | 26% |
| Data transfer out | 15.3k | 18% |
| S3 storage | 8k | 9% |
| Other | 8.7k | 11% |
We previously bucketed transfer under "misc infrastructure."
Top 3 egress hogs
- Cross-AZ replication — Kafka consumers in AZ-b reading producers AZ-a — 4.2k/month
- S3 image thumbnails without CloudFront — mobile app refetch — 3.8k/month
- NAT gateway per-AZ for outbound API calls to same-region partner VPC — 2.1k/month
Postgres REINDEX maintenance later added a 4th blip — 1.2 TB read billed as intra-region but snapshot export to analyst account was cross-region $900 one-time — caught in second audit pass.
Honest accounting method
Step 1: Cost Explorer saved report
Filter: Service = EC2-Other, CloudFront, S3, Amazon VPC + usage type *DataTransfer*. Granularity daily. Tag by Environment=prod.
Step 2: VPC Flow Logs sample
Enable 1% sample to S3 — Athena query top talkers by dstaddr outside VPC CIDR. Found forgotten telemetry batch job mirroring logs cross-region "for DR" with no reader.
Step 3: Application metric correlation
Added bytes_sent_total by endpoint in Prometheus — matched API /export/csv spike to NAT costs.
Step 4: Unit economics
egress_$ / active_customer / month — baseline $0.42; target $0.18 after fixes.
Architecture changes (uncomfortable)
Kafka: AZ-aware consumption
Changed consumer group assignment to prefer same-AZ brokers; accepted slight skew risk. Saved ~3.1k/month. Tradeoff: rebalance complexity during AZ failure — tested in game day.
Thumbnails: CloudFront + immutable cache keys
Cache-Control: public, max-age=31536000, immutable on content-addressed paths. App release busts URL prefix not query string hack.
Saved ~2.9k/month. Tradeoff: invalidation workflow for illegal content takedown — legal process doc updated.
NAT: VPC endpoints + PrivateLink
S3 and Dynamo via gateway endpoints (free). Partner API via PrivateLink — eliminated NAT GB for that path.
Saved ~1.7k/month. Tradeoff: 2 weeks eng for PrivateLink setup; partner charged $0.01/GB — still net win.
Region consolidation
Staging in us-west-2, prod us-east-1 — nightly sync copied 200 GB/week for "realistic data." Moved staging snapshot generation to synthetic subset in prod region.
Saved ~0.8k/month + sanity.
None of these were "turn off a toggle" — product and partner negotiations involved.
What we deliberately didn't do
- Multi-cloud egress discount arbitrage — ops cost > savings at our scale
- Aggressive video compression — not our payload
- Charge customers for bandwidth — competitive market; internalize efficiency instead
Policy changes
- Architecture review must answer "cross-AZ trips per request?" for new services
- RFC template adds
estimated_monthly_egress_$— see writing RFCs that get read - Monthly 15-min eng review of Cost Explorer transfer line — rotating owner, not FinOps silo
- Database maintenance windows estimate IO + transfer — ties vacuum tuning
Results after 2 quarters
- Egress: 15.3k → 6.1k/month (~60% reduction)
- Total bill: 85k → 74k — not all egress; some rightsizing same period
- p99 latency improved from CDN — side benefit
- Engineer morale: annoyed at first ("another review gate"), neutral after savings funded staging GPU for ML experiment — tangible trade
Board narrative that worked
"We were spending $183k/year shipping bits customers already paid us to store locally once." Graphic: diagram of thumbnail refetch loop. Execs understand loops.
Tooling wishlist
- AWS Cost Anomaly Detection on DataTransfer — configured after audit; caught regression when intern disabled CloudFront in dev account mirroring prod config wrong
- Infracost in CI — rough; doesn't model egress well yet
What I'd do next
Implement egress budget per service in Terraform tags — alert at 120% forecast mid-month.
Evaluate single-region DR with warm standby vs cross-region replication cost — DR policy is egress policy in disguise.
Cloud egress is a architecture smell meter. If it's 18% and you didn't know, your observability story isn't finished — and neither is your system design.
Unit economics slide that landed
| Metric | Before | After |
|---|---|---|
| Egress $/customer/mo | $0.42 | $0.17 |
| Gross margin | 61% | 64% |
| p99 API latency | 380 ms | 290 ms |
Latency win from CDN — egress project paid for itself in retention narrative, not just infra line.
Partner negotiation script
PrivateLink request: "We're paying NAT tax to call your API in same region." Most B2B vendors have done it — 3-week legal, 2-week tech.
What I'd do next
Manish Bookreader
Electronics enthusiast, Embedded Systems Expert, Linux/Networking programmer, and Software Engineer passionate about AI, electronics, books, and cooking.
