CDN and Object Storage Explained for System Design

What a CDN does#

A content delivery network is a set of servers around the world that keep copies of your files. Users download from a nearby server, rather than from your data centre.

Without CDN:  User in Chennai --> Server in the USA   (slow)
With CDN:     User in Chennai --> Edge in Chennai     (fast)
                                   | miss
                                   v
                                 Your origin server

Benefits are lower latency, less load on your servers, and protection from traffic spikes. Cloudflare, Fastly, Akamai and AWS CloudFront are examples.

CDNs suit images, video, CSS and JavaScript. Many can also cache API responses.

Pull and push#

Cache headers#

Your server tells the CDN and browser how long to keep a file:

Cache-Control: public, max-age=31536000, immutable

A year sounds risky, but it is safe with fingerprinted names: when a file changes, its name changes.

import hashlib


def fingerprint(name, content):
    digest = hashlib.sha256(content.encode()).hexdigest()[:8]
    stem, ext = name.rsplit(".", 1)
    return f"{stem}.{digest}.{ext}"


print(fingerprint("app.js", "console.log(1)"))
print(fingerprint("app.js", "console.log(2)"))

app.0a286891.js
app.f2ae6bd0.js

The new name means no stale copies, and you never need to purge. HTML pages keep short lifetimes, and they point to the new names.

How much it saves#

requests = 10_000_000
for hit in (0.5, 0.9, 0.98):
    origin = requests * (1 - hit)
    print(f"Cache hit {hit:.0%}: {origin:>9,.0f} requests reach the origin")

Cache hit 50%: 5,000,000 requests reach the origin
Cache hit 90%: 1,000,000 requests reach the origin
Cache hit 98%:   200,000 requests reach the origin

Object storage#

Files do not belong in a database, or on app servers' disks. Object storage, such as AWS S3, Google Cloud Storage or Azure Blob Storage, stores files as objects with a key.

It is cheap and very durable, keeping copies across devices, but it is not for frequently changing data.

The usual pattern: store the file in object storage, and its metadata in a database:

photos(id, user_id, storage_key, size, created_at)  --> database
users/42/photo.jpg                                  --> object storage

Direct uploads#

Do not send large uploads through your app servers. Give the client a short-lived pre-signed URL, and it uploads straight to storage:

1. Client  --> API: "I want to upload photo.jpg"
2. API     --> Client: pre-signed URL (valid for 5 minutes)
3. Client  --> Object storage: PUT the file
4. Client  --> API: "Done", and the API saves the metadata

Your servers do no heavy lifting, and access stays controlled. Downloads of private files work in the same way.

Common mistakes#

Practice#

total_tb = 2000
print(f"{total_tb * (1 - 0.95):.0f} TB from the origin")

100 TB from the origin

Key takeaway#

Next: Choosing a Database.