Atlassian's Cache Invalidation Evolution: From SNS+SQS Push to S3 Pull Architecture

The Enduring Challenge of Cache Invalidation

In the realm of computer science, few problems are as notoriously difficult as cache invalidation. Often cited alongside naming things as one of the two hardest problems, effective cache invalidation is crucial for ensuring that applications serve fresh, consistent data without incurring the latency of repeatedly hitting the primary data source. This deep dive explores Atlassian’s real-world experience with cache invalidation for a critical service, detailing their initial approach, the scalability challenges they faced, and their eventual re-architecture to a more robust solution.

Why Cache Invalidation Matters

When data is cached, it’s stored closer to the consumer for faster access. However, the source of truth (e.g., a database like MySQL or PostgreSQL) remains the definitive record. The core challenge of cache invalidation is to ensure that whenever data changes in the source of truth, the corresponding cached data is promptly marked as stale or removed. Failing to do so can lead to applications serving outdated information, impacting user experience and data integrity.

Atlassian’s Tenant Context Service (TCS)

Our discussion centers around Atlassian’s Tenant Context Service (TCS), a foundational service responsible for managing metadata. The TCS architecture comprises:

• Source of Truth: DynamoDB, serving as the primary data store.
• Ingestion Workers: Processes that perform writes and updates to the DynamoDB.
• TCS API: An API server that clients query to retrieve metadata.

Client Interaction via Sidecar Pattern

Clients consuming data from the TCS typically employ a sidecar pattern. In this setup:

• A main application (e.g., a web server) focuses on its core business logic.
• A sidecar container runs alongside the main application within the same pod or server.
• The sidecar is responsible for interacting with the TCS API, fetching metadata, and crucially, caching this data locally.

This local caching within each sidecar is key. It means the cache is distributed across many compute nodes rather than being centralized. The main application queries its local sidecar for data, benefiting from low-latency access.

Initial Cache Invalidation Architecture: Push-based with SNS + SQS

Atlassian’s initial approach to invalidating these distributed local caches was a push-based model leveraging AWS SNS (Simple Notification Service) and SQS (Simple Queue Service).

The Invalidation Flow

1. Data Change: An Ingestion Worker updates data in DynamoDB.
2. Invalidation Request: Immediately after the database update, the Ingestion Worker sends an invalidation message to an AWS SNS Topic.
3. Fan-out to SQS: The SNS topic, acting as a broadcast mechanism, fans out this invalidation message to multiple AWS SQS Queues that have subscribed to it.
4. Per-Pod SQS Consumption: Critically, each SQS queue was dedicated to a single compute node or pod. This meant one SQS queue per sidecar instance.
5. Local Cache Invalidation: The sidecar consumes the invalidation message from its dedicated SQS queue and invalidates the corresponding entry in its local cache.
6. Subsequent Request: When a subsequent request for the invalidated key arrives, the sidecar, finding the data missing or stale in its local cache, fetches the fresh data from the TCS API, re-caches it, and serves it.

Here’s a simplified representation of the initial architecture:

graph TD
    subgraph Ingestion System
        IW[Ingestion Worker]
    end

    subgraph Source of Truth
        DB[DynamoDB]
    end

    subgraph Invalidation System (Initial)
        IW --> DB
        IW --