Disabling Python Garbage Collection at Scale: How Instagram Saved 10% Infrastructure Cost

In 2017, Instagram published an engineering post detailing an optimization that seems entirely counter-intuitive: they disabled Python’s garbage collection (GC) on their web fleet and gained a 10% global capacity improvement along with a 25% reduction in RAM usage.

At hyperscale, a 10% reduction in infrastructure footprint does not just mean fewer servers—it translates directly to millions of dollars in saved annual operational expenditure. To understand why removing a memory reclamation mechanism actually reduced memory consumption and boosted performance, we must peel back the layers of CPython internals, Linux kernel memory management, and multi-process web server architectures.

---

The Architecture: Multi-Process Django and uWSGI

Instagram’s core backend server runs as a large Django application. Because CPython utilizes a Global Interpreter Lock (GIL), a single Python process can only execute bytecodes on one CPU core at any given instant, regardless of how many cores the physical machine provides.

To fully saturate multi-core bare-metal hosts and VMs (e.g., 16, 32, or 64 cores), web applications rely on a multi-process architecture orchestrated by WSGI application servers like uWSGI or Gunicorn.

+---------------------------------------------------------------------+
| Host Machine (Multi-Core)                                           |
|                                                                     |
|  +--------------------+                                             |
|  | uWSGI Master       | (Pre-loads Django codebase, modules,        |
|  | Process            |  allocates static structures)               |
|  +---------+----------+                                             |
|            | forks via Copy-on-Write (COW)                          |
|            +------------------+------------------+                  |
|            |                  |                  |                  |
|            v                  v                  v                  |
|    +---------------+  +---------------+  +---------------+          |
|    | Worker 1      |  | Worker 2      |  | Worker N      |          |
|    | (Core 1)      |  | (Core 2)      |  | (Core N)      |          |
|    +---------------+  +---------------+  +---------------+          |
+---------------------------------------------------------------------+

The Prefork Pattern

In prefork mode:

1. The uWSGI Master process boots up, parses the application code, imports libraries, and allocates common shared objects into memory.
2. Once initialized, the master calls fork() to spawn dozens of child worker processes.
3. Each worker process binds to a core and handles incoming web requests concurrently.

Copy-on-Write (COW) Semantics

When a child process is forked in Linux, the OS kernel does not duplicate the parent process’s physical memory pages immediately. Instead, both parent and child map the exact same physical memory pages marked as read-only. This is Copy-on-Write (COW):

• As long as workers merely read memory (such as loaded module code, global constants, and lookup tables), the pages remain shared.
• When either process attempts to write to a shared page, the CPU triggers a page fault exception. The Linux kernel intercepts the fault, allocates a new 4KB physical page, copies the original data into it, updates the child’s page table with write permissions, and then resumes execution.

In theory, preforking saves hundreds of megabytes per worker because all workers share the pre-warmed application footprint.

The uWSGI Safety Valve: reload-on-rss

Python workloads running large dependency trees frequently experience memory bloat or small native-extension leaks. In production, engineering teams mitigate Out-of-Memory (OOM) risks by using uWSGI worker recycling strategies:

• reload-on-rss: Automatically terminates and re-spawns a worker once its Resident Set Size (RSS) crosses a configured threshold (e.g., 500 MB).
• max-requests: Restarts the worker after processing a fixed number of requests.

This keeps process lifetimes bounded, an operational reality that becomes vital later in this optimization journey.

---

The Mystery: Rapid Degradation of Shared Memory

Instagram observed an anomaly upon worker startup:

• A worker started with an RSS of ~250 MB, of which almost all was shared memory inherited from the master process.
• Within seconds of serving requests, the shared memory plummeted from 250 MB down to 140 MB, converting into private, dirty memory.
• Across dozens of workers per host, this memory duplication forced the host to consume gigabytes of redundant RAM.

Why were read-only web requests mutating pages containing static application code and cached definitions?

---

Investigation and Failed Theories

Theory 1: Reference Counting Mutations

Every object in CPython is wrapped in a PyObject C-struct containing metadata:

typedef struct _object {
    _PyObject_HEAD_EXTRA
    Py_ssize_t ob_refcnt;
    struct _typeobject *ob_type;
} PyObject;

In CPython, memory management operates primarily via reference counting:

• When an object is referenced or passed to a function, ob_refcnt is incremented (Py_INCREF).
• When a reference goes out of scope, ob_refcnt is decremented (Py_DECREF).

Even reading an immutable object (like a string, tuple, or code object) requires incrementing its reference counter. Because ob_refcnt resides in the object header on the same memory page as the object itself, a read in Python turns into a write at the OS memory layer, provoking a Copy-on-Write page fault.

Attempt 1: Disabling Reference Counting on Code Objects

Instagram hypothesized that code objects were constantly triggering COW due to refcount bumps. They patched CPython to disable refcounting for code objects and deployed it to canary servers.

Result: Failed. The shared memory continued to drop at the exact same rate.

Core Lesson: Never deploy complex architectural changes based solely on intuition. Instrument the underlying mechanics and prove the root cause first.

---

Uncovering the Culprit: Linux perf and Page Faults

Because Copy-on-Write is inherently driven by OS page faults, Instagram used the Linux perf tool to record software events on active worker processes:

perf record -e page-faults -p <worker_pid>

When inspecting the resulting symbol trace with perf report, the page faults were not coming from refcounts on code objects. The vast majority of page faults originated inside:

collect.part.7
  PyObject_GC_New
  dict_real_next
  ...

collect is the core function of CPython’s cyclic Garbage Collector (gc module).

+-----------------------------------------------------------------------+
| How CPython Cyclic GC Breaks Copy-on-Write                           |
|                                                                       |
|  Doubly Linked List in Master:                                        |
|  [PyGC_Head A] <-----> [PyGC_Head B] <-----> [PyGC_Head C]           |
|  (Page 10)             (Page 11)             (Page 12)                |
|                                                                       |
|  Worker GC triggers collect():                                        |
|  1. Traverses generations.                                            |
|  2. Shuffles objects between lists (updates gc_next / gc_prev).       |
|  3. Writes new pointers directly into PyGC_Head.                      |
|                                                                       |
|  Result: Page 10, 11, and 12 are modified!                            |
|  Kernel triggers Copy-on-Write -> Shared memory is shattered.          |
+-----------------------------------------------------------------------+

How CPython Generational GC Triggers COW

CPython requires two memory management systems:

1. Reference Counting: Fast and deterministic, handles 95%+ of deallocations, but cannot collect reference cycles (e.g., Object A references Object B, and Object B references Object A).
2. Cyclic Garbage Collector: Periodically sweeps tracked container objects (lists, dictionaries, custom classes) to break isolated reference cycles.

To track objects, the GC prepends a PyGC_Head header to every container:

typedef union _gc_head {
    struct {
        union _gc_head *gc_next;
        union _gc_head *gc_prev;
        Py_ssize_t gc_refs;
    } gc;
    long double dummy;
} PyGC_Head;

CPython organizes these containers into three generations (Generation 0, 1, and 2) using doubly linked lists. When gc.collect() executes:

• It walks these linked lists and shifts surviving objects from younger generations to older generations.
• Shuffling items requires modifying gc_next and gc_prev pointers.
• Because these pointers live inside the memory pages of the tracked objects inherited from the master process, the GC’s pointer adjustments dirty the memory pages.
• This forces the OS kernel to trigger tens of thousands of page faults, duplicating almost the entire shared heap into worker-private memory.

---

Iterative Solutions: Disabling the Collector

Attempt 2: gc.disable() and Third-Party Traps

Instagram added gc.disable() at the earliest bootstrap stage of the application.

Result: Profiling showed page faults were still occurring.

Using GDB, they discovered that a third-party serialization library (msgpack) called gc.enable() under the hood during initialization, silently reactivating the collector.

Rather than patching msgpack (which leaves the door open for other libraries to do the same), they used a cleaner configuration trick:

import gc
# Setting generation thresholds to zero prevents automatic GC triggers
gc.set_threshold(0)

With GC effectively neutralized:

• Shared memory per worker increased from 140 MB to 225 MB.
• Host RAM usage dropped by 8 GB per machine (a 25% overall RAM reduction across the fleet).
• Instagram could now pack significantly more workers onto each host, yielding a 10% throughput capacity win.

---

The Shutdown Trap: Interpreter Teardown (Py_Finalize)

When rolling out this change fleet-wide, continuous deployment pipelines suddenly stalled. Reloading a web worker, which normally took under 10 seconds, suddenly required over 60 seconds on certain hardware.

Diagnosing Free Memory Collapse via atop

Using atop, they observed that whenever uWSGI recycled a worker:

1. The host’s free memory plummeted to near zero.
2. In response to extreme memory pressure, the Linux kernel evicted filesystem page caches (code pages, shared libraries, templates).
3. When the new worker spawned, it suffered massive cold-read disk I/O thrashing because its code had been evicted from the page cache.

The Final GC Surge in Py_Finalize()

When uWSGI cleanly shuts down a worker process, it invokes Py_Finalize():

• Py_Finalize() performs complete runtime cleanup: it destroys sub-interpreters, unloads modules, clears type caches, and executes one final, unconditional garbage collection pass.
• Because GC had been disabled during the worker’s life, thousands of cyclical references had accumulated.
• When Py_Finalize() executed that final collection on hundreds of megabytes of accumulated cyclic objects, memory churn peaked instantaneously, evicting the OS cache.

Why Not Just Remove Py_Finalize?

Removing Py_Finalize outright caused issues because mission-critical application shutdown logic depended on Python’s atexit hooks (e.g., flushing buffered metrics, terminating database connections, sending audit events).

---

The Final Two-Line Fix

Python’s atexit module executes registered functions in reverse registration order (LIFO - Last In, First Out).

Instagram realized they did not care about cleaning up CPython runtime internals upon process termination—the Linux kernel reclaims all allocated virtual memory, file descriptors, and sockets immediately when a process exits.

They only cared about running their own application-level teardown handlers.

By registering os._exit(0) at the very beginning of the bootstrap script, they guaranteed it would run last among all registered atexit callbacks:

import gc
import os
import atexit

# 1. Disable automatic GC collection passes deterministically
gc.set_threshold(0)

# 2. Force immediate OS-level exit after all app atexit hooks complete
atexit.register(os._exit, 0)

Execution Flow

1. At initialization, atexit.register(os._exit, 0) is placed at the bottom of the exit stack.
2. Later, Django and custom libraries register business teardown hooks (e.g., flush analytics, close connection pools).
3. On uWSGI worker recycle, atexit triggers:
   • Custom hooks execute first (LIFO order).
   • Finally, os._exit(0) is called.
4. os._exit(0) is a raw POSIX syscall (exit_group). It immediately halts the process at the kernel level without returning control to CPython’s runtime.
5. Py_Finalize(), module teardown, and the final sweeping GC are completely bypassed, eliminating the memory spike and page cache evictions.

Process Termination Lifecycle:

[Worker receives SIGINT/SIGTERM]
           |
           v
[Execute Python atexit stack (LIFO)]
  |-- 1. Flush telemetry buffers
  |-- 2. Close active RPC / DB channels
  \-- 3. Invoke os._exit(0)  <-- Registered first, executes last!
           |
           v
[Kernel reclaims address space immediately]
(Py_Finalize and final GC are completely bypassed!)

---

Why Didn’t Memory Leak Uncontrollably?

A natural question arises: If you disable garbage collection, won’t the server inevitably crash from Out-of-Memory errors?

Two factors prevented this:

1. Reference Counting Remains Fully Active: The cyclic garbage collector is only an auxiliary system in CPython. The primary mechanism for freeing objects is reference counting. When a variable goes out of scope or a request completes, ob_refcnt drops to zero, and the memory allocator immediately frees the object. The vast majority of transient request objects (strings, dicts, tuples, response buffers) are non-cyclical and clean up immediately.

2. Bounded Lifetimes via uWSGI: For the small percentage of objects trapped in true reference cycles, uWSGI’s reload-on-rss acts as a macro-level garbage collector. When a worker gradually drifts past its memory ceiling, it is recycled cleanly, releasing its entire memory space back to the OS kernel.

---

Summary of Key Takeaways

Observation / Bottleneck	Mechanism	Solution Applied
Shared memory drops post-fork	Cyclic GC sweeps mutate pointers in PyGC_Head, triggering OS Copy-on-Write.	Disable cyclic GC sweeps.
gc.disable() silently bypassed	Third-party dependencies (e.g., msgpack) invoke gc.enable().	Set GC collection threshold to zero via gc.set_threshold(0).
Worker reloads cause latency spikes	Py_Finalize() invokes a catastrophic final GC pass during teardown, wiping OS disk page caches.	Short-circuit interpreter teardown using atexit.register(os._exit, 0).
Safety against cyclic memory leaks	Disabling GC leaves reference cycles uncollected.	Leveraged CPython reference counting for 95%+ of objects, backed by uWSGI reload-on-rss for macro-level reclamation.

By methodically profiling page faults with perf and aligning Python’s runtime mechanics with OS-level memory semantics, a seemingly reckless idea—turning off garbage collection—became one of the most effective infrastructure optimizations in Instagram’s history.