Hacking CPython: Modifying the Internal Print Function to Inspect Object Metadata

Every Python developer uses the print() function daily, yet few explore how it operates beneath the interpreter layer. In CPython (the reference implementation of Python written in C), print() is a built-in function that manages argument unpacking, stream targeting, formatting separators, and calling object representation protocols.

By reading and modifying CPython source code directly, we can demystify how the interpreter executes Python code and observe core runtime mechanisms—such as object type resolution and reference counting—in action.

---

1. Locating Builtins in a Massive Codebase

The CPython repository contains hundreds of thousands of lines of C code. Searching naively for print returns over 5,400 results across header files, test cases, and modules. Finding the exact function entry point requires a structured approach.

The Docstring Reverse-Lookup Technique

When you invoke Python’s built-in help() on any native function, the interpreter retrieves a hardcoded documentation string attached to the function’s definition:

>>> help(print)
Help on built-in function print in module builtins:

print(*args, sep=' ', end='\n', file=None, flush=False)
    Prints the values to a stream, or to sys.stdout by default.

Because this docstring must reside alongside the C declaration (typically defined via Argument Clinic or direct builtin module tables), searching the codebase for an exact phrase like "Prints the values to a stream" narrows thousands of hits down to just two files: a C implementation file (Python/bltinmodule.c) and a header file.

In Python/bltinmodule.c, the actual entry point is defined as:

builtin_print_impl(PyObject *module, PyObject *args, PyObject *sep, 
                   PyObject *end, PyObject *file, int flush)

---

2. Anatomy of builtin_print_impl

The implementation of print() mirrors the behavior documented in the Python specification, translated into low-level C calls on Python runtime structures (PyObject*).

flowchart TD
    A[Invoke print] --> B{file == None?}
    B -- Yes --> C[Set target = sys.stdout]
    B -- No --> D[Set target = file]
    C --> E{sep == None?}
    D --> E
    E -- Yes --> F[sep = ' ']
    E -- No --> G[sep = user_sep]
    F --> H[Loop over args Tuple i = 0 to N-1]
    G --> H
    H --> I{i > 0?}
    I -- Yes --> J[Write sep to stream]
    I -- No --> K[Write args at i to stream]
    J --> K
    K --> L[More items?]
    L -- Yes --> H
    L -- No --> M{end == None?}
    M -- Yes --> N[Write newline '\n']
    M -- No --> O[Write user end]
    N --> P{flush == True?}
    O --> P
    P -- Yes --> Q[Flush stream]
    P -- No --> R[Return Py_None]
    Q --> R

Core Logic Walkthrough

1. Stream Resolution (file): Python objects are compared against Py_None (the singleton representing None). If file == Py_None, the function falls back to retrieving sys.stdout via the runtime state.

2. Separator Configuration (sep): If sep is not provided or set to None, CPython marks it to default to a single ASCII space character (" ").

3. Argument Iteration: The variable positional arguments *args are passed into C as a PyTupleObject. CPython gets the tuple size using PyTuple_GET_SIZE(args) and iterates:

   for (Py_ssize_t i = 0; i < PyTuple_GET_SIZE(args); i++) {
       if (i > 0) {
           // Write separator
           if (sep == NULL)
               PyFile_WriteString(" ", file);
           else
               PyFile_WriteObject(sep, file, Py_PRINT_RAW);
       }
       // Write the object itself
       PyObject *item = PyTuple_GET_ITEM(args, i);
       PyFile_WriteObject(item, file, Py_PRINT_RAW);
   }

4. Line Termination (end) & Flushing: If end is null, "\n" is written to the output file. If the boolean flag flush is truthy, the interpreter calls the target stream’s flush routine.

---

3. The PyObject Header

Every object pointer in CPython is represented as a PyObject*. Underneath, the header definition (PyObject_HEAD) defines the minimal metadata shared across all Python data types:

struct _object {
    _PyObject_HEAD_EXTRA
    Py_ssize_t ob_refcnt;
    PyTypeObject *ob_type;
};

• ob_refcnt: The reference count tracking how many pointers refer to this object. CPython uses reference counting as its primary garbage collection mechanism.
• ob_type: A pointer to the object’s type descriptor (PyTypeObject). Inside PyTypeObject, tp_name stores the human-readable type name (e.g., "dict", "str", "int").

---

4. Modifying print() to Output Runtime Metadata

To see this architecture in action, we can patch builtin_print_impl so that before printing an object, it outputs its type name and current reference count in the format <type:refcnt>.

The C Code Patch

Inside the loop where items are written, we intercept the object before calling PyFile_WriteObject:

for (Py_ssize_t i = 0; i < PyTuple_GET_SIZE(args); i++) {
    if (i > 0) {
        if (sep == NULL)
            PyFile_WriteString(" ", file);
        else
            PyFile_WriteObject(sep, file, Py_PRINT_RAW);
    }

    PyObject *obj = PyTuple_GET_ITEM(args, i);

    // 1. Construct metadata string: <typename:refcount> 
    PyObject *meta = PyUnicode_FromFormat("<%s:%zd> ", 
                                         Py_TYPE(obj)->tp_name, 
                                         Py_REFCNT(obj));
    if (meta != NULL) {
        // 2. Write the metadata to the target stream
        PyFile_WriteObject(meta, file, Py_PRINT_RAW);
        // Decrement refcount of temporary meta string
        Py_DECREF(meta);
    }

    // 3. Write the original object
    PyFile_WriteObject(obj, file, Py_PRINT_RAW);
}

Key CPython C-API Primitives Used:

• Py_TYPE(obj): Accesses obj->ob_type safely.
• Py_REFCNT(obj): Reads obj->ob_refcnt.
• PyUnicode_FromFormat(...): Allocates a new Python str using printf-style formatting specifiers (%s for C string, %zd for Py_ssize_t).
• PyFile_WriteObject(...): Writes a PyObject directly into the designated file stream.

---

5. Recompiling and Verifying the Behavior

After applying the patch, rebuilding CPython takes only a standard make invocation:

make -j$(nproc)
./python

Inspecting Variables

Let’s test our customized interpreter:

>>> d = {'a': 1}
>>> print(d)
<dict:3> {'a': 1}

The interpreter prepends <dict:3>, indicating that the dictionary type is dict and has a reference count of 3 at the moment of printing (held by the variable d, the local frame namespace, and temporary internals).

Observing Reference Counting in Real Time

When we append d to multiple lists, we can watch the reference count increment dynamically:

>>> L1 = []
>>> L2 = []
>>> L1.append(d)
>>> print(d)
<dict:4> {'a': 1}

>>> L2.append(d)
>>> print(d)
<dict:5> {'a': 1}

Each list appends a reference to the same dictionary object in memory, directly incrementing ob_refcnt. When elements are removed or go out of scope, the counter decrements. If ob_refcnt hits zero, the memory deallocation function (tp_dealloc) is triggered immediately.

The -1 Reference Count Phenomenon

If you inspect certain string literals or small integers:

>>> print("hello")
<str:-1> hello

You may observe a reference count of -1 (or extremely high values depending on Python version). In modern CPython (Python 3.12+), this is due to Immortal Objects (PEP 683). Certain globally shared objects—such as None, True, False, and interned static strings—are marked immortal to prevent cache invalidations across CPU cores during multi-threaded execution. Their reference counts are permanently fixed to a sentinel value and bypass normal increment/decrement cycles.

---

6. Key Takeaways

1. Navigating Unknown Codebases: When exploring large projects, identify user-facing boundaries (such as docstrings, specific error messages, or CLI flags) to quickly pinpoint internal entry points.
2. No Black Magic: High-level language constructs like print() ultimately boil down to procedural iterations over structs, format conversions, and I/O buffer writes.
3. Pattern Matching in C: You do not need complete mastery of an entire ecosystem to make meaningful modifications. Identifying local patterns (such as how CPython handles null checks and stream writing) allows you to implement functional features with confidence.
4. First-Principles Understanding: Modifying core utilities provides immediate, empirical validation of abstract concepts like memory management, type structures, and garbage collection mechanisms.