Distributed Systems (281)

Essays on distributed systems, databases, consensus, and fault tolerance.

49 blogs • 226 notes • 6 videos

2026

(76)

2025

(78)

Dec 27

NOTE ORMs are an anti-pattern once you start operating at scale

Distributed Systems #distributed
Dec 24

NOTE PostgreSQL lets you clone a 6GB database in 212 milliseconds

Distributed Systems #distributed
Dec 23

BLOG Clock Synchronization Is a Nightmare

Distributed Systems #distributed-systems #clock-synchronization #consistency
Dec 23

NOTE Here's something as simple as a distributed build system (using

Distributed Systems #distributed
Dec 23

NOTE Wrote an essay on why time is such a hard

Distributed Systems #distributed
Dec 17

NOTE Go 1.26 is not out yet, but the release notes

Distributed Systems #distributed
Dec 14

NOTE Wrote an essay on what Kafka does when we add

Distributed Systems #distributed
Dec 13

BLOG When You Increase Kafka Partitions

Distributed Systems #distributed-systems #concurrency #networking
Dec 12

NOTE Imagine you are streaming changes from Postgres to Elasticsearch in Go

Distributed Systems #distributed
Dec 10

NOTE Documentation is important, but most do it for the sake

Distributed Systems #distributed
Dec 06

NOTE Cloudflare had another outage, lasting 25 mins, on 5th Dec,

Distributed Systems #distributed
Dec 02

NOTE Py 3.11 is more performant than Py 3.12+ on AWS Lambda

Distributed Systems #distributed
Nov 25

NOTE Google just tested the largest Kubernetes cluster with 130,000 nodes

Distributed Systems #distributed
Nov 23

NOTE Rule number 1 of multitenancy: never assume all tenants are good actors

Distributed Systems #distributed
Nov 19

NOTE Range-based partitioning is like an amoeba

Distributed Systems #distributed
Nov 16

NOTE Wrote an essay on Blocking Queues, covering the intuition, implementation,

Distributed Systems #distributed
Nov 14

NOTE Google quantum computers are now solving optimization problems that even

Distributed Systems #distributed
Nov 12

BLOG Heartbeats in Distributed Systems

Distributed Systems #distributed-systems #clock-synchronization #networking
Nov 12

NOTE Wrote an essay on Heartbeats in Distributed Systems, capturing both

Distributed Systems #distributed
Nov 07

NOTE If you are building or using a distributed database where

Distributed Systems #distributed
Nov 06

NOTE SQLite 3.51 is out, and here are two things that

Distributed Systems #distributed
Nov 05

NOTE Cloudflare shared some data on Internet connections at scale, and

Distributed Systems #distributed
Nov 04

NOTE I am on a spree to explore PostgreSQL internals, and

Distributed Systems #distributed
Nov 03

NOTE LinkedIn leverages Apache Beam to power their Notification Platform

Distributed Systems #distributed
Oct 31

NOTE Active-active setups are fascinating

Distributed Systems #distributed
Oct 20

BLOG How Writes Work in Apache Cassandra

Distributed Systems #distributed-systems #consistency #databases
Oct 20

NOTE wrote an essay on how writes work in apache cassandra

Distributed Systems #distributed
Oct 17

BLOG Redis Replication Internals

Distributed Systems #distributed-systems #networking #replication
Oct 12

NOTE Here's an interesting fact about Redis - while replications on

Distributed Systems #distributed
Oct 04

NOTE how does a cdn handle content replication

Distributed Systems #distributed
Oct 02

BLOG How Does a CDN Handle Content Replication

Distributed Systems #distributed-systems #algorithms #career-growth
Sep 25

NOTE Fun Fact - When you commit a transaction in PostgreSQL,

Distributed Systems #distributed
Sep 24

NOTE grpc wouldn't be possible without http/2, but why

Distributed Systems #distributed
Sep 24

NOTE The strength of a good system is not its complexity,

Distributed Systems #distributed
Sep 20

NOTE Today I learned that we can do joins in Elasticsearch

Distributed Systems #distributed
Sep 18

NOTE There is never only one way to design something, and

Distributed Systems #distributed
Sep 17

NOTE I was going through the internals of MySQL, and I

Distributed Systems #ai
Sep 11

NOTE Yes, CDN improves latencies by caching things closer to the

Distributed Systems #distributed
Sep 07

NOTE why the private network addresses 192.168

Distributed Systems #distributed
Sep 06

BLOG Why Distributed Systems Need Consensus Algorithms Like Raft

Distributed Systems #distributed-systems #clock-synchronization #communication
Sep 05

BLOG Why Do Databases Deadlock and How Do They Resolve It

Distributed Systems #distributed-systems #concurrency #databases
Sep 05

NOTE When I joined Google, I worked on building a Remote

Distributed Systems #distributed
Sep 04

NOTE For every complex problem, there is often a simpler solution

Distributed Systems #distributed
Sep 03

BLOG Why Eventual Consistency is Preferred in Distributed Systems

Distributed Systems #distributed-systems #consistency #networking
Sep 03

NOTE db nugget (4): Picking the wrong database bites you hard in production

Distributed Systems #career-growth
Aug 27

NOTE db nugget (3): in-mem databases do not always lose data upon restart

Distributed Systems #distributed
Aug 26

NOTE db nugget (2): if you want your database to be

Distributed Systems #distributed
Aug 21

NOTE research is addictive; just thinking about one problem in one

Distributed Systems #distributed
Aug 16

NOTE Randomness improves algorithms

Distributed Systems #distributed
Aug 12

NOTE Your MySQL index isn’t slow; it's fragmented :) Databases like

Distributed Systems #distributed
Aug 07

NOTE Should two microservices share a database

Distributed Systems #distributed
Aug 04

NOTE When you need high availability, all you really need to

Distributed Systems #distributed
Jul 31

NOTE Timing Attacks are really interesting, and they can be leveraged

Distributed Systems #distributed
Jul 11

NOTE How far can you go vibecoding

Distributed Systems #distributed
Jul 08

NOTE Sometime back, I was going through PostgreSQL internals and found

Distributed Systems #distributed
Jul 03

NOTE High fork times can lead to master unavailability in Redis/Valkey,

Distributed Systems #distributed
Jul 01

NOTE Sure, horizontal scaling is great for handling large loads and

Distributed Systems #ai
Jun 18

NOTE Fun fact - Yes, most databases use B+ trees to

Distributed Systems #distributed
May 29

NOTE Amazon just made Aurora DSQL generally available

Distributed Systems #distributed
May 29

NOTE Most people get this wrong about the master-replica setup

Distributed Systems #distributed
May 23

NOTE Is CDN a reverse proxy, just like a load balancer

Distributed Systems #ai
May 13

NOTE Optimization is a zero-sum game

Distributed Systems #distributed
May 01

NOTE was reading a paper and came across this really good insight

Distributed Systems #distributed
Apr 25

NOTE While reading a paper, I came across the term multi-socket

Distributed Systems #distributed
Apr 24

NOTE If you are into distributed systems, please spend some time

Distributed Systems #distributed
Apr 11

NOTE Common misconception - CDNs can *only* cache images and videos

Distributed Systems #distributed
Mar 23

NOTE One of the best courses I took during my college

Distributed Systems #distributed
Mar 21

NOTE Some time back, in my live cohort, we were discussing

Distributed Systems #distributed
Mar 11

NOTE I learned a neat trick in Go

Distributed Systems #distributed
Mar 05

NOTE The "SQL doesn’t scale" argument is a lazy and an absurd generalization

Distributed Systems #distributed
Mar 04

NOTE Netflix reduced connection churn in its API gateway by 10x

Distributed Systems #distributed
Feb 19

NOTE It's surprising how many engineers I talk to today have

Distributed Systems #distributed
Feb 13

NOTE Time-series databases use a really simple algorithm to compress the

Distributed Systems #distributed
Feb 06

NOTE Go 1.24 is expected to be released this month, but

Distributed Systems #go #dice-db
Jan 31

NOTE Should you learn sharding and partitioning if we already have

Distributed Systems #system-design
Jan 25

NOTE YAGNI is my favorite principle when I build and implement any system

Distributed Systems #system-design
Jan 16

NOTE Sometime back, I got curious about image formats and started

Distributed Systems #distributed
Jan 01

NOTE If you plan to learn in public in 2025, spend more time learning than sharing.

Distributed Systems #software-engineering #asli-engineering #career-growth

2024

(51)

Dec 25

NOTE Bayes' Theorem, the thing we learned in our statistics class

Distributed Systems #system-design #ads-engineering
Dec 20

NOTE Enjoy all the Spotify Wrapped you like, but spare some

Distributed Systems #distributed
Dec 18

NOTE Implement what you read, and read how to make your implementation better

Distributed Systems #distributed
Dec 17

NOTE 60% of the tables I have recently designed, do not

Distributed Systems #databases #system-design
Dec 16

NOTE Good engineers are usually excellent project managers and then become solid engineering leaders.

Distributed Systems #distributed-systems #career-growth #asli-engineering
Dec 14

NOTE Most NoSQL databases should have been a jsonb column in PostgreSQL

Distributed Systems #distributed
Dec 06

NOTE PostgreSQL is a fascinating database and if you want to

Distributed Systems #system-design #databases
Nov 21

NOTE Six core computer science projects that I absolutely loved doing

Distributed Systems #distributed
Nov 17

NOTE One of the good coding practices that I picked up

Distributed Systems #distributed
Nov 16

NOTE I was crunching some numbers for JSON vs protobuf and

Distributed Systems #distributed
Nov 14

NOTE I am on a spree to explore PostgreSQL internals and

Distributed Systems #database-internals
Nov 12

NOTE For the last few weeks, I have been spending time

Distributed Systems #database-internals
Oct 18

NOTE Today, I was reading through PostgreSQL's documentation and I stumbled

Distributed Systems #databases #postgre-sql
Oct 17

NOTE PostgreSQL implements an interesting optimization to make some scans index-only

Distributed Systems #database-internals #postgre-sql
Oct 13

NOTE The root cause of performance issues is almost always a

Distributed Systems #system-design
Oct 02

NOTE Low-level design patterns are great, but beyond a point, they

Distributed Systems #distributed
Sep 13

NOTE While I was trying to figure out which paper to

Distributed Systems #system-design
Sep 11

NOTE I extensively used Elasticsearch for 2 years straight while I

Distributed Systems #system-design #elastic-search-internals
Sep 03

BLOG Paper Notes - SQL Has Problems. We Can Fix Them Pipe Syntax In SQL

Distributed Systems #distributed-systems #code-review #leadership
Sep 03

NOTE You must the isolation level of your database, as picking

Distributed Systems #databases #system-design
Aug 29

NOTE Most early engineers (in some cases even seniors) get confused

Distributed Systems #system-design
Aug 27

NOTE I always wondered how we could programmatically ensure variables get

Distributed Systems #distributed
Aug 23

NOTE Database guarantees durability using WAL, and it is possible due

Distributed Systems #database-engineering #system-design
Aug 21

BLOG Paper Notes - WTF - The Who to Follow Service at Twitter

Distributed Systems #distributed-systems #algorithms #memory-management
Aug 21

NOTE this is a really interesting question asked in my video

Distributed Systems #real-world-system-design
Aug 15

NOTE Build micro-services, not nano-services

Distributed Systems #distributed
Aug 08

NOTE PostgreSQL crumbles even when a moderate number of clients connect to it

Distributed Systems #database-internals #system-design
Jul 24

NOTE Should two microservices services share a database

Distributed Systems #system-design
Jul 21

NOTE A solution that exists for all, is a solution that

Distributed Systems #distributed
Jul 19

NOTE I saw many engineers blaming the outage on Microsoft 🤦‍♂️

Distributed Systems #distributed
Jul 17

NOTE I spent some time this afternoon going through Buffer Cache

Distributed Systems #postgre-sqlinternals #system-design
Jul 10

NOTE When you start your PostgreSQL database, a bunch of background

Distributed Systems #database-internals #postgre-sqlinternals
Jun 26

NOTE Redis is written in C, but its test cases are

Distributed Systems #redis-internals
Jun 25

NOTE I was always curious about where and how PostgreSQL stores

Distributed Systems #database-internals #postgre-sqlinternals
Jun 18

NOTE One of the most amusing features of PostgreSQL is Tablespaces,

Distributed Systems #system-design #database-internals #postgre-sqlinternals
Jun 12

NOTE Everyone: please keep your variable names self-explanatory

Distributed Systems #database-internals
Jun 05

NOTE Today, I was going through PostgreSQL internals and found something

Distributed Systems #database-internals #system-design
May 22

NOTE Zero-copy writes make Kafka fast, but how exactly

Distributed Systems #system-design
May 06

BLOG Turn System Design Interviews into Discussions

Distributed Systems #distributed-systems #interviews #networking
Apr 16

NOTE PostgreSQL is not just a relational db, but an object-relational db ⚡

Distributed Systems #postgre-sql
Apr 02

NOTE Today I learned, there are over one trillion SQLite databases

Distributed Systems #career-growth
Mar 07

NOTE Hot shards are a pain, but why do they occur

Distributed Systems #system-design
Mar 06

NOTE Whenever you are shipping any service, make it highly observable ⚡

Distributed Systems #system-design
Feb 15

NOTE Over-engineering has killed more tech startups than a bad business idea ⚡

Distributed Systems #career-growth
Jan 20

NOTE Writing quick prototypes from scratch is easy, do it, but ⚡

Distributed Systems #distributed
Jan 18

NOTE High-observability is a prerequisite for optimization

Distributed Systems #distributed
Jan 17

NOTE In the world where you have managed databases, should you

Distributed Systems #distributed
Jan 14

NOTE Caching is not the only way to handle the load ⚡

Distributed Systems #distributed
Jan 10

NOTE System Design in the real world always revolves around cost 💰

Distributed Systems #system-design
Jan 06

NOTE Projects are the best way to learn a programming language,

Distributed Systems #go #system-design
Jan 02

NOTE Nginx is one of the most powerful tools out here,

Distributed Systems #distributed

2023

(27)

Dec 26

NOTE Writing multi-threaded programs is easy, writing correct multi-threaded programs is

Distributed Systems #distributed
Dec 22

NOTE We keep making software complex enough to fit and use

Distributed Systems #ai
Nov 25

NOTE Here's an interesting question asked on Partitioning and Sharding ⚡

Distributed Systems #system-design
Nov 02

NOTE Why designing highly available services important

Distributed Systems #system-design
Oct 18

NOTE Let's break one of the biggest misconceptions about Consistent Hashing ⚡

Distributed Systems #consistent-hashing
Oct 14

NOTE During my live cohort, we were discussing BLOB vs TEXT,

Distributed Systems #system-design #my-sql
Sep 21

NOTE Changing something in staging can take down production ⚡

Distributed Systems #microservices #system-design
Sep 18

NOTE LRU is a popular but memory-intensive algorithm, hence Redis tweaked it ⚡

Distributed Systems #redis-internals
Sep 06

NOTE I used to believe that `TEXT` column type could store

Distributed Systems #database-internals
Sep 04

NOTE How to decide the optimal number of partitions for a Kafka topic

Distributed Systems #kafka
Aug 14

NOTE System Design is not about drawing boxes, you need to

Distributed Systems #system-design
Jul 30

NOTE ACID and CAP theorem, both have C standing for "Consistency",

Distributed Systems #system-design
Jul 25

NOTE Hardware failure in large infra is not a question of

Distributed Systems #distributed
Jun 10

NOTE 3 videos that cover everything about sharding, partitioning, and their

Distributed Systems #system-design
Jun 06

NOTE How do the updates happening on the master reach the replica

Distributed Systems #database-engineering #system-design
May 04

NOTE Amazon's Prime Video cut 90% of its cloud costs by

Distributed Systems #system-design
May 01

NOTE CockroachDB is fascinating

Distributed Systems #system-design #cockroach-db
Apr 22

NOTE Writes in CockroachDB are fascinating as it directly guarantees a

Distributed Systems #system-design #database-internals
Apr 18

NOTE CockroachDB Internals ] CockroachDB is distributed SQL database that replicates

Distributed Systems #system-design #database-internals
Apr 17

NOTE Consistent Hashing is not some magical solution that fits everywhere

Distributed Systems #consistent-hashing #system-design
Apr 15

NOTE CockroachDB Internals ] CockroachDB leverages multi-version concurrency control to process

Distributed Systems #cockroach-db #database-internals
Apr 12

NOTE CockroachDB Internals ] Data Structures and Algorithms are crucial in

Distributed Systems #system-design #database-internals
Apr 11

NOTE CockroachDB Internals ] CockroachDB is PostgreSQL compliant, how and why

Distributed Systems #cockroach-db #database-internals
Apr 06

NOTE CockroachDB Internals ] Let's talk about OLTP and OLAP databases

Distributed Systems #system-design
Mar 13

NOTE I am fascinated by random numbers ⚡

Distributed Systems #distributed
Mar 12

NOTE How databases are upgraded in production

Distributed Systems #system-design #databases
Feb 21

NOTE 3 in-depth videos that cover everything about Sharding along with

Distributed Systems #system-design

2022

(18)

Dec 25

NOTE Here's how you design a good distributed system 👇‍

Distributed Systems #system-design #distributed-system
Oct 15

NOTE CDN is much more than a cache for images and videos

Distributed Systems #system-design #cdn
Oct 06

NOTE was implementing graceful shutdown for some TCP server, and it

Distributed Systems #distributed
Sep 20

NOTE Just adding threads won't make your code run fast

Distributed Systems #concurrency
Sep 19

NOTE Random numbers that we generate in our programming language are

Distributed Systems #distributed
Sep 14

NOTE How databases are upgraded in production

Distributed Systems #system-design #database
Jun 19

NOTE Spent the last 4 hours understanding the GIF format, deep

Distributed Systems #image-internals
Jun 04

NOTE Just spend hour and half understanding Shannon Entropy

Distributed Systems #distributed
Mar 13

NOTE There is no better resource to learn anything in-depth than

Distributed Systems #distributed
Feb 07

BLOG Local vs Global Indexes in Partitioned Databases

Distributed Systems #distributed-systems #databases #hashing
Feb 07

NOTE Essay : Indexing on a Partitioned Data

Distributed Systems #system-design #distributed-systems
Jan 31

BLOG Partitioning Data - Range, Hash, and When to Use Them

Distributed Systems #distributed-systems #databases #hashing
Jan 31

NOTE Essay : Data Partitioning Strategies

Distributed Systems #distributed-systems #system-design
Jan 25

NOTE Essay : An Introduction to Data Partitioning

Distributed Systems #distributed-systems #system-design
Jan 24

BLOG Partitioning Strategies for Distributed Databases

Distributed Systems #distributed-systems #databases #hashing
Jan 17

NOTE Essay : Leaderless Replication to have Strong Consistency in Distributed Database

Distributed Systems #distributed-systems #system-design
Jan 16

BLOG Leaderless Replication

Distributed Systems #distributed-systems #consistency #databases
Jan 03

BLOG Multi-Master Conflicts - How to Handle Them

Distributed Systems #distributed-systems #databases #replication

2021

(20)

Nov 28

BLOG Multi-Master Replication - Why Conflicts Happen

Distributed Systems #distributed-systems #databases #replication
Nov 03

BLOG Multi-Master Replication - Scaling Writes Across Geographies

Distributed Systems #distributed-systems #databases #replication
Oct 03

BLOG Monotonic Reads - How Asynchronous Replication Creates Wormholes

Distributed Systems #distributed-systems #replication
Oct 03

NOTE Essay : Preventing Wormholes in a Distributed Datastore

Distributed Systems #system-design #distributed-systems
Sep 22

BLOG Read-Your-Writes Consistency

Distributed Systems #distributed-systems #consistency #replication
Sep 22

NOTE How would you feel when you posted on a Social

Distributed Systems #system-design #distributed-systems
Sep 07

BLOG What Happens When a Master or Replica Fails

Distributed Systems #distributed-systems #databases #replication
Aug 23

NOTE Recently I wrote a short snippet about setting up a

Distributed Systems #system-design #distributed-systems
Aug 21

NOTE Ever wondered how the Replication happens, how a new Replica

Distributed Systems #system-design #distributed-systems
Aug 15

BLOG Replication Formats - Statement vs Row-based

Distributed Systems #distributed-systems #databases #networking
Aug 15

NOTE Ever wonder how replication happens between Master and Replica

Distributed Systems #system-design #distributed-systems
Aug 10

BLOG Replication Strategies - Synchronous, Asynchronous, and Semi-Synchronous

Distributed Systems #distributed-systems #consistency #networking
Aug 07

BLOG Master-Replica Replication

Distributed Systems #distributed-systems #databases #replication
Jul 19

BLOG Decoding Durability - The D in ACID

Distributed Systems #distributed-systems #consistency #databases
Jul 07

BLOG Decoding Isolation - The I in ACID

Distributed Systems #distributed-systems #concurrency #consistency
Jul 02

BLOG Decoding Atomicity - The A in ACID

Distributed Systems #distributed-systems #consistency #databases
Jul 02

BLOG Decoding Consistency - The C in ACID

Distributed Systems #distributed-systems #consistency #databases
Jun 22

BLOG Architectural Patterns for Distributed Systems

Distributed Systems #distributed-systems #databases #system-design
Jun 17

BLOG Fallacies of Distributed Computing Every Engineer Should Know

Distributed Systems #distributed-systems #networking
Apr 19

BLOG Modeling Udemy's Categories in a Relational Database

Distributed Systems #distributed-systems #databases #python-internals

2020

(8)

Nov 29

BLOG The 2Q Algorithm - Addressing LRU's Sub-Optimality

Distributed Systems #distributed-systems #caching #databases
Aug 23

BLOG Implementing LFU in O(1)

Distributed Systems #distributed-systems #caching #memory-management
Jul 19

BLOG Bitcask - A Log-Structured fast KV store

Distributed Systems #distributed-systems #data-structures #databases
May 31

BLOG The RUM Conjecture - Storage System Trade-offs

Distributed Systems #distributed-systems #consistency #cryptography
May 24

BLOG Consistent Hashing - What It Is and How to Implement It

Distributed Systems #distributed-systems #caching #data-structures
May 03

BLOG Copy-On-Write - When to Use It, When to Avoid It

Distributed Systems #distributed-systems #databases #memory-management
Apr 26

BLOG How MySQL Avoids Performance Hits from Table Scans

Distributed Systems #distributed-systems #caching #databases
Apr 05

BLOG Sliding Window Rate Limiting - Design and Implementation

Distributed Systems #distributed-systems

2017

(3)

Mar 24

BLOG Analyzing MongoDB Pagination Performance Differences

Distributed Systems #distributed-systems #databases #evaluation
Mar 23

BLOG MongoDB Pagination - Skip is Slow

Distributed Systems #distributed-systems #databases
Mar 22

BLOG MongoDB Pagination - Two Approaches You Need to Know

Distributed Systems #distributed-systems #databases

Writings and Learnings

Courses

Applied AI Masterclass

System Design Masterclass

System Design for Beginners

Legal and Contact

Terms and Conditions

Everything Else

The Smarter Chimp

Arpit's Newsletter read by 145,000 engineers

Weekly essays on real-world system design, distributed systems, or a deep dive into some super-clever algorithm.

Subscribe on LinkedIn Subscribe on Substack

The courses listed on this website are offered by

Relog Deeptech Pvt. Ltd.
203, Sagar Apartment, Camp Road, Mangilal Plot, Amravati, Maharashtra, 444602
GSTIN: 27AALCR5165R1ZF

LinkedIn (280k)

© Arpit Bhayani, 2025