Arpit Bhayani
Projects
Blogs
Notes
Videos
Papershelf
Bookshelf
Courses
Talks
Home
Topics
Distributed Systems
#distributed-systems
(130)
49 blogs
•
58 notes
•
23 videos
All
130
Distributed Systems
105
Career Growth
12
Systems Internals
5
Databases
4
Algorithms & Data Structures
3
AI Systems
1
2026
(57)
Sep 04
VIDEO
Demystifying Kafka's Exactly-Once Semantics: Idempotence, Transactions, and Fencing
Distributed Systems
#kafka
#distributed-systems
#message-queue
Sep 03
NOTE
Cloudflare Implements Cache Transcoding to Optimize Cache Storage with Zstandard
Distributed Systems
#distributed-systems
#cloudflare
#cache-transcoding
Sep 01
VIDEO
Consistent Hashing: Demystifying the Algorithm and Its Simple Implementation
Distributed Systems
#consistent-hashing
#distributed-systems
#system-design
Aug 26
NOTE
How S3 Enables Stateless Write-Ahead Logs Despite Latency Trade-offs
Distributed Systems
#distributed-systems
#caching
#commit-latency
Aug 14
NOTE
Fast Stores Can Mask Per-Item State Check Performance Issues
Career Growth
#distributed-systems
#access-patterns
#batching
Aug 11
NOTE
AWS Route 53 Shuffle Sharding Uses Randomness To Reduce Failure Impact
Distributed Systems
#distributed-systems
#aws-route-53
#blast-radius
Jul 28
NOTE
Systematically Evaluate Design Alternatives and Prune Suboptimal Trade-offs
Distributed Systems
#distributed-systems
#delivery-timelines
#design-alternatives
Jul 27
NOTE
Huge Pages Optimize Postgres Shared Buffers by Improving TLB Efficiency
Systems Internals
#distributed-systems
#caching
#databases
Jul 24
NOTE
Bottlenecks in infrastructure never disappear they just move
Distributed Systems
#distributed-systems
#bottleneck
#bottleneck-relocation
Jul 20
NOTE
Starting From Scratch Unlocks Radical Innovation By Challenging Assumptions
Distributed Systems
#distributed-systems
#apache-spark
#building-from-scratch
Jul 11
NOTE
Bloom Filter Emerged as Key Evidence in OpenAI Copyright Dispute
Algorithms & Data Structures
#distributed-systems
#bloom-filter
#bloom-filters
Jul 09
NOTE
Cloudflare's Meerkat Service Uses QuePaxa to Avoid Leader Problems
Distributed Systems
#distributed-systems
#cloudflare
#consensus
Jul 06
NOTE
Improve MySQL Master Failover Monitoring Using Replica Consensus
Distributed Systems
#distributed-systems
#consensus
#databases
Jul 01
NOTE
Predictability Is The Most Valuable Quality For Robust System Design
Distributed Systems
#distributed-systems
#admission-control
#backpressure
Jul 01
NOTE
Spatial Join Operations Power Location-Based Services Like Uber and DoorDash
Distributed Systems
#distributed-systems
#delivery-zones
#geopandas
Jun 30
NOTE
How Erasure Coding Achieves S3 Durability with Reduced Storage Costs
AI Systems
#distributed-systems
#compute-tradeoff
#data-durability
Jun 27
NOTE
RocksDB's Sequence Numbers Simplify Database Snapshots and Concurrency
Distributed Systems
#distributed-systems
#compaction
#concurrency
Jun 23
NOTE
PostgreSQL Uses Histograms and MCVs to Estimate Rows
Distributed Systems
#distributed-systems
#databases
#histograms
Jun 19
NOTE
Separating Compute From Storage Enables Serverless Database Pay-Per-Use
Distributed Systems
#distributed-systems
#compute-storage-separation
#databases
Jun 16
NOTE
Platforms Offer Compounding Returns When Built and Adopted Effectively
Distributed Systems
#distributed-systems
#abstraction
#adoption
Jun 11
NOTE
Handle Consecutive 401 Errors with Circuit Breakers to Prevent Outages
Distributed Systems
#distributed-systems
#401-unauthorized
#blind-retries
Jun 04
NOTE
Successful Big Migrations Require a Plank-by-Plank Replacement Strategy
Distributed Systems
#distributed-systems
#databases
#full-cutover
May 31
VIDEO
How and Why Redis Overrides Malloc: Deep Dive into jemalloc and tcmalloc
Systems Internals
#redis
#memory-management
#jemalloc
May 25
NOTE
Structured Pessimism Improves System Reliability Through Proactive Questioning
Distributed Systems
#distributed-systems
#accountability
#edge-cases
May 21
NOTE
Key Considerations for Building Predictably Horizontally Scalable Systems
Distributed Systems
#distributed-systems
#asynchronous-queues
#caching
May 20
BLOG
Temporal Primer - Building Long-Running Systems
Distributed Systems
#distributed-systems
#ai-agents
#career-growth
May 19
NOTE
Approximating Least Recently Used Cache Eviction Using Two Random Choices
Distributed Systems
#distributed-systems
#2-random
#approximation
May 15
NOTE
Designing Systems for Failure Ensures Graceful Error Handling and Quick Recovery
Career Growth
#distributed-systems
#career-growth
#design-for-failure
May 12
NOTE
Distributed System Failures Often Stem From Unintended Omissions In Design
Distributed Systems
#distributed-systems
#back-pressure
#concurrency
May 02
NOTE
Keeping L1 Small in LSM Trees Prevents Excessive Write Amplification
Algorithms & Data Structures
#distributed-systems
#compaction
#data-structures
May 02
VIDEO
Implementing DEL, EXPIRE, and Auto-Cleanup in Redis Internals
Systems Internals
#redis
#database-internals
#distributed-systems
May 01
NOTE
Design Systems For Reversibility To Boost Confidence And Speed
Distributed Systems
#distributed-systems
#architectural-decisions
#databases
Apr 29
NOTE
Pushdown optimization in pg_clickhouse improves data querying efficiency
Distributed Systems
#distributed-systems
#clickhouse
#data-filtering
Apr 28
BLOG
Your Monolith Is Already A Distributed System
Distributed Systems
#distributed-systems
#databases
#information-retrieval
Apr 23
NOTE
Relational databases often suffice where graph databases are overengineering
Distributed Systems
#distributed-systems
#algorithms
#databases
Apr 22
NOTE
Crashing immediately on startup prevents late-run fixable issues
Career Growth
#distributed-systems
#career-growth
#configuration-validation
Apr 22
NOTE
Build Intuition for Distributed Systems by Asking Why and How
Distributed Systems
#distributed-systems
#debugging
#first-principles
Apr 14
NOTE
Update Heavy PostgreSQL Workloads Require Autovacuum Tuning
Distributed Systems
#distributed-systems
#autovacuum
#concurrency
Apr 13
NOTE
Correctness and Consistency are Different Things
Distributed Systems
#distributed-systems
#consistency
#correctness
Apr 11
NOTE
It Depends Must Be Followed By Explanation
Career Growth
#distributed-systems
#consistency
#databases
Apr 11
VIDEO
Redis Wire Protocol: A Deep Dive into RESP and its Data Types
Databases
#redis
#resp
#wire-protocol
Apr 11
VIDEO
The Fundamentals of Wire Protocols and Redis Serialization Protocol (RESP)
Systems Internals
#redis
#wire-protocol
#resp
Apr 08
NOTE
Partially Synchronous Systems Achieve Balance in Distributed Systems
Distributed Systems
#distributed-systems
#asynchronous-systems
#consistency
Apr 07
NOTE
Happened-before relationship brings order to distributed systems
Distributed Systems
#distributed-systems
#causality
#clock-synchronization
Apr 06
NOTE
Debugging Distributed Systems Requires Global and Local Views
Distributed Systems
#distributed-systems
#career-growth
#clock-synchronization
Apr 05
NOTE
Simpler Solutions Beat Complexity in System Architecture
Distributed Systems
#distributed-systems
#caching
#complexity
Apr 04
NOTE
Consistency, Availability, Freshness Trade-offs in Real-time Systems
Distributed Systems
#distributed-systems
#availability
#cap-theorem
Mar 17
VIDEO
How Stripe Achieves Zero-Downtime, Consistent Data Migrations at Scale
Distributed Systems
#data-migration
#zero-downtime
#schema-evolution
Mar 17
VIDEO
Stripe's Zero-Downtime, Consistent Data Migration Strategy at Scale
Distributed Systems
#data-migration
#schema-evolution
#zero-downtime
Mar 16
VIDEO
Stripe's Canonical Log Lines: A Smarter Approach to Structured Logging
Distributed Systems
#logging
#structured-logging
#canonical-log-lines
Mar 16
VIDEO
Stripe's Canonical Log Lines: A Smarter Approach to Structured Logging
Distributed Systems
#logging
#structured-logging
#canonical-log-lines
Mar 13
VIDEO
Stripe's DocDB: Scaling MongoDB to 5 Million Queries Per Second
Distributed Systems
#mongodb
#docdb
#stripe
Mar 08
BLOG
Databases Were Not Designed For This
Distributed Systems
#distributed-systems
#ai-agents
#caching
Feb 24
BLOG
JOIN Algorithms
Distributed Systems
#distributed-systems
#data-structures
#databases
Jan 27
BLOG
Multi-Paxos - Consensus in Distributed Databases
Distributed Systems
#distributed-systems
#clock-synchronization
#consensus
Jan 23
VIDEO
How OpenAI Scales PostgreSQL to Power 800 Million ChatGPT Users
Databases
#postgresql
#database-scaling
#distributed-systems
Jan 16
BLOG
MySQL Replication Internals
Distributed Systems
#distributed-systems
#concurrency
#databases
2025
(18)
Dec 23
BLOG
Clock Synchronization Is a Nightmare
Distributed Systems
#distributed-systems
#clock-synchronization
#consistency
Dec 13
BLOG
When You Increase Kafka Partitions
Distributed Systems
#distributed-systems
#concurrency
#networking
Nov 12
BLOG
Heartbeats in Distributed Systems
Distributed Systems
#distributed-systems
#clock-synchronization
#networking
Oct 20
BLOG
How Writes Work in Apache Cassandra
Distributed Systems
#distributed-systems
#consistency
#databases
Oct 17
BLOG
Redis Replication Internals
Distributed Systems
#distributed-systems
#networking
#replication
Oct 02
BLOG
How Does a CDN Handle Content Replication
Distributed Systems
#distributed-systems
#algorithms
#career-growth
Sep 06
BLOG
Why Distributed Systems Need Consensus Algorithms Like Raft
Distributed Systems
#distributed-systems
#clock-synchronization
#communication
Sep 05
BLOG
Why Do Databases Deadlock and How Do They Resolve It
Distributed Systems
#distributed-systems
#concurrency
#databases
Sep 03
BLOG
Why Eventual Consistency is Preferred in Distributed Systems
Distributed Systems
#distributed-systems
#consistency
#networking
Aug 02
VIDEO
How PayPal's Braintree Solved the Thundering Herd Problem and Simplified Architecture
Distributed Systems
#thundering-herd
#distributed-systems
#microservices
Jul 25
VIDEO
Inverted Index: The Core Data Structure Behind Search Engines
Algorithms & Data Structures
#inverted-index
#search-engines
#data-structures
Jul 11
VIDEO
Inside Stripe's Multi-Tier Rate Limiter Architecture
Distributed Systems
#rate-limiting
#distributed-systems
#system-design
May 23
VIDEO
How Amazon Aurora Achieves Durability with Quorum Writes and Segmented Storage
Databases
#amazon-aurora
#durability
#quorum
Mar 16
VIDEO
DiceDB 1.0: A Fast, Reactive, In-Memory Database for Modern Applications
Databases
#dicedb
#in-memory-database
#reactive-systems
Feb 09
VIDEO
How Atlassian Reduced Latency by 70% Using the Sidecar Pattern
Distributed Systems
#sidecar-pattern
#distributed-systems
#microservices
Feb 02
VIDEO
How Bitbucket Achieved Read-After-Write Consistency and Reduced Master Database Load by 50%
Distributed Systems
#read-after-write-consistency
#database-scaling
#postgresql
Jan 19
VIDEO
Trello's Migration from RabbitMQ to Kafka for Real-time WebSocket Updates
Distributed Systems
#trello
#rabbitmq
#kafka
Jan 12
VIDEO
Atlassian's Cache Invalidation Evolution: From SNS+SQS Push to S3 Pull Architecture
Distributed Systems
#cache-invalidation
#distributed-systems
#aws-sns
2024
(9)
Dec 16
NOTE
Good engineers are usually excellent project managers and then become solid engineering leaders.
Distributed Systems
#distributed-systems
#career-growth
#asli-engineering
Sep 15
NOTE
Every engineer should work in SRE for at least ~2 years.
Career Growth
#distributed-systems
#asli-engineering
#sre
Sep 04
NOTE
Not everything in the real world needs a syllabus.
Career Growth
#distributed-systems
#asli-engineering
#career-growth
Sep 03
BLOG
Paper Notes - SQL Has Problems. We Can Fix Them Pipe Syntax In SQL
Distributed Systems
#distributed-systems
#code-review
#leadership
Aug 21
BLOG
Paper Notes - WTF - The Who to Follow Service at Twitter
Distributed Systems
#distributed-systems
#algorithms
#memory-management
Aug 15
VIDEO
A Bug in GitHub's Redis-Based Rate Limiter: The Wobbling X-RateLimit-Reset Header
Distributed Systems
#rate-limiting
#redis
#distributed-systems
Jul 17
VIDEO
Latency vs. Response Time: Understanding Key Performance Metrics
Distributed Systems
#latency
#response-time
#performance-metrics
May 06
BLOG
Turn System Design Interviews into Discussions
Distributed Systems
#distributed-systems
#interviews
#networking
Apr 17
NOTE
LeetCode has induced instant gratification and made building boring ⚡
Career Growth
#distributed-systems
#asli-engineering
2023
(2)
Dec 20
NOTE
The coolest thing about Google's Spanner Database is its clock
Career Growth
#system-design
#distributed-systems
Nov 07
NOTE
Senior engineers always juggle the following 3 hats ⚡
Career Growth
#distributed-systems
#asli-engineering
#seniorsoftwareengineer
2022
(12)
Dec 11
NOTE
Here is a 4-step process I follow to learn new technologies, tools, and frameworks ⚡
Career Growth
#distributed-systems
#engineering
#system-design
Nov 18
NOTE
Why did the college want us to study Compiler Construction?
Systems Internals
#distributed-systems
#asli-engineering
#dice-db
Feb 07
BLOG
Local vs Global Indexes in Partitioned Databases
Distributed Systems
#distributed-systems
#databases
#hashing
Feb 07
NOTE
Essay : Indexing on a Partitioned Data
Distributed Systems
#system-design
#distributed-systems
Jan 31
BLOG
Partitioning Data - Range, Hash, and When to Use Them
Distributed Systems
#distributed-systems
#databases
#hashing
Jan 31
NOTE
Essay : Data Partitioning Strategies
Distributed Systems
#distributed-systems
#system-design
Jan 25
NOTE
Essay : An Introduction to Data Partitioning
Distributed Systems
#distributed-systems
#system-design
Jan 24
BLOG
Partitioning Strategies for Distributed Databases
Distributed Systems
#distributed-systems
#databases
#hashing
Jan 17
NOTE
Essay : Leaderless Replication to have Strong Consistency in Distributed Database
Distributed Systems
#distributed-systems
#system-design
Jan 16
BLOG
Leaderless Replication
Distributed Systems
#distributed-systems
#consistency
#databases
Jan 09
NOTE
Essay : Conflict Resolution in Multi-Master setup
Career Growth
#distributed-systems
#system-design
Jan 03
BLOG
Multi-Master Conflicts - How to Handle Them
Distributed Systems
#distributed-systems
#databases
#replication
2021
(21)
Nov 28
BLOG
Multi-Master Replication - Why Conflicts Happen
Distributed Systems
#distributed-systems
#databases
#replication
Nov 03
BLOG
Multi-Master Replication - Scaling Writes Across Geographies
Distributed Systems
#distributed-systems
#databases
#replication
Oct 03
BLOG
Monotonic Reads - How Asynchronous Replication Creates Wormholes
Distributed Systems
#distributed-systems
#replication
Oct 03
NOTE
Essay : Preventing Wormholes in a Distributed Datastore
Distributed Systems
#system-design
#distributed-systems
Sep 22
BLOG
Read-Your-Writes Consistency
Distributed Systems
#distributed-systems
#consistency
#replication
Sep 22
NOTE
How would you feel when you posted on a Social
Distributed Systems
#system-design
#distributed-systems
Sep 07
BLOG
What Happens When a Master or Replica Fails
Distributed Systems
#distributed-systems
#databases
#replication
Sep 07
NOTE
Essay : Everything you need to know about handling node
Career Growth
#system-design
#distributed-systems
Aug 23
NOTE
Recently I wrote a short snippet about setting up a
Distributed Systems
#system-design
#distributed-systems
Aug 21
NOTE
Ever wondered how the Replication happens, how a new Replica
Distributed Systems
#system-design
#distributed-systems
Aug 15
BLOG
Replication Formats - Statement vs Row-based
Distributed Systems
#distributed-systems
#databases
#networking
Aug 15
NOTE
Ever wonder how replication happens between Master and Replica
Distributed Systems
#system-design
#distributed-systems
Aug 10
BLOG
Replication Strategies - Synchronous, Asynchronous, and Semi-Synchronous
Distributed Systems
#distributed-systems
#consistency
#networking
Aug 07
BLOG
Master-Replica Replication
Distributed Systems
#distributed-systems
#databases
#replication
Jul 19
BLOG
Decoding Durability - The D in ACID
Distributed Systems
#distributed-systems
#consistency
#databases
Jul 07
BLOG
Decoding Isolation - The I in ACID
Distributed Systems
#distributed-systems
#concurrency
#consistency
Jul 02
BLOG
Decoding Atomicity - The A in ACID
Distributed Systems
#distributed-systems
#consistency
#databases
Jul 02
BLOG
Decoding Consistency - The C in ACID
Distributed Systems
#distributed-systems
#consistency
#databases
Jun 22
BLOG
Architectural Patterns for Distributed Systems
Distributed Systems
#distributed-systems
#databases
#system-design
Jun 17
BLOG
Fallacies of Distributed Computing Every Engineer Should Know
Distributed Systems
#distributed-systems
#networking
Apr 19
BLOG
Modeling Udemy's Categories in a Relational Database
Distributed Systems
#distributed-systems
#databases
#python-internals
2020
(8)
Nov 29
BLOG
The 2Q Algorithm - Addressing LRU's Sub-Optimality
Distributed Systems
#distributed-systems
#caching
#databases
Aug 23
BLOG
Implementing LFU in O(1)
Distributed Systems
#distributed-systems
#caching
#memory-management
Jul 19
BLOG
Bitcask - A Log-Structured fast KV store
Distributed Systems
#distributed-systems
#data-structures
#databases
May 31
BLOG
The RUM Conjecture - Storage System Trade-offs
Distributed Systems
#distributed-systems
#consistency
#cryptography
May 24
BLOG
Consistent Hashing - What It Is and How to Implement It
Distributed Systems
#distributed-systems
#caching
#data-structures
May 03
BLOG
Copy-On-Write - When to Use It, When to Avoid It
Distributed Systems
#distributed-systems
#databases
#memory-management
Apr 26
BLOG
How MySQL Avoids Performance Hits from Table Scans
Distributed Systems
#distributed-systems
#caching
#databases
Apr 05
BLOG
Sliding Window Rate Limiting - Design and Implementation
Distributed Systems
#distributed-systems
2017
(3)
Mar 24
BLOG
Analyzing MongoDB Pagination Performance Differences
Distributed Systems
#distributed-systems
#databases
#evaluation
Mar 23
BLOG
MongoDB Pagination - Skip is Slow
Distributed Systems
#distributed-systems
#databases
Mar 22
BLOG
MongoDB Pagination - Two Approaches You Need to Know
Distributed Systems
#distributed-systems
#databases