Redis Internals in a Nutshell

Redis is often introduced as “an in-memory key-value store”, which is true and explains almost nothing — plenty of systems keep data in memory and are still slow. What makes Redis quick is a set of deliberate removals — no disk on the read path, no locks, no thread per connection, no query planner — and a handful of data structures that change shape underneath you to stay cheap.

One thread, and why that is a feature

Command execution in Redis is single-threaded. One event loop accepts connections, reads commands, executes them and writes replies. Since Redis 6, I/O reading and writing can be spread over helper threads, but the execution of commands themselves is still serialised on one core.

That sounds like a bottleneck and mostly is not. A command that touches memory finishes in well under a microsecond, so a single core handles hundreds of thousands of operations per second. In exchange you get properties that are expensive elsewhere: every command is atomic by construction, and there are no locks, no contention and no context switching between request handlers.

The cost is that one slow command blocks everything. KEYS * on a large keyspace, a DEL of a multi-million-element set, or a careless Lua script will stall every other client. This is why SCAN exists — a cursor-based iteration that returns a bounded slice per call — and why UNLINK exists, which detaches a key immediately and frees the memory on a background thread.

Data structures that change encoding

Redis types are interfaces, not implementations. Each has a compact encoding for small values and a scalable one for large values, and it switches automatically and irreversibly upward.

A hash starts life as a listpack: a single flat, contiguous blob of entries. Lookup is a linear scan, which for a dozen fields is faster than a hash table because it is one cache-friendly allocation with no pointer chasing. Cross the configured thresholds (hash-max-listpack-entries, default 128, or hash-max-listpack-value, default 64 bytes) and it converts to a real hash table. Sets behave similarly, with an extra intset encoding for all-integer sets. Lists are a quicklist: a linked list whose nodes are each a listpack, which keeps memory overhead low while still allowing cheap pushes at both ends.

Sorted sets are the interesting one. Above the small-value threshold a zset is two structures over the same data — a hash table mapping member to score for O(1) score lookup, and a skip list ordered by score for O(log n) range queries. A skip list rather than a balanced tree because it needs no rotations and range iteration is a plain linked-list walk.

Strings use SDS, a length-prefixed structure that keeps STRLEN at O(1) and is binary-safe rather than null-terminated.

The main hash table itself grows by incremental rehashing. When it needs to resize, Redis allocates a second table and migrates a few buckets on every subsequent operation rather than stopping the world. During the migration reads check both tables and writes go only to the new one.

Expiry is sampled, not scheduled

A key with a TTL is not removed by a timer. Two mechanisms handle it. Lazy expiry checks the TTL whenever a key is accessed and deletes it then. Active expiry runs about ten times a second, samples twenty keys with TTLs at random, deletes the expired ones, and repeats immediately if more than a quarter of the sample was expired.

The consequence matters for capacity planning: expired keys occupy memory until one of those two things happens. A large set of keys that all expire at once is reclaimed over seconds, not instantly.

Eviction, and why it is approximate

When maxmemory is reached, the configured maxmemory-policy decides what happens. noeviction — the default — starts failing writes, which surprises people using Redis as a cache and is exactly right for people using it as a store. allkeys-lru and allkeys-lfu evict from the whole keyspace; the volatile-* variants only consider keys that have a TTL.

Redis does not maintain a true LRU list, because that would cost a pointer pair per key and a list update on every access. Instead each object carries a 24-bit clock field, and eviction samples a handful of keys (maxmemory-samples, default 5) and evicts the oldest of the sample, keeping a pool of good candidates between rounds. With five samples the choice is very close to true LRU at a fraction of the memory cost. LFU replaces the clock with a probabilistic counter that decays over time, which is the better default for a cache where a one-off scan should not evict genuinely hot keys.

Persistence: forks, copy-on-write, and honest durability

RDB snapshots call fork(). The child inherits a copy-on-write view of memory and writes it out while the parent keeps serving. Cheap in CPU, but the parent’s writes during the snapshot duplicate pages, so peak memory can spike — which is why a Redis instance holding more than half the machine’s RAM is a liability.

AOF appends every write command to a log. appendfsync everysec, the sensible default, bounds worst-case data loss to about one second; always gives durability per command at a large throughput cost. The log is rewritten periodically to a compacted form. Modern Redis defaults to a hybrid: the rewrite emits an RDB preamble followed by the subsequent AOF commands, giving fast loading and a small window of loss.

Replication is asynchronous, and this is the point most often missed: a primary acknowledges a write before any replica has it. WAIT lets you block until n replicas confirm, but Redis is not a consensus system and a failover can lose recent writes. Design around that rather than hoping.