Kafka vs RabbitMQ: A Log Is Not a Queue

The comparison is usually run as a feature table, which makes the two look like competing products with different tick boxes. They are not the same kind of thing. One is a queue, one is a log, and if you get that distinction right, most of the feature table becomes derivable.

The distinction everything else follows from

RabbitMQ is a broker with queues. A producer publishes to an exchange, the exchange routes to one or more queues by rules you configure, and a consumer takes a message off a queue. When the consumer acknowledges it, the message is deleted. The queue is a to-do list, and the broker’s job is making sure each item is done once.

Kafka is an append-only log. Producers append to a partition. Each record gets an offset — its position in the file — and stays there until a retention policy removes it, whether or not anyone read it. Consumers track their own offset. The log is a record of what happened, and reading does not consume it.

The consequences

Replay. Kafka: set your offset backwards and read it again — reprocess a month of events after fixing a bug, or bootstrap a new service from history. RabbitMQ: the message is gone; there is nothing to rewind to.

Multiple independent consumers. Kafka: add a new consumer group and it reads everything from wherever it likes, without affecting anyone else. RabbitMQ: you must have set up an additional queue and binding in advance — a consumer added today cannot see yesterday’s messages.

Ordering. Kafka guarantees order within a partition, and the partition is chosen by the record’s key. Same key, same partition, same order — so ordering per customer or per entity is easy, and total ordering across everything means one partition and no parallelism. RabbitMQ preserves order in a queue, but the moment you add a second consumer for throughput, messages are processed concurrently and effective ordering is gone.

Retry of a single message. RabbitMQ excels here: negatively acknowledge one message and it can be redelivered or dead-lettered, while everything else continues. Kafka has no per-message acknowledgement — only a position. A record you cannot process blocks its partition until you skip it, and “skip it” means writing it to a dead-letter topic yourself.

Back-pressure. RabbitMQ queues grow in memory and disk and the broker starts pushing back on publishers; a slow consumer becomes the producer’s problem. Kafka producers append at full speed regardless of how far behind consumers are; lag is a number you monitor, not a force applied to producers.

The comparison, as consequences

	RabbitMQ (queue)	Kafka (log)
After consumption	deleted	retained until retention
Replay history	no	yes, by offset
New consumer sees past	no	yes
Ordering	per queue, lost with concurrency	per partition, by key
Retry one message	native, per message	manual, via dead-letter topic
Routing	rich (topic, header, fanout)	by partition key only
Delayed / priority delivery	yes	no
Typical throughput ceiling	tens of thousands/s	millions/s
Operational weight	modest	substantial

Choosing without ceremony

Use RabbitMQ when the message is a task. Send this email. Resize this image. Generate this invoice. These want per-message retry, dead-lettering, priorities and routing — all native. Nobody needs to replay last Tuesday’s password resets.

Use Kafka when the message is a fact. Order placed. Payment captured. Meter reading recorded. Facts are interesting to more than one consumer, often to consumers that do not exist yet, and being able to rebuild a projection by replaying history is worth the operational cost.

The clearest tell: ask whether a second team might want the same messages later for a different purpose. If yes, you want a log. If the message is an instruction to do one thing once, you want a queue.

What neither of them gives you

Exactly-once delivery. Kafka’s “exactly-once semantics” is real but narrow — it covers Kafka-to-Kafka processing with transactions, not the moment your consumer charges a card and then fails before committing its offset.

Both systems give you at-least-once in the general case. Duplicates are not an edge case; they are the normal consequence of a retry after an ambiguous failure. Consumers must be idempotent regardless of which broker you chose, and that requirement is not something the broker can take off your hands.

What you have actually chosen

Not a vendor. You have chosen whether reading destroys the message.

If it does, you get a to-do list with excellent per-item control and no memory. If it does not, you get a rewindable history and must handle failures yourself, per partition, at the position level. Both are correct designs. Picking the log because it is the one with higher benchmark numbers is how teams end up operating a distributed commit log to send welcome emails.