TCP Keep-Alive vs Application Heartbeat — Three Different Things Called “Keep-Alive”

The naming catastrophe — three things called "keep-alive"

Before anything else: engineers use the word "keep-alive" for three completely different things. Most arguments about which is "better" come from two people who actually mean different things.

For the rest of this post, "keep-alive" means the TCP socket option (the middle column). This post describes the HTTP header once, and then ignores it:

How TCP actually tracks "connected"

To understand why long-lived sockets die silently, you need one uncomfortable fact about TCP. It has no built-in liveness check by default.

The 3-way handshake establishes a connection — SYN, SYN-ACK, ACK. After that, both sides keep a record of the connection. The 4-tuple (src IP, src port, dst IP, dst port) identifies it. Each side tracks sequence numbers, so neither side reorders or duplicates the data. The OS marks the socket as ESTABLISHED.

And that's it. The socket can sit in ESTABLISHED forever, even if somebody unplugged the other machine a week ago. TCP only learns about a problem when something tries to send. Then the peer's TCP stack replies with RST, if it is still around but does not recognize the connection. Otherwise the local TCP retransmits and eventually times out.

Both TCP keep-alive and application heartbeats try to close this gap. They do it in different ways and at different layers.

The half-open socket — how connections silently die

"Half-open" describes a TCP connection where the two sides disagree. One side believes the connection is alive. The other does not, or no longer exists. Four common ways this happens in production:

1. Peer machine power-cut or kernel panic. The peer never sends a FIN. Your side never learns about it unless it tries to send.
2. NAT or stateful firewall drops the flow mapping. Home routers typically expire idle TCP flows around 5 minutes. AWS NLB defaults to 350 seconds. AWS ALB defaults to 60 seconds. Once the device drops the mapping, the NLB blackholes packets between the two sides. The ALB answers with RST instead.
3. Network path change. The mobile client moves from Wi-Fi to cellular. The 4-tuple's source IP changes. That orphans the old socket, and the server keeps a zombie.
4. Middlebox idle-flow eviction. Stateful firewalls and load balancers cap the number of concurrent flows they track. They evict the idle flows first — silently.

If you SSH into a server running a long-lived socket service and run ss -tan, you'll see something like:

$ ss -tan state established | wc -l
83214

$ ss -tan state established '( dport = :443 )' | head -5
ESTAB  0  0  10.0.1.4:43221  10.0.2.7:443
ESTAB  0  0  10.0.1.4:43227  10.0.2.7:443
ESTAB  0  0  10.0.1.4:43231  10.0.2.7:443

83,214 ESTABLISHED sockets. The kernel is happy. How many of those have a peer that will ever speak again? The kernel has no idea. It has not tried to send anything, so it has not noticed.

TCP keep-alive — what the kernel actually does

When you set SO_KEEPALIVE on a socket, the kernel periodically sends a probe packet on idle connections. The probe is a strange little thing. It is a TCP segment with no payload and a sequence number of current_seq - 1. The peer's stack sees a duplicate ACK request and answers with the current ACK. If the peer is gone, no answer comes. After enough silence, the kernel declares the connection dead. It surfaces an error to your app on the next read or write.

Three knobs control this on Linux:

# Defaults on most Linux distros
$ sysctl net.ipv4.tcp_keepalive_time      # 7200    (idle seconds before first probe)
$ sysctl net.ipv4.tcp_keepalive_intvl     # 75      (seconds between probes)
$ sysctl net.ipv4.tcp_keepalive_probes    # 9       (failed probes before giving up)

Do the math. A freshly-broken connection takes 7200 + (9 × 75) = 7875 seconds before the kernel detects the break. The delay is about 2 hours and 11 minutes. That is the default. For anything that matters, you must override per-socket:

// Node.js — first probe after 30s idle
socket.setKeepAlive(true, 30_000);

// Go — first probe after 30s idle
tcpConn.SetKeepAlive(true)
tcpConn.SetKeepAlivePeriod(30 * time.Second)

(Note: setKeepAlive in most high-level runtimes only exposes the idle time, not the probe interval or count. To tune those you use setsockopt with TCP_KEEPIDLE, TCP_KEEPINTVL, TCP_KEEPCNT directly.)

Application heartbeat — what your code does

An application heartbeat is just a message your protocol defines. Your code sends it on a timer and expects a reply on a timer. The crucial difference from TCP keep-alive is that the heartbeat traverses your application code. To answer it, the peer's event loop must spin. The peer must parse the message and write a reply. If the peer process is hung, deadlocked, or mid-GC for too long, no reply arrives. That is exactly what you wanted to detect.

Three patterns cover almost every case:

• Ping/pong. Built into the WebSocket protocol (RFC 6455 §5.5.2). The server sends a ping frame. The client's WebSocket library replies with a pong frame automatically. If a pong does not arrive in time, the server closes the socket.
• Periodic empty message. MQTT uses PINGREQ/PINGRESP. Kafka uses a consumer group heartbeat thread. The protocol defines a no-op message specifically for liveness.
• Read-deadline reset. Every successful read moves the deadline forward. If the deadline expires with no data, kill the socket. Go uses this with conn.SetReadDeadline. gRPC does the same with its own keepalive subsystem over HTTP/2.

The canonical Node.js ws library pattern looks like this:

// Server-side heartbeat — the pattern from the ws README
const wss = new WebSocketServer({ port: 8080 });

function heartbeat() {
  this.isAlive = true;
}

wss.on('connection', (ws) => {
  ws.isAlive = true;
  ws.on('pong', heartbeat);   // client replied — mark alive
});

const interval = setInterval(() => {
  wss.clients.forEach((ws) => {
    if (ws.isAlive === false) return ws.terminate(); // missed last round
    ws.isAlive = false;
    ws.ping();                  // send ping; pong handler resets the flag
  });
}, 30_000);

Two things to notice. First, terminate(), not close(). The latter waits for a graceful close handshake, and the peer can no longer answer it. Second, the design tolerates exactly one missed round before it kills the socket. A single dropped packet therefore does not cause a disconnect.

The decision framework

This is the load-bearing section. The right answer almost always depends on what sits between you and the other side. It also depends on the kind of failure you actually need to catch.

Four rules of thumb:

• "Is the route alive?" → TCP keep-alive.
• "Is the peer process alive and processing?" → application heartbeat.
• "Is there a NAT, LB, or firewall in the middle with an idle timeout?" → application heartbeat at an interval comfortably below that timeout.
• "Could my app GC-pause for 30s under load?" → tune heartbeat tolerance (how many misses before close), not just frequency. Otherwise a stop-the-world pause kills every healthy connection at once.

Cost — why you can't just heartbeat every second

Heartbeats look cheap and they mostly are — until they aren't.

Bandwidth math. 100,000 connections × one ~60-byte heartbeat every 30 seconds = ~200 KB/s on the wire. Trivial. Drop the interval to 1 second: 6 MB/s. Still fine for a single host on a 10G NIC.

The real cost is not bandwidth. It's wakeups. Every heartbeat is a timer firing, an event loop iteration, and a syscall to write a few bytes. The read side adds another syscall when the reply arrives. 100,000 connections at 1Hz heartbeat = 100,000 timer wakeups per second on each side. Add the inverse storm of replies. CPU use rises, and latency-sensitive work suffers.

If you need sub-second heartbeats at scale, batch them into a timer wheel. One timer tick then wakes many connections at once. Netty's HashedWheelTimer is the canonical implementation. Otherwise, keep the interval as long as the slowest middlebox in your path will tolerate.

Reference cheatsheet

The lesson

"Connected" is a lie your kernel tells you by default.

TCP does not probe. NAT boxes evict idle flows. Load balancers drop sockets after 60 seconds. Your peer process can hang while its kernel cheerfully answers probes. Pick the layer that answers the question you actually care about — route liveness or peer liveness. For anything long-lived, configure both. The cheapest debugging session is the one you avoid by setting two timers correctly the first time.