The Cost of a Network Packet
Sending a network packet is not free. Every UDP datagram carries overhead: IP headers (~20 bytes), UDP headers (~8 bytes), CPU interrupts to process the packet, and network switch processing. More importantly, the number of packets in flight determines network congestion, not just their total size. A network handling 100,000 small packets per second is more stressed than one handling 1,000 larger packets per second carrying the same total bytes.
Piggybacking Defined
Piggybacking means attaching additional information to a message you were already going to send, rather than sending a separate message for that information.
In SWIM/Memberlist, every PING and ACK message that goes out for failure detection purposes also carries a small gossip payload — recent state changes (node joins, node failures, node suspicions) that need to be disseminated. The failure detection traffic and the dissemination traffic share the same packets.
Without Piggybacking:
[PING packet] → 28 bytes, sent every 1s
[GOSSIP packet] → 40 bytes, sent separately
Total: 2 packets, 68 bytes
With Piggybacking:
[PING + gossip payload] → 68 bytes, sent every 1s
Total: 1 packet, 68 bytes
The byte count is the same, but the packet count is halved. At 10,000 nodes each sending 1 ping per second, that is 10,000 fewer packets per second in the network — for free.
What Gets Piggybacked?
Not all gossip is equally urgent. Memberlist maintains a broadcast queue — a priority queue of pending gossip items. Items are ordered by how many times they have been transmitted (less-transmitted items go first) and by urgency (a FAILED event is more urgent than a JOIN event). Each outgoing packet pulls the top items from this queue up to a configurable byte limit, ensuring the most important and least-propagated information gets sent first.
Items are removed from the broadcast queue after they have been transmitted $O(\log N)$ times — at that point, the mathematical guarantee says the information has almost certainly reached every node.