Pipes and Named Pipes (FIFOs)

The Oldest Trick in the Book

A pipe is the simplest IPC mechanism: a one-way channel where one process writes bytes in one end and another process reads them out the other. Think of it as a physical pipe — water flows in one direction, and whatever you pour in comes out the other side in the same order. Pipes are the backbone of the Unix philosophy: small programs that do one thing well, chained together.

Anonymous Pipes (|) are what you use every time you run a shell pipeline — for example, chaining ls, grep, and wc together so each command's output feeds directly into the next command's input.

The shell creates an anonymous pipe between each pair of commands in the chain. ls writes to the write-end of the first pipe; grep reads from the read-end of that pipe and writes to the write-end of the second; wc reads from the read-end of the second. The data flows as a raw byte stream — the kernel buffers it in a small in-kernel ring buffer (typically 64 KB on Linux). If the writer fills the buffer before the reader consumes it, the writer blocks (is suspended by the kernel) until space is available. This natural backpressure is a feature, not a bug.

Anonymous pipes have one critical limitation: they only work between processes that share a common ancestor (typically a parent-child relationship), because the file descriptors must be inherited. Two completely unrelated processes cannot use an anonymous pipe.

Named Pipes (FIFOs) solve this. A FIFO is created with mkfifo and appears as a special file in the filesystem — its permission string starts with a p (rather than - for a regular file), marking it as a pipe rather than something holding actual data on disk.

Any two processes that can see the filesystem path can use it as a rendezvous point. Process A writes to /tmp/my_pipe; Process B reads from it. The data never touches the disk — the filename is just a lookup key. The kernel holds the data in memory. If Process A writes before Process B is ready, Process A blocks until Process B opens the read end. This blocking behavior is deterministic and useful for synchronization.

Key limitations of pipes:
- Unidirectional: Data flows one way only. For bidirectional communication, you need two pipes.
- Byte stream, not messages: The kernel does not preserve message boundaries. If Process A writes "Hello" and "World" in two separate write() calls, Process B might read "HelloWorld" in one read() call. You must implement your own framing (e.g., newline-delimited, length-prefixed).
- No random access: You cannot seek within a pipe. It is strictly FIFO.