The Power of Columnar: Parquet and ORC

Why Columnar Wins for Analytics

Besides reading less data, columnar storage enables two massive performance features for analytical workloads:

1. Superior Compression

Compression works by finding repeated patterns. The uniformity of columnar data makes it dramatically more compressible than row data:

• Row data is noisy: [1, "Alice", "USA", 30, "[email protected]", 2, "Bob", "Canada", 25, ...]. Mixed types (integers, strings, floats) interleaved together don't compress well.
• Column data is uniform: ["USA", "Canada", "USA", "Mexico", "USA", "USA", ...]. A column of country codes has high repetition and compresses extremely well. A column of ages is just integers — a compression algorithm can apply integer-specific encoding (like run-length encoding or delta encoding) to shrink it further.

In practice, columnar formats with good compression achieve 5–10x better compression ratios than equivalent row-based storage with the same algorithm.

2. Column Pruning (Projection Pushdown)

If your table has 100 columns but your query selects only 3, a columnar format is smart enough to physically read only the files or byte ranges corresponding to those 3 columns. It "prunes" (skips) the other 97% of the I/O entirely. This optimization is called projection pushdown — the query engine "pushes down" the column selection to the storage layer before any data is read.

3. Predicate Pushdown and Statistics

Modern columnar formats store column statistics (min value, max value, null count, distinct count) in the file's metadata. When a query includes WHERE age > 50, the query engine reads the metadata first. If a file's age column has a max value of 40, the engine skips the entire file without reading a single data byte. This is called predicate pushdown or statistics-based pruning.

Parquet vs. ORC

These are the two dominant open-source columnar file formats:

Parquet is the gold standard for interoperability — virtually every modern data tool can read and write Parquet. ORC often performs slightly better within the Hive and Presto/Trino ecosystems due to its more advanced built-in indexing (bloom filters allow skipping row groups based on exact-match predicates, not just range predicates).

As a DBRE in a modern data environment, moving analytical data out of a row-based RDBMS and into Parquet files in an object store (like S3) is a standard pattern for improving query performance and dramatically lowering costs.