Introduction: The Distributed Python Problem

Scaling is Hard

Python is the dominant language of AI and data science, but it has a fundamental constraint: the Global Interpreter Lock (GIL). The GIL is a mutex inside the CPython interpreter that prevents more than one thread from executing Python bytecode at a time. In practice, this means a Python script runs on one CPU core, no matter how many cores your machine has.

• Scenario: You have a function process_image() that takes 1 second. You have 1 million images.
• Sequential (1 core): 1,000,000 seconds ≈ 11.5 days.
• Goal: Run it on a cluster of 100 machines with 8 cores each (800 cores total) and finish in roughly 20 minutes.

The math is simple. The engineering is not.

The Old Way

Before frameworks like Ray existed, distributing Python work across a cluster required you to:

1. Manually SSH into each machine and copy your code.
2. Partition your dataset into equal chunks and distribute them.
3. Launch the script on each machine with the correct chunk.
4. Wait for all machines to finish, then collect and merge results.
5. Handle failures: if one machine crashes mid-job, you lose that chunk's work.

This approach is brittle, slow to develop, and nearly impossible to debug. Every team reinvented the same fragile scaffolding.

The Ray Way

Ray is an open-source distributed computing framework that lets you scale Python code from a laptop to a cluster of thousands of machines with minimal code changes. It handles scheduling, inter-process communication, fault tolerance, and resource management — so you focus on your logic, not the plumbing.

Ray was created at UC Berkeley's RISELab and is now maintained by Anyscale. It has become the de facto standard for distributed Python in the AI/ML ecosystem, used by companies like OpenAI, Shopify, and Uber.