Ray Ecosystem: High-Level Libraries

Ray's primitives (Tasks and Actors) are powerful but low-level. The Ray ecosystem provides high-level libraries for the most common distributed AI/ML patterns, each built on top of these primitives.

Ray Tune: Hyperparameter Optimization

Training a neural network requires choosing hyperparameters: learning rate, batch size, number of layers, dropout rate. Finding the best combination manually is impractical.

Ray Tune automates this. You define a training function and a search space, and Tune launches hundreds of parallel trials across your cluster, using algorithms like Bayesian Optimization or Population-Based Training (PBT) to intelligently explore the space.

from ray import tune

def train_model(config):
    # config contains sampled hyperparameters
    model = build_model(layers=config["layers"], lr=config["lr"])
    accuracy = train_and_evaluate(model)
    tune.report(accuracy=accuracy)

tuner = tune.Tuner(
    train_model,
    param_space={
        "lr": tune.loguniform(1e-4, 1e-1),
        "layers": tune.choice([2, 4, 8]),
    },
    tune_config=tune.TuneConfig(num_samples=50),
)
results = tuner.fit()
best_config = results.get_best_result().config

Tune also supports early stopping: if a trial is clearly underperforming after a few epochs, Tune kills it and reallocates those resources to more promising trials. This can reduce total compute by 10x.

Ray Train: Distributed Model Training

Training large models on a single GPU takes too long. Ray Train distributes training across multiple GPUs and machines, handling the complexity of gradient synchronization.

It integrates with PyTorch, TensorFlow, and Hugging Face Transformers. The key abstraction is the Trainer:

from ray.train.torch import TorchTrainer
from ray.train import ScalingConfig

trainer = TorchTrainer(
    train_loop_per_worker=my_training_function,
    scaling_config=ScalingConfig(num_workers=4, use_gpu=True),
)
result = trainer.fit()

Ray Train handles: spawning worker processes, setting up torch.distributed communication backends (NCCL for GPU-to-GPU), checkpointing, and fault tolerance (if one worker crashes, the job can resume from the last checkpoint).

Ray Serve: Model Serving

Once a model is trained, it needs to be served behind an HTTP endpoint. Ray Serve is a scalable model serving library that lets you:

• Deploy multiple models behind a single endpoint.
• Compose models into pipelines (e.g., preprocess → classify → postprocess).
• Scale each stage independently (e.g., 2 preprocessors, 8 GPU inference workers, 1 postprocessor).
• Perform online A/B testing by routing traffic between model versions.

from ray import serve

@serve.deployment(num_replicas=4, ray_actor_options={"num_gpus": 1})
class ImageClassifier:
    def __init__(self):
        self.model = load_model()

    async def __call__(self, request):
        image = await request.json()
        return self.model.predict(image)

serve.run(ImageClassifier.bind())

Ray Data: Scalable Data Pipelines

Ray Data is a distributed data loading and transformation library designed for ML pipelines. It handles:

• Reading from S3, GCS, HDFS, or local disk in parallel.
• Applying transformations (map, filter, batch) across the cluster.
• Streaming data into training jobs without loading the entire dataset into memory.

import ray.data

ds = ray.data.read_parquet("s3://my-bucket/training-data/")
ds = ds.map(preprocess_image)
ds = ds.filter(lambda row: row["label"] != -1)

# Stream directly into a training loop
for batch in ds.iter_batches(batch_size=256):
    train_step(batch)