AI Infrastructure & DataOps

Introduction: The Digital Supply Chain

Two Paradigms: ETL vs. ELT

Batch Processing: The Heavy Lifter

Spark Internals: RDDs vs. DataFrames and the Catalyst Optimizer

Stream Processing: The Need for Now

Flink Concept 1: Windowing and State

Flink Concept 2: The Problem of Time and Watermarks

A Note on Kafka Streams

Choosing the Right Tool: A Decision Framework

Introduction: Giving AI Long-Term Memory

The Core Idea: Embeddings

RAG Architecture: The "Open-Book" Test

The Challenge of Search: Indexing Algorithms

Indexing Algorithm 1: IVF (Inverted File Index)

Indexing Algorithm 2: HNSW (Hierarchical Navigable Small World)

Choosing a Vector Database: The Landscape

Introduction: The "Last Mile" of Machine Learning

Inference APIs: How Apps Talk to Models

Serving Tools: From Simple to Supercharged

The Model Registry: Version Control for ML Artifacts

Testing in Production: Shadow Mode

Testing in Production: A/B Testing and Canary Releases

Introduction: MLOps on Steroids

The VRAM Bottleneck and Quantization

Efficient Serving: The Magic of vLLM

Token Economics: The Currency of AI

Context Window Optimization

Customization: The Problem with Fine-Tuning

Adapters: LoRA and QLoRA

Introduction: The High Cost of Intelligence

GPU Slicing with MIG: The "Pizza Oven" Problem

Training on a Budget: Spot Instances

The Art of Checkpointing: Surviving the Eviction

The Hidden Bottleneck: NUMA and CPU-GPU Affinity

Introduction: It's Not About the FLOPS

HBM (High Bandwidth Memory): The Firehose

The HBM Bottleneck: Memory-Bound vs. Compute-Bound Workloads

Topology Level 1: NVLink and NVSwitch (Intra-Node)

Topology Level 2: Ethernet vs. InfiniBand (Inter-Node)

The All-Reduce Operation: Why Network Topology Determines Training Speed

GPU Direct RDMA: Eliminating the CPU from the Data Path

Introduction: Hitting the Wall

Data Parallelism (DDP): The Cloning Technique

Model Parallelism: Tensor Parallelism (TP)

Model Parallelism: Pipeline Parallelism (PP)

ZeRO (Zero Redundancy Optimizer): The Memory Saver

Combining Strategies: 3D Parallelism

Introduction: The "Plumbing" Problem in ML

The Enemy: Training-Serving Skew

Architecture: Online vs. Offline Stores

Point-in-Time Correctness (Time Travel)

Feature Reuse and the Feature Registry

Introduction: The Silent Failure

Data Contracts & Schema Registries

Data as Code: Great Expectations & Soda

The Silent Killers: Data Drift vs. Concept Drift

Introduction: The Curse of Dimensionality

HNSW: Hierarchical Navigable Small World

IVF: Inverted File Index (The "Clustering" Part)

PQ: Product Quantization (The "Compression" Part)

The Great Trade-off: HNSW vs. IVF-PQ

Introduction: The GPU is a Hungry Mouth

Parallel Filesystems: Lustre & GPFS

Object Storage: The Data Lake

The Fast Path: GPUDirect Storage (GDS)

The "Small File Problem" (The Inode Killer)

Solving Small Files: Aggregation (TAR/TFRecord/Parquet)

Streaming Architecture: Training Without Downloading Everything

Introduction: REPL on Steroids

Architecture: The Frontend and The Kernel

The Hidden Trap: Hidden State

Visualizing for Debugging: Matplotlib & Seaborn

Notebooks in Production: When to Graduate to Scripts

Introduction: The "Dark Data" Problem

The Data Catalog: Google for Your Data

Data Lineage: The Family Tree of Your Data

Architecture: Pull vs. Push Metadata

Introduction: The Centralization Trap

The Algorithm: How FedAvg Works

The Challenge: Non-IID Data and Client Drift

Privacy-Preserving AI: Differential Privacy and Secure Aggregation

Introduction: The Distributed Python Problem

Core Concept: Tasks (Remote Functions)

Core Concept: Actors (Stateful Services)

Ray Architecture: The Object Store and Cluster Internals

Ray Ecosystem: High-Level Libraries

Introduction: Batch vs. Service

Slurm Architecture: The Controller and The Nodes

The Language of Slurm: SBATCH

Resource Allocation: GRES, Tasks, and srun

Partitions and QoS: The Traffic Lights

Checkpointing: Surviving Preemption and Node Failures

Slurm vs. Kubernetes: The Cage Match

Introduction: The Precision Trap

The Physics of Quantization: Mapping the Range

Calibration: Finding the Boundaries

PTQ vs. QAT: When to Quantize?

Hardware Acceleration: Why Integers Are Cheaper

Mixed-Precision Quantization: Not All Layers Are Equal