The Universal Translator Problem
Your e-commerce website is likely written in JavaScript. Your mobile app is in Swift or Kotlin. Your ML model is almost certainly written in Python using PyTorch or scikit-learn. These systems cannot directly call each other's code — they run in separate processes, often on separate machines, and speak different programming languages.
The solution is a network API: a well-defined contract that any language can speak. In practice, this is almost always a REST API over HTTP, or increasingly gRPC for high-performance internal services.
The Request-Response Lifecycle
Here is exactly what happens when a client application asks your model for a prediction:
Step 1 — Client sends a request.
The client (your website, app, or another service) sends an HTTP POST request to a URL endpoint on your model server. The body contains the input data, serialized as JSON:
POST /predict HTTP/1.1
Content-Type: application/json
{
"customer_age": 25,
"items_in_cart": 4,
"session_duration_seconds": 312
}
Step 2 — The serving layer preprocesses the input.
The server receives the JSON and deserializes it. It then applies any preprocessing required: scaling numeric features, encoding categoricals, reshaping arrays into tensors. This preprocessing must be identical to what was applied during training — a mismatch here is one of the most common sources of silent production bugs.
Step 3 — Inference runs.
The preprocessed input is passed to the loaded model. The model executes its forward pass and produces a raw output (a probability, a class index, an embedding vector, etc.).
Step 4 — The server returns the response.
The raw output is postprocessed (e.g., converting a probability to a human-readable label) and returned as JSON:
HTTP/1.1 200 OK
Content-Type: application/json
{
"purchase_probability": 0.85,
"recommendation": "show_discount_banner"
}
REST vs. gRPC
REST uses JSON over HTTP/1.1. It is human-readable, easy to debug with tools like curl, and universally supported. It is the right default for most use cases.
gRPC uses Protocol Buffers (a compact binary format) over HTTP/2. It is significantly faster and more bandwidth-efficient than REST, making it the preferred choice for high-throughput internal microservice communication — for example, when a recommendation service calls a model server thousands of times per second. The trade-off is added complexity: you must define a .proto schema file and generate client/server code from it.
Interviewers often ask: "How would you expose your ML model to a production application?" A strong answer distinguishes between REST (simple, universal, good default) and gRPC (binary, low-latency, better for high-throughput internal services), and mentions the preprocessing consistency problem — the fact that training-serving skew in feature transformations is a leading cause of silent model degradation in production.