Turning Meaning into Numbers
Before you can store data in a vector database, you need to convert it into a form the database understands: a vector embedding.
Think of it this way: if you plotted every word in the English language on a giant map, words with similar meanings would cluster together. "Happy," "joyful," and "elated" would be neighbors. "Automobile," "car," and "vehicle" would form their own cluster, far from the happiness cluster. An embedding model does exactly this — it assigns every piece of data a precise location on a high-dimensional map.
Concretely, an embedding is an array of floating-point numbers, for example:
"The cat sat on the mat" → [0.12, -0.45, 0.88, 0.03, -0.71, ...]
A typical embedding vector has 768 to 3,072 dimensions depending on the model. Each dimension captures some latent feature of the data — though individual dimensions are not human-interpretable. What matters is the geometric relationship between vectors: vectors for semantically similar content point in similar directions in this high-dimensional space.
Embedding models are specialized neural networks trained to produce these vectors. Common examples include:
| Model | Provider | Typical Use |
|---|---|---|
text-embedding-3-small |
OpenAI | General text |
text-embedding-3-large |
OpenAI | High-accuracy text |
all-MiniLM-L6-v2 |
Sentence Transformers | Lightweight, local |
CLIP |
OpenAI / open-source | Images and text together |
A critical rule: you must use the same embedding model at ingestion time and at query time. If you embed your documents with text-embedding-3-large and then embed your user's query with all-MiniLM-L6-v2, the resulting vectors live in completely different mathematical spaces and the similarity scores will be meaningless.
Similarity Metrics: How Closeness Is Measured
Once vectors are stored, the database needs a way to measure how similar two vectors are. The two most common metrics are:
- Cosine Similarity: Measures the angle between two vectors. A score of 1.0 means identical direction (same meaning), 0 means perpendicular (unrelated), -1 means opposite. This is the most common choice for text because it's insensitive to vector magnitude — only direction matters.
- Euclidean Distance (L2): Measures the straight-line distance between two points in space. Smaller distance means more similar. Better suited for image embeddings and cases where magnitude carries meaning.
Most vector databases let you choose the metric at index creation time. Choosing the wrong metric for your embedding model can significantly degrade search quality.
Interviewers often ask: "Why use cosine similarity instead of Euclidean distance for text embeddings?" The answer is that text embedding models are trained to encode meaning in the direction of the vector, not its magnitude. Two documents about the same topic but of different lengths will have vectors pointing in similar directions but with different magnitudes. Cosine similarity correctly identifies them as similar; Euclidean distance may not.