What are embeddings?

The idea

Computers can’t compare meanings directly, but they can compare numbers. An embedding model reads a sentence and returns a list of a few hundred numbers, a vector. The model is trained so that sentences with similar meanings get vectors that point in similar directions. “How do I reset my password?” and “I forgot my login” end up close together, even though they share almost no words. “Reset the router” ends up further away.

You can think of each embedding as a position on a huge map of meaning. Finding related things becomes finding nearby points.

What they’re used for

• Search by meaning: find documents that answer a question, not just the ones that share its words.
• RAG chatbots: find the passages from your documents that a language model should read before answering.
• Recommendations: suggest products, articles or songs whose embeddings are close to what someone liked.
• Grouping and deduplication: cluster similar support tickets, or find near-duplicate records.
• Features for a model: feed embeddings of text into a classifier when you have few labeled examples.

How similarity is measured

The usual measure is cosine similarity: how closely two vectors point in the same direction, from -1 to 1. In practice, scores above a certain level mean “related,” but that level depends on the model, so check it on your own examples rather than using a fixed number.

# pip install sentence-transformers
from sentence_transformers import SentenceTransformer, util

model = SentenceTransformer("all-MiniLM-L6-v2")   # small, runs on a laptop CPU
texts = ["How do I reset my password?", "I forgot my login", "Reset the router to factory settings"]
vectors = model.encode(texts)
print(util.cos_sim(vectors[0], vectors[1:]))   # the login question scores higher

Storing and searching them

For a few thousand items, keep the vectors in memory and compare them directly. For millions, use a vector database or a vector index, which finds the nearest vectors quickly without comparing every one. Whatever you use, embed your documents and your queries with the same model; vectors from different models can’t be compared.