Embeddings: measure relatedness without mistaking it for truth

Represent text for comparison

An embedding model maps input into a vector: an ordered list of numbers. Training encourages useful relationships in that representation space. Search compares a query vector with document vectors to find candidates. The dimensions are not usually human-readable labels such as “returns” or “delivery.”

Use a compatible representation for both sides. Equal vector length is necessary for comparison but does not prove that two models share a meaningful space. Record the embedding model and preprocessing version with your index. Some models prescribe different query and document prefixes; follow the chosen model's documentation.

Work through cosine similarity

Cosine similarity compares vector direction using the dot product divided by the product of lengths. It ranges from -1 to 1 for nonzero real vectors. A high value means closeness under that representation, not a calibrated probability that a passage answers the question. Zero indicates orthogonality, not a universal linguistic judgment of “unrelated.”

from math import sqrt

def cosine(a, b):
    if len(a) != len(b) or not a:
        raise ValueError("Vectors must have equal, nonzero dimensions")
    na = sqrt(sum(x*x for x in a))
    nb = sqrt(sum(x*x for x in b))
    if na == 0 or nb == 0:
        raise ValueError("Cosine is undefined for a zero vector")
    return sum(x*y for x, y in zip(a, b)) / (na * nb)

print(round(cosine([1, 0], [0.8, 0.2]), 3))  # 0.970
print(round(cosine([1, 0], [0, 1]), 3))      # 0.000

These two-dimensional vectors demonstrate the arithmetic only. They are not actual sentence embeddings and cannot establish how a real retrieval model ranks Northstar's policies.

Make a small, honest search experiment

Embed the three policies and several queries with one selected model. Store the vectors alongside IDs and original text. For each query, rank the policy vectors and inspect the top results. The expected match for “send it back” is returns; the expected match for a broken parcel is damage.

Include a difficult negative: “Can I return a clearance item?” A passage about ordinary returns may be close while omitting the exclusion. Retrieval must supply the relevant condition, not simply a topic match. Add exact identifiers and unfamiliar product codes as well; lexical search may handle these more predictably than semantic similarity.

Manage changes and scale deliberately

Changing the embedding model usually means rebuilding document embeddings and matching query encoding. Keep the old and new indexes separate while comparing them. Store document versions so deleted or superseded text does not remain searchable forever.

Batching, caching unchanged content, and selecting an appropriate vector size can reduce operational work, but measure the tradeoff. Embeddings can support clustering, duplicate detection, and recommendations in addition to retrieval. Each use requires its own evaluation. A model that works for short English policies may behave differently on multilingual text, tables, or long technical documents.

References and further reading

Use these primary references for deeper study and current API details. Examples in this lesson use fictional Northstar data.

• Official OpenAI documentation — embeddings