Vector Fundamentals

Course overview

A vector is both an ordered list of numbers and a geometric object with a direction and a length. Embeddings map text, images, and other objects to vectors; retrieval then uses a similarity measure to find neighbors. This path connects linear algebra with practical semantic search. Its central concern is choosing and evaluating a metric, rather than treating a similarity score as mysterious.

Where this fits in the overall path

Vector fundamentals belongs to the Mathematics and data foundations stage. After linear algebra, it leads into embeddings, vector databases, semantic search, and RAG. It focuses on how to compare representations, choose a measure, and verify the quality of nearest-neighbor retrieval.

Learning objectives

• Calculate and explain vector norms, normalization, dot products, and cosine similarity.
• Compare when cosine, dot product, and Euclidean distance are appropriate.
• Understand the basic trade-offs of high-dimensional spaces and approximate nearest-neighbor retrieval.
• Build an embedding-retrieval path and evaluate top-k with labeled relevance sets.

Prerequisites

You should be comfortable with squares, square roots, and loops over lists. Python Fundamentals is a useful starting point. Linear Algebra helps with coordinates, dot products, norms, and batched matrix computation, but it is not a hard prerequisite for this path.

Recommended order

1. Representations, coordinates, and norms: understand components, dimensions, length, and normalization.
2. Dot products and cosine similarity: understand directional similarity and the effect of magnitude.
3. Distance, normalization, and high-dimensional spaces: compare Euclidean distance, cosine, and dot product.
4. Embeddings and semantic search: go from objects to vectors to candidate results.
5. Project: a minimal vector retriever: implement top-k and Recall@k with the standard library.

Hands-on entry point

Start with Project: a minimal vector retriever, then run vector_search.py and test_vector_search.py. The example uses only the Python standard library to implement dot products, norms, unit normalization, cosine, Euclidean distance, top-k, and Recall@k from first principles. It also shows how vector magnitude, normalization, and metric choice change a ranking.

Mastery criteria

• I can calculate L1/L2 norms, a dot product, cosine similarity, and Euclidean distance.
• I can explain the relationship between dot product and cosine after normalization.
• I can explain why vector dimension is not an interpretable count of “topics.”
• I can choose a metric from the model documentation and avoid comparing raw scores across models.
• I can implement top-k, handle zero vectors, and calculate Recall@k from labeled relevance data.
• I can distinguish the speed/recall trade-off between exact search and ANN.
• I can run the example and its nine tests in both normal mode and python -O, and explain why assert is not production validation.

Connections to other knowledge bases

• Linear Algebra supplies the ideas of space, projection, and matrix operations.
• Embeddings explains how models create vectors; Vector Databases covers indexing, filtering, and persistence.
• Semantic Search, Reranking, and RAG connect candidate retrieval to a complete answer path.
• Evaluation Framework and Benchmark Design cover representative query sets, relevance labels, and regression gates.

Primary references

• Google ML Crash Course: Embeddings
• Google: Measuring Similarity from Embeddings
• scikit-learn 1.9.0: Pairwise metrics, affinities and kernels
• Malkov and Yashunin: HNSW
• MIT OCW 18.06 Linear Algebra

Verified on 2026-07-14. The mathematical relationships are stable; the stable API, ANN implementation, and a model’s metric convention can change. Check the version and model card you actually use.