Semantic Search Learning Path

Semantic Search

About this knowledge base

The input to semantic search is not “a string,” but an information need with identity, time, and business conditions. Its output is not a final answer, but a set of traceable, authorized evidence candidates with sufficient recall. A complete first-stage retrieval system commonly combines:

• lexical signals such as keywords/BM25;
• dense-vector signals produced by embeddings;
• optional learned sparse representations;
• hard filters for tenant, ACL, status, language, product, and validity period;
• Reciprocal Rank Fusion (RRF) or calibrated score fusion;
• query-document relevance judgments and offline/online evaluation.

Semantic search is not synonymous with vector similarity: error codes, model numbers, numbers, negation, and proper nouns often need a lexical channel. Nor is “semantically similar” synonymous with “answers the question.” A retrieval system must explain which route produced each candidate, which authorization it satisfies, which representation version it used, and whether it improves business relevance on a local query set.

Capability boundary

Vector Databases handles storage, ANN, filter execution, and lifecycle. This course handles queries, recall, fusion, and relevance evaluation. Reranking uses a more expensive joint model to reorder candidates. RAG organizes evidence, citations, and answers. Neither a reranker nor a generative model can recover a document completely missed at first stage.

Where this course fits in the overall route

This knowledge base belongs to the Retrieval and Data domain. In the RAG role track, Document Parsing, Chunking Strategies, and Knowledge Base Construction create traceable retrieval units; Embeddings and Vector Databases provide dense representations and execution foundations. This course combines them into an evaluable recall system before Reranking. The sequence describes integration order; it does not add entire courses as hard prerequisites.

Learning objectives

After completing this knowledge base, you can:

• distinguish a user’s words, an information need, a query, a document/chunk, qrels, and a final answer;
• define query/document encoding, text concatenation, truncation, and version contracts for asymmetric retrieval;
• explain the intuition behind BM25 term frequency, document frequency, and length normalization;
• use cosine, dot product, and L2 correctly, rejecting direct comparison of raw scores across models;
• compare independent sparse/keyword, dense, and hybrid baselines;
• use RRF to fuse incomparable channel rankings, and understand the rank window and candidate budget;
• build tenant/ACL filters from a trusted identity, failing closed for an empty identity or abnormal state;
• design a query set that includes paraphrases, error codes, numbers, negation, multilingual queries, no-answer cases, and authorization boundaries;
• distinguish ANN recall, business Recall@k/Hit@k, MRR, nDCG, and safety gates;
• use reproducible experiments to decide whether query rewriting, filtering, fusion, or a model upgrade truly improves the system.

Prerequisites

• Vector Fundamentals: cosine, dot product, distance, and top-k.
• Embeddings: model roles, space contracts, and migration.
• Vector Databases: exact/ANN search, payload filters, and indexes.
• Data Annotation: annotation specifications, agreement, and adjudication.
• Evaluation Framework: later incorporates offline retrieval metrics into system-level evaluation.

You only need to run a Python 3 standard-library script. This course requires no real model, database, or API key.

Core terminology

Term	Beginner explanation	Do not confuse it with
information need	The problem the user actually wants solved	The raw string
query	A replayable retrieval request with text, identity, filters, and version	A rewritten form must retain the original query
document / passage	A retrieval unit that can be returned, judged, and traced to a source	It should match the unit returned online
qrels	Relevance annotations for query-document pairs	A database authorization table
sparse / lexical	A term-and-weight representation suited to an inverted index	BM25 and learned sparse retrieval are not the same algorithm
dense	Fixed-dimensional nearest-neighbor vector retrieval	Similarity is not a relevance probability
first-stage retrieval	Inexpensively produces a larger candidate set	It does not seek perfect final ranking
fusion	Combines candidates from multiple routes	Raw scores cannot be assumed to add directly
rank window	Maximum candidates per route entering fusion	Final top-k
hard filter	Non-negotiable constraints such as tenant, ACL, status, and expiry	Something generation can repair afterward

Recommended order

Order	Lesson	Learning outcome
1	Retrieval boundaries and units	Queries/documents/qrels, relevance grades, and no-answer rules
2	Query and document representations	Sparse/dense input, encoding, and version contracts
3	Similarity, scores, and thresholds	Score direction, candidate budget, thresholds, and deduplication experiments
4	Sparse, dense, and hybrid retrieval	Comparable BM25, dense, and RRF baselines
5	Query processing and filtering	A degradable query pipeline and fail-closed filters
6	Recall and offline evaluation	Stratified queries/qrels, metrics, error classification, and release gates
7	Project: Offline Hybrid Retrieval	A BM25 + toy dense + RRF experiment on a strict fixture

The first two lessons define what to search and how to represent it. Lessons 3–5 build a safe recall pipeline, lesson 6 establishes evidence, and the project reproduces the results.

Hands-on project

• semantic-search-fixture.json: 10 documents, 7 queries, graded qrels, tenant/ACL/status fields, and hand-authored unit vectors.
• toy_semantic_search.py: strict loading, Chinese two-character/BM25 tokenization, exact dense retrieval, RRF, metrics, and a protected_audit safety report.
• test_toy_semantic_search.py: 34 contract, math, retrieval, authorization, evaluation, and CLI tests, including rejection of duplicate rankings and safety audit across the full candidate window.

From the project root (containing docs-CN/, docs-EN/, and .website/), run:

$env:PYTHONDONTWRITEBYTECODE = '1'
python -B -W error '.\docs-EN\semantic-search\examples\toy_semantic_search.py' `
    --fixture '.\docs-EN\semantic-search\examples\semantic-search-fixture.json' `
    --top-k 3 `
    --rank-window 5 `
    --rrf-constant 60

The fixture’s dense vectors are hand-authored one-hot vectors for testing pipeline causality only. They are not an embedding model and cannot demonstrate Chinese semantic quality. Its tenant/ACL fields are teaching data; a real system must separately implement real identities, authorization, revocation, cache invalidation, and audit access control. The project’s protected_audit envelope includes qrels, forbidden-return sets, and the full candidate window, and must not be projected directly to a user.

Mastery checklist

• Can write the online query/document unit, relevance grades, and no-answer rules.
• Query/document representations have model, revision, prefix, truncation, normalization, and metric contracts.
• Can explain why BM25 favors rare matches and reduces excessive term-frequency gain in long documents.
• Sparse, dense, and hybrid retrieval are evaluated separately, and fusion records each route’s rank/score.
• Tenant, ACL, status, and validity take effect before scoring, and an empty identity fails closed.
• Do not treat a cosine threshold, RRF constant, or vendor default as a universal truth.
• The query set covers real frequency, high-risk long tails, hard negatives, and authorization boundaries.
• Report Recall/MRR/nDCG, p95/p99, no-result rate, safety errors, and index freshness.
• A model or query-pipeline upgrade can replay old requests, compare slices, and roll back.

Relationship to other knowledge bases

Knowledge base	Relationship
Chunking Strategies	Determines document/passage boundaries, context, and deduplication units
Embeddings	Provides dense query/document representations and compatibility contracts
Vector Databases	Executes exact/ANN search, payload filters, sharding, and lifecycle
Reranking	Jointly judges query-document relevance on recalled candidates
RAG	Consumes authorized candidates and organizes citations and generation
Evaluation Framework	Connects retrieval metrics and safety gates to end-to-end task quality
Runtime Monitoring	Monitors latency, no-results, channel contribution, drift, and freshness

Primary references

• Robertson & Zaragoza, The Probabilistic Relevance Framework: BM25 and Beyond
• Karpukhin et al., Dense Passage Retrieval
• Cormack, Clarke & Buettcher, Reciprocal Rank Fusion
• Thakur et al., BEIR
• Muennighoff et al., MTEB
• Sentence Transformers: Semantic Search
• Elasticsearch: Reciprocal rank fusion
• Qdrant: Hybrid and Multi-Stage Queries

Sources were obtained/checked on 2026-07-22. Models, SDKs, product APIs, fusion parameters, and defaults change; after locking versions, validate again with local queries and qrels.