Lesson 3 of 6
Retrieve the Most Relevant Chunks
Rank chunks by cosine similarity to a question and select the top few to provide context.
7 minAI Engineering
After this lesson, you will be able to turn a question into a vector, compare it with the vectors of all chunks, and pick the chunks most similar to the question. You will use cosine similarity, a common way to measure how alike two vectors are.
From chunks to vectors
In the previous lessons, you split a document into chunks and turned each chunk into a vector of character trigram counts. Now you need a way to compare a question with those chunks. The question is also text, so you can turn it into a vector using the same vocabulary. Then you measure how similar the question vector is to each chunk vector.
You already have a file called vectorizer.py with the functions make_trigrams, build_vocabulary, and chunk_to_vector. You will reuse them here. If you have not done the exercise from lesson 2, create vectorizer.py now with this code:
import re
def make_trigrams(text):
text = ' ' + text.lower() + ' '
return [text[i:i+3] for i in range(len(text)-2)]
def build_vocabulary(chunks):
vocab = set()
for chunk in chunks:
trigrams = make_trigrams(chunk)
vocab.update(trigrams)
return sorted(vocab)
def chunk_to_vector(chunk, vocab):
trigrams = make_trigrams(chunk)
counts = {}
for tri in trigrams:
counts[tri] = counts.get(tri, 0) + 1
vector = []
for tri in vocab:
vector.append(counts.get(tri, 0))
return vectorCosine similarity
Cosine similarity measures the angle between two vectors. If the vectors point in the same direction, the similarity is 1. If they are perpendicular, the similarity is 0. If they point in opposite directions, the similarity is -1. For our vectors, which contain only non-negative counts, the similarity will be between 0 and 1.
The formula for cosine similarity between two vectors A and B is:
similarity = (A dot B) / (||A|| * ||B||)Here A dot B is the sum of the products of corresponding entries. ||A|| is the length of vector A, calculated as the square root of the sum of the squares of its entries.
You can write a function to compute cosine similarity in Python. Create a new file called retriever.py:
import math
def cosine_similarity(vec_a, vec_b):
dot = sum(a * b for a, b in zip(vec_a, vec_b))
norm_a = math.sqrt(sum(a * a for a in vec_a))
norm_b = math.sqrt(sum(b * b for b in vec_b))
if norm_a == 0 or norm_b == 0:
return 0.0
return dot / (norm_a * norm_b)The function takes two vectors of the same length. It computes the dot product, then the norms, and returns the similarity. If either vector is all zeros, it returns 0 to avoid division by zero.
Turn a question into a vector
To compare a question with chunks, you need to turn the question into a vector using the same vocabulary as the chunks. Add this function to retriever.py:
from vectorizer import make_trigrams, chunk_to_vector
def question_to_vector(question, vocab):
return chunk_to_vector(question, vocab)This function simply calls chunk_to_vector with the question text. The vocabulary must be the one built from the document chunks.
Rank chunks by similarity
Now you can compute the similarity between the question vector and each chunk vector. You will store the similarity along with the chunk text and its index, then sort them in descending order. Add this function to retriever.py:
def retrieve_top_chunks(question, chunks, vocab, top_k=3):
q_vec = question_to_vector(question, vocab)
scored = []
for i, chunk in enumerate(chunks):
c_vec = chunk_to_vector(chunk, vocab)
sim = cosine_similarity(q_vec, c_vec)
scored.append((sim, i, chunk))
scored.sort(reverse=True, key=lambda x: x[0])
return scored[:top_k]The function takes the question, the list of chunks, the vocabulary, and an optional top_k (default 3). It builds the question vector, then loops over each chunk, computes its vector, and calculates the cosine similarity. It stores a tuple with the similarity, the chunk index, and the chunk text. After sorting in descending order by similarity, it returns the top top_k tuples.
Test the retriever
Create a small script to test the retriever. You can use the same sample text from lesson 1. Create a file called test_retriever.py:
from vectorizer import build_vocabulary
from retriever import retrieve_top_chunks
def read_file(file_path):
with open(file_path, 'r', encoding='utf-8') as f:
return f.read()
def chunk_words(words, chunk_size, overlap):
chunks = []
start = 0
while start < len(words):
end = start + chunk_size
chunk = words[start:end]
chunks.append(' '.join(chunk))
start += chunk_size - overlap
return chunks
def main():
text = read_file('sample.txt')
words = text.split()
chunks = chunk_words(words, 10, 5)
vocab = build_vocabulary(chunks)
question = 'What is retrieval-augmented generation?'
top = retrieve_top_chunks(question, chunks, vocab, top_k=2)
for sim, idx, chunk in top:
print(f'Similarity: {sim:.4f}')
print(f'Chunk {idx+1}: {chunk}')
print()
if __name__ == '__main__':
main()This script reads sample.txt, chunks it with size 10 and overlap 5, builds the vocabulary, and retrieves the top 2 chunks for a sample question. Run it:
python test_retriever.pyYou will see the similarity scores and the chunk texts. The chunks most similar to the question should appear first. The exact scores depend on the trigrams in the text, but the order is what matters.
Choosing the number of chunks
The parameter top_k controls how many chunks you pass to the generator later. A small number keeps the prompt short, but you might miss relevant information. A large number gives more context but may include noise. In the next lessons, you will use the retrieved chunks to build a prompt. For now, top_k=3 is a reasonable default.
You have now built the retriever. In the next lesson, you will combine the question and the retrieved chunks into a prompt for a language model.
Key takeaways
- Cosine similarity measures the angle between two vectors; for non-negative vectors it ranges from 0 to 1.
- Turn the question into a vector using the same vocabulary as the chunks.
- Rank chunks by cosine similarity and select the top few.
- The number of chunks to retrieve is a parameter you can tune.
Try it
Write a Python script that reads a plain-text file of your choice, splits it into chunks of 50 words with 10 words overlap, builds the vocabulary, and retrieves the top 3 chunks for a question you make up. Print the similarity score and the chunk text for each. Then, change the question to something unrelated to the document and run again. A good result shows that the top chunks for a relevant question have higher similarity scores than those for an unrelated question, and the chunks are printed in descending order of similarity.
Official documentation
Written with AI and checked automatically. Tools change, so a command may behave differently in the version you install. Spotted a mistake? Tell us.