Skip to content

Lesson 4 of 6

Build a Prompt with Retrieved Context

Combine the question and the retrieved chunks into a clear instruction for a language model.

5 minAI Engineering

After this lesson, you will be able to combine a question and the retrieved chunks into a clear prompt for a language model.

Why a prompt matters

A language model needs a well-formed instruction to produce a useful answer. If you just pass the question, the model may guess or make things up. If you pass the retrieved chunks, the model can base its answer on them. A prompt is the text you send to the model. It usually has three parts: an instruction, the context, and the question.

The instruction tells the model what to do. The context is the retrieved chunks, the passages that are likely to be relevant. The question is what the user wants to know. Together they form a prompt.

Build a simple prompt

Start with a function that takes a question and a list of retrieved chunks. Each retrieved chunk is a tuple of similarity score, index, and chunk text, as returned by retrieve_top_chunks from the earlier lesson.

def build_prompt(question, retrieved_chunks):
    instruction = 'Answer the question based only on the following context.\n\n'
    context = '\n\n'.join(chunk for _, _, chunk in retrieved_chunks)
    prompt = instruction + 'Context:\n' + context + '\n\nQuestion: ' + question + '\nAnswer:'
    return prompt

This function builds a prompt with a clear instruction, the context, and the question. The instruction says to answer based only on the context, which helps the model stay grounded. The context is the retrieved chunks joined by blank lines. The question is placed at the end, followed by Answer: to signal where the model should start.

Test it with a small example. Create a list of two fake retrieved chunks and a question.

retrieved = [
    (0.9, 0, 'Retrieval-augmented generation combines a retriever and a generator.'),
    (0.8, 1, 'The retriever finds relevant passages from a document.')
]
question = 'What is retrieval-augmented generation?'
prompt = build_prompt(question, retrieved)
print(prompt)

The output shows the instruction, the two chunks, and the question. This is the prompt you would send to a language model.

Add chunk labels

When you have several chunks, it helps to label them. The model can then refer to specific chunks in its answer. Modify the function to include a label for each chunk, such as [1] and [2].

def build_prompt(question, retrieved_chunks):
    instruction = 'Answer the question based only on the following context.\n\n'
    context_parts = []
    for i, (_, _, chunk) in enumerate(retrieved_chunks):
        context_parts.append(f'[{i+1}] {chunk}')
    context = '\n\n'.join(context_parts)
    prompt = instruction + 'Context:\n' + context + '\n\nQuestion: ' + question + '\nAnswer:'
    return prompt

Now each chunk is prefixed with a number in square brackets. This makes it easier for the model to cite sources.

Use the prompt in the pipeline

You can integrate this into the script from the earlier lessons. The script reads a document, splits it into chunks, builds the vocabulary, retrieves the top chunks for a question, and now builds a prompt.

Create a new file called prompt_builder.py with the build_prompt function. Then write a script that uses it.

from vectorizer import build_vocabulary
from retriever import retrieve_top_chunks
from prompt_builder import build_prompt

def read_file(file_path):
    with open(file_path, 'r', encoding='utf-8') as f:
        return f.read()

def chunk_words(words, chunk_size, overlap):
    chunks = []
    start = 0
    while start < len(words):
        end = start + chunk_size
        chunk = words[start:end]
        chunks.append(' '.join(chunk))
        start += chunk_size - overlap
    return chunks

def main():
    text = read_file('sample.txt')
    words = text.split()
    chunks = chunk_words(words, 10, 5)
    vocab = build_vocabulary(chunks)
    question = 'What is retrieval-augmented generation?'
    top = retrieve_top_chunks(question, chunks, vocab, top_k=2)
    prompt = build_prompt(question, top)
    print(prompt)

if __name__ == '__main__':
    main()

Save this as build_prompt_demo.py and run it. You will see the prompt printed. This is the text that the next lesson will pass to a placeholder language model.

Keep the prompt flexible

Different models may need different prompt formats. For example, some models expect a system message and a user message. You can make the instruction a parameter so you can change it without rewriting the function.

def build_prompt(question, retrieved_chunks, instruction='Answer the question based only on the following context.\n\n'):
    context_parts = []
    for i, (_, _, chunk) in enumerate(retrieved_chunks):
        context_parts.append(f'[{i+1}] {chunk}')
    context = '\n\n'.join(context_parts)
    prompt = instruction + 'Context:\n' + context + '\n\nQuestion: ' + question + '\nAnswer:'
    return prompt

Now you can pass a different instruction if needed. The default works for many simple models.

You have built a prompt that combines the question and the retrieved chunks. This prompt is ready to be sent to a language model.

Key takeaways

  • A prompt has an instruction, context, and question.
  • The instruction tells the model to answer based only on the context.
  • Label each chunk with a number so the model can refer to it.
  • Keep the instruction as a parameter for flexibility.

Try it

Write a Python script that reads a plain-text file, splits it into chunks of 80 words with 20 words overlap, builds the vocabulary, retrieves the top 3 chunks for a question you make up, and builds a prompt using the build_prompt function from this lesson. Print the prompt. Then, change the instruction to 'Use the context to answer the question. If the context does not contain the answer, say so.' and print the new prompt. A good result shows two prompts: the first with the default instruction, the second with your custom instruction, both containing the same context and question.

Official documentation

Ready to use this in a job?

Rolebound applies to matching roles for you.

Start now

Written with AI and checked automatically. Tools change, so a command may behave differently in the version you install. Spotted a mistake? Tell us.

    Cookies on Rolebound

    We use necessary cookies to sign you in and keep the site working — those cannot be turned off. We would also like to set analytics and advertising cookies so we can see which pages help people get interviews and which adverts brought them here. Nothing optional is set unless you say yes, and you can change your mind at any time from “Cookie settings” in the footer. Read our Privacy Policy.

    Build a Prompt with Retrieved Context | Build a Retrieval-Augmented Generation System in Python | Rolebound