← All cheatsheets

AI CLI

RAG & Embeddings — retrieval-augmented generation that works

Ground LLM answers in your own data: chunk, embed, store in a vector DB, retrieve top-k by similarity, and generate cited answers that resist hallucination.

Docs-checkedragembeddingsvector-searchretrievalllm

Quick reference — reach for these first

Chunksplit(DOC, size=512, overlap=64)Break docs into passages small enough to embed and retrieve cleanly; overlap preserves cross-boundary context.
Embedembeddings.create(MODEL, input=CHUNK)Map each chunk to a dense vector. Use the SAME model for documents and queries, always.
Storepgvector / FAISS / managed indexPersist vectors + metadata in a vector store with an ANN index for fast cosine similarity search.
Retrievetopk = index.search(embed(Q), k=5)Rank candidates by cosine similarity; add hybrid keyword search and a reranker to lift precision.
GenerateLLM(SYSTEM + context + Q) -> [n] answerAnswer ONLY from retrieved passages, each claim cited by passage number, within the context budget.

RAG pipeline: index offline, retrieve and cite at query time

RAG pipelineOffline, documents are chunked, embedded, and written to a vector database. At query time the question is embedded, the vector store returns the top-k most similar chunks, those passages are placed in the prompt, and the LLM returns a cited answer.Indexing (offline)Query timeDocsChunkEmbedVector DBtop-k ANNQueryEmbedVector SearchTop-kPrompt + ContextLLMCited Answer [n]

Text equivalent: RAG pipeline. Offline indexing row: Docs to Chunk to Embed to Vector DB. Query-time row: Query to Embed to Vector Search, which pulls the top-k most similar chunks from the Vector DB, to Top-k to Prompt plus Context to LLM to a Cited Answer with bracketed citations.

Common tasks

Task · Command / query · Check

TaskCommand / queryCheck
Enable the pgvector extension [prod: DDL]CREATE EXTENSION IF NOT EXISTS vector;\dx lists 'vector' among installed extensions
Create a table with a vector columnCREATE TABLE docs (id bigserial PRIMARY KEY, content text, content_hash text, embedding vector(1536));\d docs shows an 'embedding' column of type vector(1536)
Build an ANN index for cosine distance [prod: DDL]CREATE INDEX ON docs USING hnsw (embedding vector_cosine_ops);EXPLAIN on a search query shows an Index Scan, not a Seq Scan
Chunk a document (size + overlap)chunks = split_text(DOC, chunk_size=512, chunk_overlap=64)len(chunks) > 1 and adjacent chunks overlap by ~64 tokens
Embed chunks in one batch callvecs = client.embeddings.create(model='text-embedding-3-small', input=CHUNKS).datalen(vecs) == len(CHUNKS) and len(vecs[0].embedding) == 1536
Insert chunk + embedding rowsINSERT INTO docs (content, content_hash, embedding) VALUES (:CHUNK, :HASH, :EMBEDDING);SELECT count(*) FROM docs equals the number of chunks
Retrieve top-k by cosine similaritySELECT id, content FROM docs ORDER BY embedding <=> :QUERY_VEC LIMIT 5;Returns 5 rows; the known-relevant chunk is among them
Hybrid search (keyword + vector)SELECT id FROM docs WHERE to_tsvector(content) @@ plainto_tsquery(:Q) ORDER BY embedding <=> :QUERY_VEC LIMIT 20;Recall on exact-term/rare-keyword queries beats vector-only
Rerank the candidate setranked = rerank(model=RERANKER, query=Q, docs=CANDIDATES, top_n=5)The gold chunk moves into the top 3 after reranking
Assemble a grounded promptprompt = '\n'.join(f'[{i}] {c}' for i,c in enumerate(TOPK,1)) + '\n\nQUESTION: ' + QPrompt contains numbered passages and fits the context budget
Generate a cited answer (Claude Opus 4.x / Sonnet)msg = client.messages.create(model=MODEL, system=SYSTEM, messages=[{'role':'user','content':prompt}])Answer cites passages like [1] and adds no uncited claims
Detect stale rows and re-embed on change [prod: rewrites rows]UPDATE docs SET embedding = :NEW_VEC, content_hash = :SRC_HASH WHERE content_hash <> :SRC_HASH;Edited sources get fresh vectors; hashes match after the run
Measure retrieval hit-rate @khit = sum(gold in topk(embed(q), k=5) for q,gold in LABELED) / len(LABELED)hit@5 printed as a fraction on the labeled set (aim > 0.8)
Score answer faithfulness with an LLM judgescore = judge(MODEL, question=Q, answer=A, context=TOPK) # 1 if every claim is supportedGroundedness near 1.0; flagged rows contain uncited claims

Minimal grounded RAG query with citations

AI

1
Problem: answer a question using ONLY retrieved chunks, and cite them — no outside knowledge.
2
Copy-ready system prompt (the grounding contract)
You are a retrieval QA assistant. Answer the QUESTION using ONLY the numbered CONTEXT passages. Cite every claim with its passage number like [1]. If the answer is not in the context, reply exactly: 'Not in the provided context.' Never use outside knowledge.
3
Sample input: the question plus the top-2 retrieved chunks
QUESTION: What is the default dimensionality of OpenAI's text-embedding-3-small?
CONTEXT:
[1] text-embedding-3-small returns 1536-dimensional vectors by default; the dimensions parameter can shorten them.
[2] Cosine similarity is the recommended distance metric for OpenAI embeddings.
4
Generate with a current Claude model (Opus 4.x / Sonnet)
client.messages.create(model=MODEL, system=SYSTEM, messages=[{'role':'user','content':Q_PLUS_CONTEXT}])
5
Expected grounded answer — one cited claim, nothing uncited
6
Negative control: ask something absent from the context
7
Evaluate — faithfulness + retrieval hit-rate on a small labeled set
faithful = every_claim_cited_and_supported(answer, context)   # LLM judge -> 1/0
hit_at_5 = mean(gold_id in retrieve(q, k=5) for q, gold_id in LABELED)
8
Verify: faithfulness is 1 on the sample and hit@5 clears your bar (e.g. > 0.8). If hit@5 is low, retrieval is the bottleneck — fix chunking/embedding/k before touching the prompt.

Troubleshooting — symptom → first check → next check

SymptomFirst checkEstablishesNext check
Retrieval misses the obviously relevant chunkRe-run the query with a larger k and print the raw similarity scoresWhether it is a ranking/k problem or the chunk was never indexed wellLower chunk size / add overlap, confirm query and docs use the SAME embedding model, raise k, then add a reranker
Answer ignores the retrieved context and uses model priorsLog the final assembled prompt and confirm the passages are actually in itWhether context was truncated out or the grounding instruction is too weakAdd a strict 'answer ONLY from context' instruction, put passages near the question, and trim to fit the window
Answer cites a source that does not support the claim (hallucinated citation)Map each [n] back to its passage and verify the claim is present thereWhether the model fabricated support or retrieval returned the wrong chunkEnforce cite-from-context in the prompt, run a faithfulness judge in CI, and fail the answer on any uncited claim
Good answers yesterday, wrong or outdated answers todayCompare the source content_hash to the stored row's hash / updated_atWhether the index is stale relative to the source of truthRe-embed changed records on write (or on a schedule) and reindex; never edit a source without re-embedding it
Prompt exceeds the context window or token cost spikesCount tokens of system + passages + question before the callWhether too many or too-large chunks are being stuffedRerank and keep only the top 3-5 passages, shrink chunk size, or summarize passages before stuffing
Similarity scores are all near-identical or meaninglessConfirm query and documents were embedded by the same model and the index metric matches (cosine vs L2)Whether a model/metric mismatch is corrupting the rankingUse one embedding model everywhere, normalize vectors, and match the index op-class (vector_cosine_ops) to your distance

References

Examples use pgvector + OpenAI text-embedding-3-small (1536 dims, cosine) and a current Claude model (Opus 4.x / Sonnet); swap in FAISS or a managed index and your own embedding model as needed. The query and documents must always be embedded by the same model.