A voice diagnostic that probes past your prepared answers, then hands you one thing to study.
LLM Engineer diagnostic
Gap map ready
Study next
Measuring retrieval on its own
Coverage
3 of 6 areas checked
Coverage: 3 of 6 areas checked
Study next: Measuring retrieval on its own
Watch it find a gap
Illustrative sampleA RAG assistant is citing the wrong document. Four moments from a sample session — the third one is the product.
The full sample session, start to finish.
Veda
Your RAG assistant keeps giving answers that sound right but cite the wrong document. Where do you start?
Step 01
Veda chooses the concept, not you.
The opening question is selected from the benchmark and from what the diagnostic has no evidence for yet — not from a topic you nominated.
Candidate
I'd tighten the prompt first — tell it to answer only from the provided context and say it does not know otherwise. Then a stronger model, and a re-ranker on top if it is still drifting.
Step 02
Correct, confident — and not an answer.
Every fix named here is a real fix. None of them checks whether the right document ever reached the model. Most self-review stops here, because nothing said was wrong.
Veda
Those all change what the model does with the context. Before that — how do you know the right document was in the context at all?
Candidate
I'd read a few outputs and see if they got better. I have not measured retrieval on its own — I have only ever looked at the final answer.
Step 03
You never separated retrieval from generation. The diagnostic did.
Veda notices the answer jumped straight to what the model does with the context, and follows it back one step: whether retrieval is ever measured on its own.
LLM Engineer
Preliminary picture formingRAG failure modes
Named real fixes — grounding instructions, a re-ranker, a stronger model.
Evaluating retrieval separately
Judged the system by its final answer only, with no measurement of what retrieval actually returned.
Chunking and embeddings
No evidence collected in this area yet.
Based on your session
Named three generation-side fixes, then said retrieval had never been measured on its own.
The gap comes from this answer and nothing else. Confidence stays preliminary because one follow-up is thin evidence for a concept this broad.
Study this next
Measuring retrieval before you tune the prompt
Score retrieval on its own — did the right chunk come back, and at what rank — so a wrong answer tells you whether the problem is retrieval or generation.
Step 04
One gap, traced back to what was said.
Retrieval and generation fail for different reasons and are fixed in different places, so they have to be measured apart. That is the concept the session surfaced, and the only one it recommends.
This walkthrough uses a synthetic session written to show how the diagnostic behaves. It is not a real user result.
Not a score, not a percentile, not a ranking against other candidates. These three artifacts, on a page you keep.
Coverage
3 of 6 areas checked
Evaluating retrieval separately
Based on your session
Named three generation-side fixes, then said retrieval had never been measured on its own.
One next action
Measuring retrieval before you tune the prompt
Roughly 40 minutes of reading.
Nothing else is recommended. That is the whole output.
Nothing to install. Talk it through, keep the map.
One diagnostic. One gap map. One next step.
Start free diagnostic15 free minutes every week