3.1 Consistency-based detection
Most methods rest on one observation: contradictory answers across samples often mean the model is fabricating. These methods need no access to model internals.
The catch is cost: each query needs several calls.
Researchers
From dozens of papers to a review framework you can verify
Import papers and lectures in bulk. AI organizes the ideas by research question, method and finding. Every claim traces back to its source, and new papers fold in as you go.
1
Import papers
Bring in papers as PDFs, lecture videos and web pages.
- SurveyPDFProcessing
- ProbingPDFProcessing
- RetrievalPDFProcessing
- LLM talkYouTubeProcessing
- BenchmarksWeb pageProcessing
2
Read closely, annotate, take notes
Bilingual side by side; turn text into a plain version; highlight and annotate; drag passages onto a whiteboard as card notes.
Side by side
大语言模型幻觉检测：综述
Detecting Hallucinations in Large Language Models: A Survey
3.1 基于一致性的检测
3.1 Consistency-based detection
对同一问题多次采样时，若答案相互矛盾，往往说明模型在编造。
When a model is sampled several times on the same question, contradictory answers often indicate fabrication.
这类方法无需访问模型内部，但每次查询都要多次调用模型。
These methods need no access to model internals, but they require multiple calls per query.
3.2 探测内部状态
3.2 Probing internal states
在隐藏层激活上训练的探针，能在答案生成之前识别出多数幻觉。
Probes trained on hidden activations can flag many hallucinations before the answer is generated.
- Survey
- Probing
- Retrieval
- LLM talk
- Benchmarks
Research questions
Detection methods
Drag to the whiteboard to make a card
Self-consistency
contradictory answers across samples often mean the model is fabricating
Surveyp.123
AI organizes across papers
By question, method, finding and debate, into a review whiteboard. Every claim leads back to its source.
5 papers
Organize these 5 papers by question, method, finding and debate, with sources.
Organizing 5 papers
Can we catch them from outside the model?
Surveyp.3
Can we flag them before the answer?
Probingp.2
Self-consistency: sample, then compare
Surveyp.12
Probes on hidden activations
Probingp.5
Retrieval helps, but does not fix it
Retrievalp.9
Bigger models hallucinate less
Benchmarks§2
Is a consistent answer a correct one?
LLM talk38:05Surveyp.18
On the same QA benchmarks, larger models hallucinate less: from 7B to 70B, the rate drops by half.
4
Keep iterating
Add new papers and let AI build them onto what you have. Every change waits for your approval.
3 new papers
Fold the 3 new papers into the review and flag where they disagree with what we have
Folding in 3 new papers
Can we catch them from outside the model?
Surveyp.3
Can we flag them before the answer?
Probingp.2
Self-consistency: sample, then compare
Surveyp.12
Probes on hidden activations
Probingp.5
Added
Probes work across languages too
Multilingual probingp.4
Edited
Retrieval helps, but does not fix it
Retrieval helps, but less on long texts
Retrievalp.9Retrieval revisitedp.3
Deleted
Bigger models hallucinate less
Benchmarks§2
Is a consistent answer a correct one?
LLM talk38:05Surveyp.18
Added
Self-consistency may fail on long texts
Long-context testp.6