Intelligence & Tradecraft
Cognitive biases in intelligence analysis: examples and mitigation
A worked source-independence example, cognitive-bias reference, and downloadable worksheet for checking claims, comparing alternatives, and recording uncertainty.

The short version
- Five reports can repeat one underlying claim. Establish where the information originated before treating repetition as corroboration.
- Record the evidence, assumptions, alternatives, and reasons for confidence separately. An unresolved claim can remain unresolved.
- Structured methods make an assessment easier to inspect. Their accuracy still needs testing; neither a checklist nor an AI assistant guarantees an unbiased result.
Intelligence analysis turns incomplete information into judgments that someone must act on. The work includes national security, but the same problems appear in investigative journalism, business research, and open-source investigation: which information deserves attention, what does it establish, and how much uncertainty remains?
Cognitive biases are recurring tendencies in judgment that can distort those answers. Confirmation bias can shape which evidence we seek. Anchoring can keep an early estimate influential after its basis has weakened. Overconfidence can make the final assessment sound firmer than the evidence warrants.
Those labels are useful starting points. A practical review also needs a specific claim, its source trail, and a record of how the conclusion was reached. The example below supplies that record. The downloadable worksheet lets you repeat the exercise with your own material.
Why recognizing a bias is only a beginning
Objectivity requires a process that other people can inspect. An analyst can sincerely try to be fair while selecting evidence that favors an existing explanation. Nickerson’s review describes confirmation bias as a family of ways that information seeking and interpretation favor beliefs or hypotheses already in hand.1
A source problem also needs its own diagnosis. Five articles based on one announcement are dependent evidence whether or not the analyst has a cognitive bias. Confirmation bias can make that dependence easier to overlook; the underlying defect is still the shared source. Fixing it means tracing the reporting, rather than merely reminding yourself to be open-minded.
Worked example: five reports, one source
Teaching example: Project Alder, every document and every event in this section are fictional. This is an exercise in evidence handling, not a report about a real data center or a measured debiasing experiment.
Suppose you are checking this claim:
Project Alder’s new data center is already operating at full capacity.
Your first search produces five publications. At a glance, several outlets appear to agree. Reading them reveals a much narrower evidence base.
Scroll horizontally
| Record | What the publication contains | Traceable origin | What it independently establishes |
|---|---|---|---|
| R1: operator announcement | Describes an opening ceremony; gives no operating-load measurement | Operator’s own announcement, family F1 | The operator made that announcement |
| R2: newswire item | Repeats R1’s opening claim | R1, family F1 | No additional observation of operations |
| R3: industry article | Summarizes the newswire item | R2 → R1, family F1 | No additional observation of operations |
| R4: local article | Quotes R1; reports no site visit or separate interview | R1, family F1 | No additional observation of operations |
| R5: AI-generated briefing | Cites R2 and R3 as support | R2 and R3 → R1, family F1 | No additional observation of operations |
The five publications belong to one source family for this claim. They may be separate organizations, and their other reporting may be independent. Here, their statements about the opening share an origin. R5’s two citations do not supply two independent checks.
There is a second problem: an opening ceremony does not establish full-capacity operation. Before searching further, define the claim. Does “full capacity” mean all planned halls are commissioned, rated IT power is available, or the machines are running at a particular utilization? Specify the site, measure, and time period. Otherwise two sources can appear to agree while answering different questions.
Revise the assessment before collecting more links
The initial statement, “Five independent reports confirm full-capacity operation,” should become:
The operator announced an opening. All five publications trace that claim to the operator’s announcement. This source family does not establish current full-capacity operation.
This correction does not prove the operator is wrong. It removes two unsupported steps: treating the reports as independent and treating an opening as evidence of full operation.
Now add two further fictional records:
- R6, family F2: a regulator’s register records conditional connection approval. Its published scope does not report actual operating load.
- R7, family F3: a separate inspector’s dated report documents one hall operating during a visit. It does not inspect every planned hall or establish site-wide full-capacity operation.
These add different information. R7 supports limited operation at the time of the inspection. R6 provides regulatory context. Neither warrants silently upgrading the claim to full-capacity operation. The register’s limited scope also means that an absent entry would not, by itself, prove the site was closed.
State what would change the judgment
A useful next step would be a current, independently checkable commissioning record covering the specified halls and measure, or operating data with a clear method and reporting period. Another article repeating R1 would add reach, not a new observation of the relevant activity.
The judgment is therefore full-capacity operation is not established by the available evidence. That is a statement about what this evidence supports, not a probability estimate that the site is or is not operating.
A worksheet you can reuse
Download the blank claim-and-source worksheet or the completed fictional example. Both are plain text so you can copy them into a research note, edit them, or print them.
Use one worksheet per specific claim. A single document can support one part of a claim and leave another unanswered.
Scroll horizontally
| Field | What to record |
|---|---|
| Claim and scope | Exact proposition, entity, time period, and definition of the measure |
| Observation or report | What the source actually records, with a page, passage, timestamp, or other locator |
| Original source and family | Earliest traceable basis for this claim; record “unknown” when you cannot establish it |
| Independence | What information was collected separately, and what evidence supports that conclusion |
| Support and limits | Which part of the claim is supported, challenged, or left unanswered |
| Alternatives and assumptions | Other explanations and the assumptions connecting evidence to your conclusion |
| Judgment and confidence | Your assessment, with reasons grounded in source quality, coverage, and remaining gaps |
| Revision trigger | A specific observation or document that would change the assessment |
| Review record | Reviewer, date, changes made, and any unresolved disagreement |
Do not merge sources merely because they use similar words, or call them independent merely because their domains differ. Trace explicit citations, quoted passages, collection methods, and access to the underlying event. When the trail stops, retain the uncertainty.
ODNI’s Analytic Standards distinguish underlying information, assumptions, and judgments, and require explanations of source quality and uncertainty. They also distinguish the likelihood of an event from confidence in the basis for an assessment. Those are useful distinctions for the worksheet; the form itself is an editorial aid, not an official ODNI template or a validated accuracy score.2
The intelligence analysis cycle in a nutshell

Collection
Decide what evidence the question requires before collecting what is easiest to find. In Project Alder, a search for opening announcements answers a different question from a search for verified operating capacity.
Evaluation
Check what each source could know, how it obtained the information, and which other records depend on it. Source reputation is relevant context; it does not replace checking whether a particular passage supports a particular claim.
Synthesis
Compare plausible explanations, including those the evidence does not resolve. An opening announcement, a conditional permit, and a partial inspection need not contradict one another. They can describe different stages of one project.
Dissemination
Write a conclusion whose scope matches the evidence. Include the important limitations in the assessment itself, where a decision-maker will encounter them, rather than relying on a distant footnote to undo an overconfident headline.
Common cognitive biases in intelligence analysis
The entries below are a reference for examining an assessment. The descriptions of anchoring, availability, and illusory correlation draw on Tversky and Kahneman’s 1974 paper.3 The examples and checking questions are illustrative applications, not findings about every analyst or guarantees that a particular intervention will work.

Confirmation bias
Seeking or interpreting information in ways that favor an existing belief. An analyst expecting a project to succeed might collect launch coverage while neglecting unresolved commissioning conditions. Ask what evidence would count against the current explanation, then actively look for it.
Anchoring bias
An initial value or explanation can influence later judgments even when it has a weak basis. Record where an estimate came from and make a fresh assessment from the relevant evidence before comparing it with the original number.
Availability heuristic
Examples that are easy to recall can dominate judgments of frequency or likelihood. A vivid recent incident is a prompt to investigate, not a reliable estimate of how often the event occurs. Look for a relevant comparison class.
Overconfidence effect
Overconfidence can involve overestimating performance, placing oneself above others, or expressing excessive precision. These are distinct problems. In an assessment, ask whether the stated certainty exceeds the evidence, and whether earlier judgments have been checked against outcomes.4
Groupthink
Pressure toward agreement can suppress alternatives and criticism. A unanimous briefing does not establish that its authors formed independent judgments. Record unresolved objections and the evidence behind them; the Iraq example below shows why agreement itself is an inadequate quality check.
Hindsight bias
Knowing an outcome can make it seem more predictable than it was beforehand. Preserve dated assessments, available evidence, and alternatives before learning the result. Reconstructing those records is more informative than asking why someone failed to see what now looks obvious.5
Attribution bias (fundamental attribution error)
Explaining another person’s conduct through character or intent while giving insufficient weight to the situation. Before inferring an official’s motive, check the constraints, instructions, and information available to that person.6
Selection (sampling) bias
A distorted evidence sample can produce a distorted assessment. This is a property of the collection process as well as a possible consequence of human preferences. Five English-language articles selected from one newswire do not represent five independent views of a multilingual information environment.
Self-serving bias
An analyst or organization may explain success in ways that credit its own judgment while explaining failure through outside conditions. Compare the standards used for both outcomes. Attribution patterns can also have informational explanations; they do not automatically establish deliberate self-protection.7
Status quo bias
Giving the existing option disproportionate preference because it is already the default. Review whether keeping the current assessment follows from new evidence or from the cost and discomfort of changing it. Actual switching costs still deserve consideration.8
Framing effect
Equivalent descriptions can produce different choices. Presenting a result as a gain or a loss can change its appeal without changing the underlying outcomes. Restate the proposition in neutral terms and inspect the absolute quantities and denominator.9
Sunk cost fallacy
Letting an unrecoverable past investment drive a decision that should depend on future costs and benefits. Time already spent building an assessment is not evidence that its conclusion is right. Ask what you would recommend if you were encountering the question today.10
Illusory correlation
Perceiving a relationship more strongly than the evidence supports. Memorable coincidences can suggest a pattern that fails when the full set of cases is examined. Check counterexamples, base rates, and what evidence would distinguish correlation from a proposed causal explanation.
Normalcy bias
This label is often used for treating familiar conditions as a reason to expect continuity. Make the actual assumption explicit: which condition do you expect to persist, and what would reveal a change? A surprise or disaster alone cannot diagnose a particular bias in the people involved. The Pearl Harbor record below illustrates the need to examine the decision and communication chain.
Why biases matter in intelligence analysis
Impact on policy and decision-making
An assessment can influence what gets funded, investigated, protected, or ignored. In the teaching example, a customer considering a capacity contract could act on a claim the source base never established. The appropriate next step depends on the decision’s stakes and what further evidence is obtainable.
Unintended consequences
The confidence conveyed by an assessment can outlive its caveats. Later writers may cite the conclusion without reproducing the original uncertainty, creating another layer of apparently corroborating material. Keep the scope, attribution, and remaining gaps close to the main judgment.
Mitigating cognitive biases: tools and techniques
Structured analytic techniques (SATs)
Heuer’s account of analysis of competing hypotheses encourages an explicit comparison of explanations and attention to inconsistent evidence. A useful question is which observation would distinguish alternatives, rather than how many items can be listed in support of a favored story.11
That recommendation is not proof that a technique improves accuracy. In a 2018 experiment with 50 UK intelligence analysts, Mandel, Karvetski, and Dhami found that using ACH did not improve probability-judgment accuracy on their task. The study does not settle every possible use of structured analysis, but it rules out presenting ACH as a guaranteed debiasing cure.12
Checklists and standard operating procedures (SOPs)
Use a checklist to make specific omissions visible: the claim’s definition, source dependence, contrary evidence, and the assumptions needed for the conclusion. The worksheet here serves that purpose. It has not been tested as an intervention that improves intelligence accuracy, and completing every box cannot establish that the underlying evidence is good.
Collaborative peer review
Give a reviewer the source material and ask for an initial judgment before showing your preferred conclusion where practical. Then compare the evidence used, alternatives considered, and reasons for disagreement. Several people endorsing the same source chain still leave one source chain.
Continual training and awareness programs
Training deserves evaluation rather than automatic dismissal or endorsement. Morewedge and colleagues reported improvements on tested bias measures after single-session game or video interventions in two longitudinal experiments, with effects still present at least two months later. Those results do not establish that reading this guide, or any untested workplace workshop, improves real-world intelligence decisions.13
Data-driven and evidence-based methodologies
For questions with observable outcomes, keep a dated record of the prediction, its stated probability if one was made, and the outcome. Review performance over a suitable set of cases. A correct outcome does not excuse a poorly supported assessment, and a well-supported probabilistic forecast can still be wrong on one occasion.
Personal mindfulness and reflexivity
Use reflection to identify a concrete decision to revisit: an estimate copied from the first briefing, a source excluded without a reason, or a criticism answered with a different standard. Personal sincerity and a feeling of having been careful are difficult for someone else to audit. A recorded source trail and revision history give a reviewer something to inspect.
Real-world examples and case studies
Historical intelligence missteps
The Senate Intelligence Committee’s account of its Iraq inquiry says major judgments in the October 2002 National Intelligence Estimate overstated or lacked support from underlying reporting. It describes a shared presumption about Iraqi WMD programs that shaped the interpretation of ambiguous and contrary information. This is a documented institutional finding, not a basis for attributing every aspect of the war to one psychological mechanism.14

For Pearl Harbor, a more concrete starting point than “normalcy bias caused the surprise” is the surviving Opana radar plot. The U.S. Capitol Visitor Center’s account of the National Archives record describes operators detecting approaching aircraft and an officer interpreting the activity as American planes. It connects the record to the congressional inquiry’s examination of intelligence communication. That supports examining interpretation and communication failures; it does not establish a complete psychological explanation for the attack.15
Corporate and financial analysis
Project Alder offers a deliberately fictional business application: the customer can ask for evidence covering the contracted capacity before signing. It also shows why a source may be accurate about an opening while being insufficient for a different commercial claim. Separate an unsupported inference from a false statement or an accusation of deception.
AI-assisted analysis needs the same source discipline

An assistant can help organize passages, propose alternatives, or turn a source log into a table. Those outputs remain material to check. In Project Alder, the AI briefing adds no independent observation merely by citing two articles derived from the same announcement.
Automation can also introduce errors through reliance on its recommendations. Skitka, Mosier, and Burdick’s simulated-flight research documented participants missing events that an automated aid failed to flag and following erroneous advice despite other available indicators. This was a particular experimental task, not a test of today’s language models, but it supplies evidence for the risk of accepting an aid’s output too readily.16
Citation checking is a separate task. Walters and Wilder’s 2023 study of GPT-3.5 and GPT-4 literature reviews found both fabricated references and errors in real references. Its results describe those models and prompts at that time, not a current failure rate for all assistants.17
For an AI-assisted assessment, open the cited source, locate the supporting passage, check whether it establishes the exact claim, and trace its own sources where relevant. Record the model and date when its output materially affects the analysis. A different model repeating the answer does not, by itself, establish that the underlying evidence was collected independently.
The same discipline applies to a persuasive visual: the companion guide to ethics and deception in data visualization examines how presentation can shape a conclusion. The research on Russian influence in the Balkans provides a separate, sourced regional investigation to read alongside this methodological guide.
Conclusion
A useful bias review changes something inspectable: the evidence collected, the source attribution, the alternatives compared, or the certainty of the published judgment. Remembering a bias’s name can prompt that review. The source trail shows whether the review actually addressed the problem.
Before you circulate an assessment
- Can you state the claim precisely enough to check it?
- Have you traced the relevant reports to their underlying source families?
- Does each cited passage establish the part of the claim assigned to it?
- Have you recorded plausible alternatives, important assumptions, and contrary evidence?
- Does the conclusion communicate what remains unknown and what would change it?
About this revision
First published on 5 January 2025; substantially revised on 16 November 2026. This edition adds the fictional source-independence exercise, blank and completed worksheets, primary references, and narrower claims about historical failures, debiasing, and AI. The established URL, original publication date, and existing illustrations are preserved. The worksheet is an aid to documenting analysis; this article reports no original experiment showing that it improves accuracy.
Raymond S. Nickerson, “Confirmation Bias: A Ubiquitous Phenomenon in Many Guises,” Review of General Psychology 2(2), 175–220 (1998). DOI and publisher record. A review of confirmation-bias mechanisms, not a test of this worksheet. ↩︎
Office of the Director of National Intelligence, Intelligence Community Directive 203: Analytic Standards, sections D.6.e(1)–(4), including technical amendments. Official archived PDF, printed pp. 4–6. These are analytic standards, not experimental evidence that a form improves accuracy. ↩︎
Amos Tversky and Daniel Kahneman, “Judgment under Uncertainty: Heuristics and Biases,” Science 185(4157), 1124–1131 (1974). Paper hosted at Stanford; DOI: 10.1126/science.185.4157.1124. ↩︎
Don A. Moore and Paul J. Healy, “The Trouble With Overconfidence,” Psychological Review 115(2), 502–517 (2008). Author-hosted paper; DOI: 10.1037/0033-295X.115.2.502. ↩︎
Baruch Fischhoff, “Hindsight Is Not Equal to Foresight: The Effect of Outcome Knowledge on Judgment Under Uncertainty,” Journal of Experimental Psychology: Human Perception and Performance 1(3), 288–299 (1975). Publisher record; DOI: 10.1037/0096-1523.1.3.288. ↩︎
Lee Ross, “The Intuitive Psychologist and His Shortcomings: Distortions in the Attribution Process,” Advances in Experimental Social Psychology 10, 173–220 (1977). Paper hosted at MIT; DOI: 10.1016/S0065-2601(08)60357-3. ↩︎
Dale T. Miller and Michael Ross, “Self-Serving Biases in the Attribution of Causality: Fact or Fiction?” Psychological Bulletin 82(2), 213–225 (1975). Author’s institutional record. The review cautions against treating every attributional asymmetry as proof of a motivational explanation. ↩︎
William Samuelson and Richard Zeckhauser, “Status Quo Bias in Decision Making,” Journal of Risk and Uncertainty 1, 7–59 (1988). Paper hosted at MIT; DOI: 10.1007/BF00055564. ↩︎
Amos Tversky and Daniel Kahneman, “The Framing of Decisions and the Psychology of Choice,” Science 211(4481), 453–458 (1981). Paper hosted at Stanford; DOI: 10.1126/science.7455683. ↩︎
Hal R. Arkes and Catherine Blumer, “The Psychology of Sunk Cost,” Organizational Behavior and Human Decision Processes 35(1), 124–140 (1985). Publisher record; DOI: 10.1016/0749-5978(85)90049-4. ↩︎
Richards J. Heuer Jr., Psychology of Intelligence Analysis (CIA Center for the Study of Intelligence, 1999), chapters 8 and 10. Official book and download. Tradecraft guidance; its description of ACH is distinct from evidence about the method’s effectiveness. ↩︎
David R. Mandel, Christopher W. Karvetski, and Mandeep K. Dhami, “Boosting Intelligence Analysts’ Judgment Accuracy: What Works, What Fails?” Judgment and Decision Making 13(6), 607–621 (2018). Open-access original study; DOI: 10.1017/S1930297500006628. The experiment used a particular probability-assessment task; its results are not a universal estimate for all analysis. ↩︎
Carey K. Morewedge and colleagues, “Debiasing Decisions: Improved Decision Making With a Single Training Intervention,” Policy Insights from the Behavioral and Brain Sciences 2(1), 129–140 (2015). Accepted manuscript at City Research Online; DOI: 10.1177/2372732215600886. Training results concern the studied interventions and outcome measures, not this article or worksheet. ↩︎
U.S. Senate Select Committee on Intelligence, Committee Activities, 2003–2004, S. Rpt. 109-360 (16 November 2006), section “Iraq WMD Inquiry,” recounting the 2004 report’s findings. Committee’s account at the Government Publishing Office. This source is the committee’s own account of its inquiry. ↩︎
U.S. Capitol Visitor Center, “Original Radar Plot of Detector Station Opana, Pearl Harbor, Hawaii, December 7, 1941,” using a National Archives congressional record. Artifact and historical account. ↩︎
Linda J. Skitka, Kathleen L. Mosier, and Mark Burdick, “Does Automation Bias Decision-Making?” International Journal of Human-Computer Studies 51(5), 991–1006 (1999). Publisher abstract; DOI: 10.1006/ijhc.1999.0252. ↩︎
William H. Walters and Esther Isabelle Wilder, “Fabrication and Errors in the Bibliographic Citations Generated by ChatGPT,” Scientific Reports 13, 14045 (2023). Open-access original study. The study evaluated GPT-3.5 and GPT-4 literature reviews generated in 2023. ↩︎