Skip to main content

Analysis of Competing Hypotheses: Cutting Bias from OSINT Conclusions

How to apply Heuer's Analysis of Competing Hypotheses to OSINT: the eight steps, the biases it counters, diagnosticity, a fully worked ACH matrix for a copycat web shop attribution question, a scoring script and a template for recording it in a report.

· By UserSearch Team · 14 min read

Disclaimer: This article is for education and for lawful, authorised professional research. Use these methods only where you have a legitimate purpose and a lawful basis, and follow the laws and platform terms that apply to you, including data protection law such as the UK GDPR and EU GDPR. See our Terms of Service.

TL;DR

  • OSINT analysts rarely fail for lack of data. They fail because the first plausible explanation wins and every later finding gets read as support for it.
  • Analysis of Competing Hypotheses (ACH), set out by Richards Heuer for the CIA, forces you to test every reasonable explanation against every piece of evidence and to look for what disproves, not what confirms.
  • The key idea is diagnosticity: evidence that fits every hypothesis tells you nothing, however striking it looks.
  • We walk through a full ACH matrix for an OSINT attribution question (who runs a cluster of copycat web shops), with a short script to score it and a template for recording it in a report.
  • UserSearch Cases and source-attributed results give each evidence row a checkable origin, which is what makes an ACH matrix defensible.

How the First Explanation Takes Over

Analysis of competing hypotheses (ACH) is a structured method for choosing between several possible explanations by testing every piece of evidence against each of them and looking for what disproves, not what confirms. Picture a familiar OSINT moment. You are looking at twelve web shops copying a client's brand. Nine of them share the same analytics ID. It feels like the answer: one operator, twelve shops. From that point on, every new finding slots neatly into the story. Same registrar? One operator. Same shop template? One operator. Photos lifted from the client's site? Obviously.

Except that a popular shop template can ship with the template author's analytics ID baked in, the registrar is the cheapest one in the market, and every copycat copies the brand's photos. Each item "fits", but none of them discriminates between one operator and twenty unrelated ones using the same kit. The analyst has not tested the explanation. They have decorated it.

This is the problem ACH was designed to fix. It is not a clever technique for finding more data. It is a disciplined way of deciding what the data you already have actually shows, and of writing that reasoning down so a reviewer can challenge it.

What Is Analysis of Competing Hypotheses, in Plain Terms?

Analysis of Competing Hypotheses is a structured method for choosing between several possible explanations. Richards J. Heuer, Jr. described it in Psychology of Intelligence Analysis, published by the CIA's Center for the Study of Intelligence in 1999 and free to read online. The Wikipedia summary of ACH is a useful quick primer, but the book's chapter 8 is short and worth reading in full.

Heuer's eight steps, paraphrased for OSINT teams:

  1. Identify the possible hypotheses. Brainstorm with colleagues who see the problem differently.
  2. List the significant evidence and arguments for and against each, including the absence of things you would expect to see.
  3. Build a matrix with hypotheses across the top and evidence down the side, and assess the diagnosticity of each item.
  4. Refine the matrix. Reword or merge hypotheses, and drop evidence that has no diagnostic value.
  5. Draw tentative conclusions by trying to disprove hypotheses rather than prove them.
  6. Test sensitivity. Ask what happens if a few critical items are wrong, misleading or open to another reading.
  7. Report conclusions, discussing the relative likelihood of all the hypotheses, not just the winner.
  8. Identify milestones, the future observations that would show events taking a different course.

Heuer makes a point that matters enormously in open-source work: keep unproven hypotheses alive until they are disproved. An unproven hypothesis simply lacks supporting evidence. A disproved one has evidence against it. Dropping the first kind too early means you never look for the evidence that might support it.

Why Does Structured Attribution Matter Now?

OSINT findings increasingly end up in decisions with consequences: closing accounts, declining suppliers, reporting domains, publishing stories. Look at how large platforms word their own attribution. Google's TAG Bulletin for Q2 2025 describes channels removed as part of coordinated influence operations "linked to" particular countries. That careful phrasing is not timidity. It reflects a judgment about how far the evidence goes, and it is exactly the distinction ACH trains you to make.

UK government guidance has moved the same way. The Professional Head of Intelligence Assessment's guidance on explaining uncertainty in UK intelligence assessment, updated in March 2025, separates how likely a judgment is from how confident the analyst is in it. And the 2025 edition of the Aqua Book, the government's guidance on producing quality analysis, published in July 2025, stresses proportionate assurance and a clear record of assumptions and limitations. ACH gives you both: a likelihood ranking and a written record of why.

AML and financial crime teams are where we see the most demand, because a network attribution can lead to exits and suspicious activity reports. Our page for anti-money laundering teams covers how attributed, dated research supports those decisions.

Three Biases That Distort OSINT Judgments

Heuer's book is mostly about the mind, not the method. Three of the biases he describes do the most damage in open-source research.

Confirmation Bias

Information consistent with what we already believe is processed easily; information that conflicts is overlooked, distorted or explained away. In OSINT this shows up as searching for more of what you found first. Found a shared email domain? You run twenty more searches on that domain and none on the alternative explanation. The fix built into ACH is simple: work across the matrix, one evidence row at a time against every hypothesis, rather than down one column.

Anchoring

Heuer describes anchoring as a natural starting point that acts as a drag on later adjustment, so the final estimate stays closer to the start than it should. The classic OSINT anchor is the client's brief. "We think the former reseller is behind these shops" becomes the starting point, and even strong contrary evidence moves the analyst only a little. Previous reports on the same entity anchor just as hard. Write the hypotheses before you read the old file.

Mirror Imaging

Mirror imaging is filling gaps in your knowledge by assuming the other side would act as you would. In OSINT it sounds like "nobody would reuse the same certificate across unrelated shops" or "a professional group would never make that mistake". Operators of copycat shops, disinformation networks and mule recruitment schemes work under very different constraints from your team. Their shortcuts, budgets and tools are not yours. When a hypothesis rests on "they wouldn't do that", mark it as an assumption and test it.

Two more habits deserve a mention. Satisficing is stopping at the first answer that seems good enough. And the absence of evidence is easy to ignore: Heuer reminds analysts to ask what should be visible if a hypothesis were true, and to treat its absence as evidence.

How Do You Build an ACH Matrix by Hand?

You do not need special software. A spreadsheet works, and so does a CSV file in version control. The CIA's own Tradecraft Primer on structured analytic techniques (2009) sets out ACH alongside the Key Assumptions Check and Quality of Information Check, both of which pair well with it.

Dedicated tools have come and gone. The open-source Open Synthesis platform, built for collaborative ACH, went into maintenance mode in 2024 and was archived in May 2026, and the domain of an older free ACH web tool no longer serves the tool at all. That is a good argument for keeping your matrix in plain files you control.

Scoring Conventions

Mark each cell with how the evidence relates to the hypothesis:

  • CC very consistent, C consistent
  • N neutral or not applicable
  • I inconsistent, II very inconsistent

Then do the thing that feels unnatural: rank hypotheses by how much evidence is against them, not how much is for them. Consistent evidence is cheap; almost any evidence is consistent with several explanations. Inconsistent evidence does the eliminating.

Diagnosticity, the Idea That Does the Work

Heuer's illustration is medical. A high temperature tells a doctor the patient is ill, but it says little about which illness, because it fits so many. In a matrix, a row where every cell reads C is a high temperature. It may be important context, and you should keep it in the report, but it cannot help you choose. Diagnostic evidence is the row where the cells differ, especially where one hypothesis gets an I or II and others do not.

A Small Script to Score the Matrix

This uses only the Python standard library, so there is nothing to install and no API key. Save your matrix as matrix.csv with an evidence column, a weight column (for example 1 for a weak source, 2 for a strong one) and one column per hypothesis.

python3 - <<'EOF'
import csv
score = {"CC": 0, "C": 0, "N": 0, "I": 1, "II": 2}
rows = list(csv.DictReader(open("matrix.csv")))
hyps = [h for h in rows[0] if h.startswith("H")]
totals = {h: 0.0 for h in hyps}
for r in rows:
    marks = {r[h].strip() for h in hyps}
    if marks <= {"C", "CC"} or len(marks) == 1:
        print("non-diagnostic:", r["evidence"])
    for h in hyps:
        totals[h] += score[r[h].strip()] * float(r["weight"])
for h, t in sorted(totals.items(), key=lambda x: x[1]):
    print(f"{h}: weighted inconsistency {t}")
EOF

The score map counts only inconsistency, following Heuer. The first loop flags rows with no diagnostic value; the second totals weighted inconsistency per hypothesis. The lowest total is the least disproved hypothesis, not a proven one. Treat the numbers as a prompt for discussion, never as the conclusion.

Worked Scenario: Who Runs the Copycat Shops?

Context. Wendleby Home Goods (fictional) finds twelve web shops selling what look like its products at 60 to 70 percent below the recommended price. Customers are complaining about goods that never arrive. The brand team asks: who is behind these shops, and is it one group? The research stays on the shops, their infrastructure and public business records.

Hypotheses. After a short brainstorm with a colleague from the fraud team, the analyst records four:

  • H1: one operator group runs all twelve shops.
  • H2: several unrelated operators are using the same off-the-shelf shop kit.
  • H3: a former authorised Wendleby reseller is running unauthorised shops.
  • H4: these are grey-market resellers of genuine stock, not copycats.

Actions. Using a Case in Forensic Mode, the analyst runs Domain Intelligence on all twelve domains, reviews certificates and archived pages, checks shared page resources and searches the company register for the shops' trading names. The matrix, with weights reflecting source quality:

EvidenceWeightH1 one groupH2 shared kitH3 ex-resellerH4 grey market
E1 Same shop template on all twelve2CCCC
E2 Product photos copied from Wendleby's site2CCCC
E3 Same analytics ID on nine shops2CCICI
E4 Four domains on one multi-domain TLS certificate2CCIICI
E5 All registered through one registrar within 72 hours2CICN
E6 Prices far below any plausible trade price1CCNI
E7 Archived "About" page on one shop names a former Wendleby reseller1NICCC
E8 No registered company matches any trading name2CCII
E9 Repeated non-delivery complaints1CCCI

Weighted inconsistency: H1 scores 0, H3 scores 2, H4 scores 8 and H2 scores 9. E1 and E2 are non-diagnostic: they would be true under every hypothesis, however telling they looked on day one.

Sensitivity. Two items carry most of the weight, so the analyst tests them. For E3, a search on the analytics ID shows it appears only on these nine shops, not on hundreds of unrelated sites, so it is unlikely to be a kit default. That keeps E3 against H2. For E7, the analyst notices the "About" text is word for word identical to the former reseller's old website, which is itself in the archive. Copied text is also consistent with H1 or H2 lifting it. E7 is downgraded and H3 now rests on a single weak item.

Outcome. The report's key judgment reads: "It is likely (moderate confidence) that a single operator group runs at least nine of the twelve shops. We cannot currently distinguish whether that group is connected to the former reseller; the only link is copied text, which is also consistent with an unrelated operator." The milestone: if the former reseller's current domains share certificates or analytics IDs with the shops, H3 rises sharply. The brand team acts on the nine linked domains with evidence and parks the reseller question, rather than confronting a former partner on one copied paragraph.

Worked Scenario: Is the Warehouse Flood Video Recent?

Context. A newsroom receives a video said to show flooding at a named retailer's distribution centre this week. The question: does the video show that site, this week?

Hypotheses. H1 genuine, this site, this week. H2 genuine footage of this site from an earlier flood. H3 genuine footage of a different site. H4 edited or synthetic.

Actions and matrix in brief. Signage matching the retailer's current branding is consistent with H1, H2 and H4, and inconsistent with H3: useful, but it does not separate the top two. A reverse image search on key frames finds nothing earlier, which is weak evidence because not everything is indexed. The decisive row is weather. Recent daily observations show no significant rainfall in the area this week, while the Met Office's historic station data shows an exceptionally wet month at the nearest station two winters ago, when local news reported flooding at the same site. That row is strongly inconsistent with H1 and consistent with H2. A Picture search shows no signs of synthetic generation, weighing against H4.

Outcome. The newsroom reports the video as recirculated footage of an earlier flood and says how it reached that view. Without the matrix, the branding match would have felt like confirmation and the story would have run.

Supporting ACH with UserSearch

An ACH matrix is only as defensible as its evidence rows. Each row needs a source, a date and a way for a reviewer to see it. That is where most hand-built matrices fall down: the evidence lives in browser tabs and screenshots, and nobody can tell which search produced which cell.

In UserSearch, a Case in Forensic Mode stores your search history and bookmarks, so every evidence row can cite a stored search. OneScan runs one input across several data sources and merges the results with source attribution, which is what your weight column needs: you grade the source, not the platform. Domain Intelligence, IP Intelligence, Cyber Intelligence through Shodan Modules, Corporate Intelligence and Picture cover most attribution questions from one account and 100+ third-party data sources. Teams can share a Case so a second analyst fills in the matrix independently, which is one of the best defences against anchoring.

SargeBot, our AI research assistant, helps you set a lawful objective, build entity searches and draft a PDF report, with a choice of Claude, GPT or Grok as the model. Use it to speed up collection for each hypothesis, then verify everything it produces; AI output is a lead, like any other source, and it gets a row and a weight like everything else. For the pivots that generate evidence in the first place, our advanced username OSINT guide is a good companion.

Recording ACH in an OSINT Report

A matrix in an appendix is not enough. The report body should tell a reader, in a few lines, which explanations you tested and why the others fell away. We use this block:

ANALYSIS OF COMPETING HYPOTHESES
Question:        [one sentence, as agreed in the brief]
Hypotheses:      H1 ... H2 ... H3 ... H4 ...
Method:          ACH (Heuer); matrix in Appendix C (matrix.csv, SHA-256 ...)
Diagnostic items: E3, E4, E5 (see Appendix C)
Non-diagnostic:  E1, E2 (consistent with all hypotheses)
Result:          Least disproved: H1. H3 not disproved; rests on E7 only.
Sensitivity:     If E3 is a kit default, H2 cannot be excluded.
Key judgment:    [likelihood term] ([confidence]) that ...
Milestones:      [observations that would change the judgment]
Reviewer:        [name], independent matrix completed YYYY-MM-DD

Three rules keep this honest. Report the relative standing of every hypothesis, not just the winner. State which items the conclusion depends on, so a reader knows where to push. And use the UK probability yardstick for likelihood and a separate confidence level, so "likely, moderate confidence" means the same thing in every report your team produces.

Advanced ACH Practices for OSINT Teams

Always Include a Deception Hypothesis

Heuer warns that a well-run deception leaves little evidence, so its absence proves little. In OSINT, planted "About" pages, recycled branding and borrowed analytics IDs are cheap. If the stakes justify it, add a hypothesis that the obvious evidence was placed to mislead, and keep it until something disproves it.

Complete Matrices Independently, Then Compare

Two analysts filling in the same matrix separately, then discussing the cells where they differ, surfaces assumptions faster than any checklist. The disagreements are the valuable part.

Use ACH-Lite Under Time Pressure

For a one-hour verification, skip the weights. Write three hypotheses, list the five strongest items, mark I where something contradicts, and spend your remaining time on the single most diagnostic check.

Revisit When Milestones Arrive

The milestones in step 8 are not decoration. When one of them occurs, reopen the matrix, add the new row and record the change in the report's change log.

Fairness and Accuracy in Attribution Judgments

Attribution judgments can attach serious conclusions to organisations and, sometimes, to people. The UK GDPR's accuracy principle applies to opinions as well as facts; the ICO's guidance on the accuracy principle expects records to make clear when something is an opinion and what it is based on. An ACH section that states hypotheses, evidence and confidence does exactly that.

  • Set a higher evidential bar before attributing activity to a named individual than to infrastructure or an organisation, and keep research within the purpose recorded at the start.
  • Describe people factually and neutrally. "Linked to" and "consistent with" are honest phrases when they reflect the evidence.
  • Preserve the evidence behind each row. The Berkeley Protocol sets out how open-source material should be preserved and verified if it may be relied on later.
  • Treat every search result as a lead to corroborate. Coverage and freshness depend on the source.

Frequently Asked Questions About Analysis of Competing Hypotheses

What is analysis of competing hypotheses?

Analysis of competing hypotheses (ACH) is a structured method for choosing between several possible explanations. Richards J. Heuer, Jr. described it in Psychology of Intelligence Analysis, published by the CIA's Center for the Study of Intelligence in 1999. ACH tests every piece of evidence against every hypothesis and ranks hypotheses by the evidence against them, not the evidence for them.

What are the eight steps of analysis of competing hypotheses?

Heuer's eight steps of analysis of competing hypotheses are: identify the possible hypotheses, list the significant evidence and arguments, build a matrix and assess diagnosticity, refine the matrix, draw tentative conclusions by trying to disprove hypotheses, test sensitivity to critical items, report the relative likelihood of all hypotheses, and identify milestones that would show events taking a different course.

What is diagnosticity in an ACH matrix?

Diagnosticity is how well a piece of evidence helps you choose between hypotheses. Evidence that is consistent with every hypothesis has no diagnostic value, however striking it looks, just as a high temperature shows a patient is ill without showing which illness. Diagnostic evidence is a matrix row where the cells differ, especially where one hypothesis scores inconsistent.

How do you score an ACH matrix?

An ACH matrix marks each cell as very consistent (CC), consistent (C), neutral (N), inconsistent (I) or very inconsistent (II). Following Heuer, analysts count only the inconsistency, optionally weighted by source quality, and rank hypotheses by how much evidence is against them. The lowest total is the least disproved hypothesis, not a proven one, and the numbers prompt discussion.

Which biases does analysis of competing hypotheses help reduce?

Analysis of competing hypotheses helps reduce confirmation bias, where conflicting information is overlooked, anchoring, where a starting point such as a client's brief drags on later judgment, and mirror imaging, where analysts assume others would act as they would. It also counters satisficing, stopping at the first good-enough answer, and the habit of ignoring missing evidence.

What software is used for analysis of competing hypotheses?

Analysis of competing hypotheses needs no special software. A spreadsheet or a CSV file in version control works, and a short Python script using only the standard library can score the matrix. Dedicated ACH tools have come and gone: the open-source Open Synthesis platform was archived in May 2026, which argues for keeping matrices in plain files you control.

Better Attribution Starts with the Hypotheses You Reject

ACH does not make analysis certain. It makes it honest. You write down every reasonable explanation, test each piece of evidence against all of them, discard what does not discriminate, and report the least disproved answer together with what would change it. That is the difference between a judgment a reviewer can challenge and a story they have to take on trust.

Stop guessing. Start researching with UserSearch. One account gives your team 100+ third-party data sources for the evidence rows, OneScan with attribution on every result for your weights, Cases that store each search behind each cell, shared Cases so a second analyst can build an independent matrix, and SargeBot to speed up collection and draft the report you then verify and sign.

About the author

UserSearch Team
Updated on Nov 18, 2026