Methodology v1.7.1 · September 2026

How Verum Signal works.

Documented. Versioned. Same standards applied to every outlet. Built to answer the questions skeptics actually ask.

METHODOLOGY v1.7.1
The pipeline at a glance
Every article runs the same pipeline.
Two rules govern independence.
Eight verdicts. Six scoreable.
Outlets earn their tier by volume.
01
Ingest
URL → text
language · source ID
02
Extract
claims via Sonnet
up to 7 per article
03
Priority
score ≥ 30
filters lifestyle / fluff
04
Verify
opinion filter
→ web search
Breaking-news gate
6 hours before a claim affects an outlet's score
Wire reprints
Scored as an ordinary outlet claim — the outlet selected and published it
Quoted speakers
Scored on quote accuracy, not the speaker's claim
Claim · April 2026
"The Senate passed the Western Water Security Act 61–38."
Sentinel Pressvia Reuters
Beacon Dailyvia Reuters
Northwind Newsvia Reuters
Crosswirevia Reuters
Citywirevia Reuters
Independence ruleFive outlets cite the same wire — counted as one source, not five.
= 1 source
Consensus exception5+ outlets reporting consistently → claim assigned at reduced confidence.
+0.75 weight
Outlet · all-time tally
Supported128× +1.0
Plausible42× +0.5
Corroborated31× +0.75
Overstated18× −0.5
Disputed9× −1.0
Not supported4× −1.5
Opinion37excluded
Not verifiable22excluded
weighted_sum = Σ (weight × count)
scoreable = total − (opinion + not_verifiable)
score = (sum / scoreable + 1.5) / 2.5 × 100
86/100
Published100+ verdicts
142
Stabilizing50–99
59
Limited Data20–49
31
Excludedunder 20
12
Inclusion threshold · 20 verdicts
Excluded outlets still analysed
01
The skeptic asks
"How do I know you're not just confirming what you already think?"

The independence rule does the work.

Two sources only count as independent if they obtained their information through genuinely different means. Five outlets republishing the same Reuters report counts as one source, not five. This is how the system avoids confusing reach with corroboration.

Outlet claims vs attributed claims.

A claim made directly by the outlet is scored against the outlet. A claim attributed to a named speaker — a politician, an official, a CEO — is still analyzed and still receives a verdict, but that verdict does not affect anyone's score. It does not count against the outlet, which was reporting rather than asserting, and we do not maintain a score for any speaker. The outlet is evaluated on one thing only: whether it quoted accurately.

Worked example · hypothetical claim
Sentinel Press · April 2026
"The Senate passed the Western Water Security Act 61 to 38, allocating $4.2 billion over seven years to permanently resolve interstate water disputes."
Step 01
Ingestion
Article fetched. Source identified. Publication timestamp parsed. Article is older than the 6-hour gate, so any verdicts produced will affect the outlet's live score.
Step 02
Extraction
Three checkable factual claims extracted from the sentence: (1) the Senate vote count, (2) the appropriation amount and duration, (3) the framing claim that the bill will "permanently resolve" disputes.
Step 03
Priority
Each claim scored against the priority threshold. All three score above 30 and enter the verification queue.
Step 04
Verification
Vote count: confirmed by Senate.gov roll-call record and AP wire reporting. Independent sources. → Supported · +1.0
Appropriation: confirmed by the Congressional Budget Office summary and the bill text. → Supported · +1.0
"Permanently resolve" framing: water-rights legal scholars in the cited reporting describe the bill as a 7-year settlement, not a permanent one. → Overstated · −0.5
Result
Two supported claims, one overstated. Sources cited in the published report. The independence rule was applied at verification — the AP wire and Senate.gov are independent.
Article: 78/100
02
The skeptic asks
"What if the AI gets it wrong?"

Same prompt. Same standards. Every outlet.

Every verdict is generated automatically using the same prompt and the same standards, applied identically to every outlet. No human editorial decisions sit between an article and its verdict.

That means a verdict is reproducible: re-running the same article through the same methodology produces the same result. It also means there's no editor with a thumb on the scale for any outlet — favorable or unfavorable.

6-hour gate
Claims from articles less than 6 hours old don't yet affect outlet scores. They need time for corroborating sources to surface. The gate is enforced uniformly across every scoring surface — leaderboard, outlet pages, score history, article badges.
Opinion excluded
"Is this accurate?" is the wrong question for editorial commentary. Opinion content is excluded from outlet scoring — not because opinion is bad, but because reliability is the wrong axis to judge it on.
Not verifiable
Absence of evidence is not evidence of inaccuracy. When a claim cannot be confirmed or denied — sources unavailable, paywalled, empirically untestable — it's marked Not verifiable and excluded from scoring rather than counted against the outlet.
Wire reprints
Verbatim wire-service content published under an outlet's own byline is scored as an ordinary outlet claim. The outlet selected it, headlined it, and published it under its masthead — an editorial decision outlets are accountable for.
Quoted speakers
When an outlet reports what a politician said, the verdict evaluates whether the outlet quoted the politician accurately — not whether the politician's underlying claim is true. The politician's claim is scored against the politician's record, not the outlet's.
03
The skeptic asks
"What's your political bias?"

The methodology doesn't know what political affiliation an outlet has.

The pipeline ingests an article, extracts factual claims, runs each claim against the same evidence-finding process, and assigns a verdict from the same eight categories. Every outlet runs the same code path. There is no dial labeled "left" or "right." There is no editor who decides which outlets get scrutiny.

What we never call ourselves.

Verum Signal never describes itself as a fact-checker. Fact-checking is a craft — one that involves human judgment, editorial framing, and choices about what to cover. Verum Signal does something narrower: it evaluates claims on evidence using a documented methodology, applied uniformly. We provide the signals. You decide.

What the leaderboard reflects.

Outlet scores are a function of the verdicts that outlet has accumulated under this methodology. They are not curated. They are not adjusted for editorial reasons. They change as more verdicts are accumulated, and the trajectory of any outlet's score is itself public.

04
The skeptic asks
"What changed and when?"

The methodology is versioned. Changes are documented. Old versions are archived.

Methodology versions are bumped when scoring math, verdict definitions, or inclusion rules change. Substantive changes are documented. Archived versions remain available and continue to be browsable.

v1.7.1
September 2026
Current
  • Speaker Attribution (Debates) added as Section 2.6: acoustic voice confirmation against enrolled candidate prints; until a voice is confirmed, its lines carry no speaker or a proposed speaker marked as uncertain, never a confirmed one; and a post-event check that corrects attributions after the debate, with debate ratings provisional until it has run.
  • Quoted claims never count toward an outlet's score, and a quoted claim's check is of the underlying assertion. Since 25 September 2026 their ratings are not published: a quoted claim is shown as Quoted · Not rated.
  • Verdicts Can Change (7.5) distinguishes a correction (verdict reset, claim re-checked) from a repair (records aligned, verdict kept); both propagate to every surface within one refresh cycle.
  • The date requirement and 6-hour gate are applied identically wherever a score is published — web, mobile and API.
v1.7
July 2026
Archived
  • Verification pipeline rewritten to match the live engine: an opinion pre-filter, then web-search verification. The cache and consensus steps described in v1.6 were removed from the engine in May.
  • Corroborated weight corrected from +0.5 to +0.75, so corroborated and plausible are no longer tied.
  • The 30-point priority threshold documented as governing the background queue only; on-demand reports check claims regardless of priority.
  • Wire-reprint exclusion retired: wire content published under an outlet's own byline is scored as an ordinary outlet claim.
v1.6
May 5, 2026
Archived
  • Single canonical module for scoring constants — verdict weights, scoreable verdict types, tier thresholds, formula divisor — consolidated to one module.
  • The 6-hour breaking-news gate is now applied uniformly across every scoring surface (leaderboard, outlet aggregates, score history, article badges, individual outlet endpoints).
  • Articles with no scoreable claims now display as "Unscored" instead of receiving a misleading 0/100 score.
  • Extraction prompt rewritten to be genre-agnostic — opinion pieces and analysis can contain checkable factual claims, and aren't skipped because of their genre label.
  • Paid extraction depth raised from 3 to 7 claims per article. Free reports continue to verify the top 2 claims.
  • Brand language aligned: user-facing strings consistently use "verification engine" and "claim analysis platform" framing.
  • Known data limitation acknowledged: a subset of ingested articles arrive without a parseable publication timestamp and are excluded from outlet scoring under the date-required policy. The ingestion pipeline is being improved to reduce this exclusion rate.
v1.5
retrospective entries
Archived
  • Eight verdict types defined (six scoreable, two excluded).
  • Score formula established: (weighted_sum / scoreable + 1.5) / 2.5 × 100.
  • Public leaderboard launched.
  • 6-hour breaking-news gate defined.
Read the archived v1.7 methodology
05
The skeptic asks
"What can't you do?"

The honest limitations of this approach.

A methodology should be honest about what it can't do. Some of what follows is by design; some is a function of where the system is in its lifecycle. Either way, it's worth saying out loud.

Limited outlet coverage today.

The leaderboard is still maturing. Outlets need at least 20 verdicts to receive a public score, and many outlets are below that threshold. Coverage will expand as the corpus grows.

Paywalled sources can limit verification.

When the only credible source for a claim sits behind a paywall the system cannot access, the claim may land in Not verifiable — excluded from scoring rather than counted against the outlet.

Articles missing publication timestamps are excluded.

A subset of ingested articles arrive without a parseable publication date and are excluded from outlet scoring under the date-required policy. The ingestion pipeline is being improved to reduce this exclusion rate.

Opinion content is not "judged."

Opinion is excluded from scoring rather than rated, because reliability is the wrong axis to judge editorial commentary on. This means an opinion-heavy outlet has fewer scoreable claims, not a lower score.

Verdict definitions

Eight possible verdicts.

Every claim lands in one of eight categories. Six contribute to the outlet's reliability score. Two (Opinion and Not verifiable) are excluded from scoring. The thresholds are fixed, not negotiable.

The number on each card is the verdict's weight in the outlet's reliability score. Positive weights push the score up; negative weights pull it down.

+1.0 Supported

Supported

Two genuinely independent sources confirm the claim.

+0.5 Plausible

Plausible

Consistent with one credible source but cannot be corroborated.

+0.75 Corroborated

Corroborated

Five or more outlets consistently report the claim without contradiction. Reduced confidence vs Supported.

−0.5 Overstated

Overstated

Core fact is real but framing is misleading or lacks critical context.

−1.0 Disputed

Disputed

At least one credible independent source contradicts the claim.

−1.5 Not supported

Not supported

Available evidence directly contradicts the claim.

Excluded from scoring
Excluded Opinion

Opinion

Editorial commentary, columns, or content explicitly framed as opinion. Excluded from outlet scoring — opinion is the wrong category for the question "is this accurate?", not the wrong kind of content.

Excluded from scoring
Excluded Not verifiable

Not verifiable

Cannot confirm — necessary sources are unavailable, paywalled, or the claim cannot be empirically tested. Excluded from outlet scoring because absence of evidence is not evidence of inaccuracy.

Read the full Methodology v1.7.1.

The complete reference — implementation details, scoring constants, the unabridged changelog, and every rule that produces a verdict.