• Info
• Contribute

Purpose

This project creates reliable human annotations for Arabic numerical and temporal factual claims. For each task, decide whether the item is a valid numerical claim, identify its primary numerical category, and determine how the supplied evidence relates to the complete claim.

Judge only from the claim, evidence text, metadata, and source links supplied in the task. Do not assume that a statement is true because it appeared in Wikipedia, and do not use personal knowledge or independent web searches.

What to check

Always compare the complete meaning of the claim with the evidence:

• entity or entities;
• numerical or temporal value;
• unit;
• date or time scope;
• comparison group or range boundaries; and
• qualifiers such as “approximately,” “more than,” “first,” or “between.”

The highlighted numerical spans are automatic hints. They may be incomplete or incorrect, so evaluate the full claim.

Annotation steps

1. Claim validity

Choose one option:

• Valid claim (Valid): a checkable numerical or temporal factual assertion.
• Not numerical (NotNumerical): the detected expression is only part of a name, code, model, team, project, or another non-quantitative expression.
• Not factual (NotFactual): an opinion, question, recommendation, prediction without a checkable assertion, or figurative statement.
• Missing context (NotSelfContained): an essential entity or proposition cannot be identified from the task.
• Unclear (Unclear): no other validity option can be selected reliably. Explain the problem in Notes.

If the claim is not valid, the remaining evidence questions are skipped.

2. Primary numerical category

For a valid claim, select the condition that primarily determines whether it is true:

• Statistical: count, amount, percentage, rate, money, measurement, fraction, or other quantity.
• Temporal: date, year, age, duration, order, or timing is the primary quantity.
• Interval: explicit range, minimum, maximum, or lower/upper boundary.
• Comparison: comparison between quantities, periods, groups, ranks, ratios, increases, decreases, or relative changes.

Use the priority Interval → Comparison → Temporal → Statistical when more than one category appears applicable. For example, a population value with a census year is normally Statistical, while a comparison of populations across two years is Comparison.

3. Evidence document status

Assess the selected evidence document:

• Readable & matched (ReadableMatched): it is readable and is the cited document relevant to this claim.
• Wrong document (ReadableWrongDocument): it is readable but unrelated, a homepage, login page, error page, or otherwise not the cited content.
• Unreadable (Unreadable): a document exists but is corrupt or cannot be interpreted reliably.
• Unavailable (Unavailable): no usable evidence document is supplied.

When multiple evidence documents are available, inspect them using the selector above the evidence panel. Select the most decisive document before submitting.

A wrong, unreadable, or unavailable document provides Insufficient evidence. It does not show that the claim is false.

4. Evidence stance

Choose exactly one stance for a valid claim:

Supports (Supports)

A specific, sufficiently reliable passage agrees with every decision-relevant part of the claim: entity, value, unit, time scope, direction, boundaries, and qualifiers.

Refutes (Refutes)

A specific, sufficiently reliable passage explicitly contradicts at least one required part of the claim while discussing the same entity, unit, definition, and time scope.

A different value from another year, place, population, or measurement definition is not automatically a refutation.

Conflicting (Conflicting)

Reliable supplied passages make incompatible statements about the same proposition, and the disagreement cannot be resolved from their dates, scope, definitions, or source quality.

Record the additional conflicting passage and source in Notes.

Insufficient (Insufficient)

The supplied evidence cannot establish Supports, Refutes, or Conflicting. Examples include:

• no usable evidence;
• the decisive quantity is not mentioned;
• the evidence concerns a different entity, unit, or time period;
• only part of a multi-part claim is addressed;
• the passage is ambiguous or unsuitable for establishing the fact; or
• the required inference cannot be made reliably.

Explain what is missing in Notes. Insufficient does not mean False.

Evidence passage and locator

For Supports, Refutes, and Conflicting:

1. copy the smallest passage that contains the necessary entity, value, unit, and time context;
2. use Use selected text after selecting text in the evidence panel, or paste the passage manually; and
3. record its page, section, paragraph, table row, or other locator.

For a PDF, use the printed page number when available. For a table, identify the table, row, column, and any essential footnote.

Do not copy an entire article when a short passage is sufficient.

Important decision rules

Multi-part claims

All required components must be supported for the overall stance to be Supports. If the evidence explicitly refutes any required component, use Refutes. If it supports some components but is silent about others, use Insufficient.

Approximate and bounded language

Respect the wording of the claim:

• “approximately 20” does not require exactly 20;
• “more than 20” is not supported by exactly 20;
• “at least 20” includes 20; and
• range endpoints and inclusiveness must be checked when they affect the decision.

Conversions and dates

Equivalent units or calendars may be used only after a transparent, deterministic conversion. Record the conversion in Notes. Check the appropriate date for currency conversions and the correct calendar for temporal claims.

Ranks and “first” claims

A rank requires the correct comparison set. Evidence that an event occurred does not, by itself, establish that it was “the first in the world.”

Before submitting

Confirm that:

• you evaluated the complete claim rather than only the highlighted number;
• missing evidence was not labeled Refutes;
• Supports or Refutes has an exact passage and locator;
• Insufficient and Conflicting decisions have an explanation in Notes.

Thank you for contributing to this research.