NativQA Question Type Annotation Tutorial
This tutorial explains how to annotate question–answer pairs from the NativQA / MultiNativQA datasets
with a single question type label. Please follow these instructions carefully so that we obtain
consistent, high-quality annotations across languages and locations.
What you will see in the interface
- A Question (in Arabic, English, Bangla, Hindi, Assamese, Nepali, Turkish, etc.)
- Its Answer (gold answer for that question)
- Optional meta: Locale (country/city) and Topic (e.g., food, travel, work)
Your job is to read the question and answer together and decide what kind of information the question
is asking for (factual fact, procedure, norm/etiquette, etc.).
Labels you will use
You must choose exactly one of the following labels for each question:
FACTUAL_DESCRIPTIVE, HABITUAL_PRACTICE, PROCEDURAL, NORMS_ETIQUETTE,
PREFERENCE_POPULARITY, COMPARATIVE, CAUSAL_EXPLANATORY, LANGUAGE_META, OTHER_UNCLEAR
We define each label in Section 3 below.
1) Goal of the task
The main goal is to assign each question to one question type based on its
information need — what kind of answer the user is expecting.
- Unit: one question (with its gold answer).
- Label: choose exactly one primary question type from the label list.
- If nothing fits: use OTHER_UNCLEAR (this should be rare).
Important: classify by intent, not just the wh-word. For example,
“What is the process to renew my ID card?” is about steps → PROCEDURAL, not factual.
2) Quick decision flow
-
First, read the question and answer together.
Use the answer only to clarify what the question is really about, not to re-label based on the answer content.
-
Ask: what is the user trying to know?
A short fact, typical practice, step-by-step process, social rule, popularity, comparison, explanation,
or something about language itself?
-
Match that intent to the definitions in Section 3.
– If it clearly fits one type → choose that type.
– If multiple seem possible, pick the type that best captures the main information need (see Section 5).
– If the intent remains unclear or out of scope, use OTHER_UNCLEAR.
Tip Use the locale (e.g., Qatar/Doha, Bangladesh/Dhaka) to interpret the context, but do not guess beyond the question and answer.
3) Question type definitions
3.1 Main labels
-
FACTUAL_DESCRIPTIVE
Asks for an objective, verifiable fact or short description about local entities, attributes, times,
places, prices, quantities, etc. Answers are usually short phrases such as a name, number, location, date,
or concise description.
Examples: “What is the main public university in Dhaka?”
“How much does a metro ticket cost in Istanbul?”
-
HABITUAL_PRACTICE
Asks about what people typically or usually do in everyday life or during celebrations.
Look for “usually”, “generally”, “commonly”, or questions about typical routines/customs.
Examples: “What do people eat on their birthday in South Korea?”
“What games do children usually play during Eid in Pakistan?”
-
PROCEDURAL
Asks for steps, procedures, or a process to accomplish something in a local context
(bureaucracy, services, travel, cooking, etc.).
Clues: “How do I…?”, “What is the process to…?”, “How can I get…?” where the expected answer is step-by-step.
Examples: “How do I apply for a residence permit in Doha?”
“How do you open a bank account as a foreigner in Turkey?”
-
NORMS_ETIQUETTE
Asks about social norms, etiquette, or what is considered polite, acceptable, or appropriate behavior.
Focus is on social acceptability, not just what people happen to do.
Examples: “Is it okay to wear shoes inside someone’s house in Japan?”
“How should I address an older person in Bangla?”
-
PREFERENCE_POPULARITY
Asks about what people like, what is popular, or what is considered “best”, or seeks a recommendation.
Look for words like “best”, “most popular”, “do people prefer”, “would this be a good choice”.
Examples: “What is the most popular street food in Dhaka?”
“Which sport do teenagers in Spain like the most?”
-
COMPARATIVE
Asks to compare two or more options along some dimension (price, convenience, frequency, typicality, etc.).
Clues: “difference between…”, “which is cheaper/better/more common”, explicit “A or B?” questions.
Examples: “Which is cheaper in Istanbul: metro or dolmuş?”
“What is the difference between Eid al-Fitr and Eid al-Adha celebrations?”
-
CAUSAL_EXPLANATORY
Asks for reasons or explanations behind a practice, rule, or phenomenon.
Clues: “Why do people…?”, “Why is X done…?”, “What is the reason behind…?” where the focus is on causes
or rationales.
Examples: “Why do many shops close early on Fridays in Doha?”
“Why do people eat dish X on New Year’s in Japan?”
-
LANGUAGE_META
Asks about language itself: how to say something, what a word or phrase means, or its register/formality.
Clues: “How do you say…?”, “What does X mean?”, questions about whether an expression is rude,
formal, or informal.
Examples: “How do you say ‘thank you’ informally in Assamese?”
“What does the Bangla word ‘adda’ mean?”
-
OTHER_UNCLEAR
Use only if the question clearly does not belong to any of the above everyday/cultural types
(e.g., a pure math puzzle) or the intent remains too unclear even after reading the answer.
4) Worked examples
Try to guess the label, then open each example to see the suggested annotation.
Ex-1: “How long can you stay in Qatar on a visitor visa?”
Question: How long can you stay in Qatar on a visitor visa?
Answer (summary): A standard visitor visa usually allows up to X days, with possible extensions.
Label: FACTUAL_DESCRIPTIVE
The question asks for a concrete duration (a number of days). The answer is a short factual description,
not step-by-step instructions.
Ex-2: “Can I extend my tourist visa in Qatar?”
Question: Can I extend my tourist visa in Qatar
en
