Text related annotation
Human annotation of MultiNativQA question–answer pairs (across languages and locations) to categorize each question into a single question type (e.g., factual, procedural, norms/etiquette, etc.).
NativQA Framework Answer Editing - Verification
Translation post editing MMLU (Levantine), Global facts and Jurisprudence
Verify 3 question–answer pairs for each image. For each pair: (1) mark whether the Q/A is correct and relevant to the image, and (2) mark whether the question cannot be answered without seeing the image.
Verify 3 question–answer pairs for each image. For each pair: (1) mark whether the Q/A is correct and relevant to the image, and (2) mark whether the question cannot be answered without seeing the image.
Verify 3 question–answer pairs for each image. For each pair: (1) mark whether the Q/A is correct and relevant to the image, and (2) mark whether the question cannot be answered without seeing the image.