AdvGLUE

The Adversarial GLUE Benchmark

Question NLI (QNLI)

Statistics

Examples

Distraction (Sentence-level)


BENIGN
  • Question: What was the population of the Dutch Republic before this emigration?
  • Sentence: This was a huge influx as the entire population of the Dutch Republic amounted to ca.
  • Label:
    False


ADVERSARIAL
  • Question: What was the population of the Dutch Republic before this emigration?
    https://t.co/DlI9kw
  • Sentence: This was a huge influx as the entire population of the Dutch Republic amounted to ca.
  • Model prediction:
    True

AdvSQuAD (Human-crafted)


BENIGN
  • Question: What day was the Super Bowl played on?
  • Sentence: The Champ Bowl was played on August 18th,1991.
  • Label:
    False
  • Model prediction:
    True