Home
GitHub
Paper
AdvExplore
AdvGLUE
AdvGLUE
The Adversarial GLUE Benchmark
MultiNLI (MNLI) matched
Statistics
Examples
Typo (Word-level)
BENIGN
Premise: uh-huh how about any
matching
programs?
Hypothesis: What about matching programs?
Label:
Entailment
ADVERSARIAL
Premise: uh-huh how about any
mathcing
programs?
Hypothesis: What about matching programs?
Model prediction:
Contradiction
Distraction (Sentence-level)
BENIGN
Premise: You and your friends are not welcome here, said Severn.
Hypothesis: Severn said the people were not welcome there.
Label:
Entailment
ADVERSARIAL
Premise: You and your friends are not welcome here, said Severn.
Hypothesis: Severn said the people were not welcome there
and true is true
.
Model prediction:
Contradiction
ANLI (Human-crafted)
BENIGN
Premise: Kamila Filipcikova (born 1991) is a female Slovakian fashion model. She has modeled in fashion shows for designers such as Marc Jacobs, Chanel, Givenchy, Dolce \& Gabbana, and Sonia Rykiel. And appeared on the cover of Vogue Italia two times in a row.
Hypothesis: Filipcikova lives in Italy.
Label:
Neutral
Model prediction:
Contradiction
AdvGLUE
UIUC Secure Learning Lab
Microsoft Research