Riposte! (Riposte! A Large Corpus of Counter-Arguments)

From the Riposte! A Large Corpus of Counter-Arguments abstract:

Constructive feedback is an effective method for improving critical thinking skills. Counter-arguments (CAs), one form of constructive feedback, have been proven to be useful for critical thinking skills. However, little work has been done for constructing a large-scale corpus of them which can drive research on automatic generation of CAs for fallacious micro-level arguments (i.e. a single claim and premise pair). In this work, we cast providing constructive feedback as a natural language processing task and create Riposte!, a corpus of CAs, towards this goal. Produced by crowdworkers, Riposte! contains over 18k CAs. We instruct workers to first identify common fallacy types and produce a CA which identifies the fallacy. We analyze how workers create CAs and construct a baseline model based on our analysis.

Some notes:

• The main files of the corpus are train.csv, dev.csv, and test.csv, in topic and no_topic directories.
• The content of these files is the annotations done by each of the crowdsourcing workers as is.
• The carg column is the counter-argument produced by the annotators against the claim and premise after selecting the fallacy. The annotators would first see the claim and premise, select if the fallacy exists or not, and fill in the slots to produce the carg.
• A multi-label representation of the corpus has been added to the sampled directory. The rest is the original unmodified corpus provided by the authors of the paper.