An Evaluation Dataset for Identifying Communicative Functions of Sentences in English Scholarly Papers

Formulaic expressions, such as {`}in this paper we propose{'}, are used by authors of scholarly papers to perform communicative functions; the communicative function of the present example is {`}stating the aim of the paper{'}. Collecting such expressions and pairing them with their communicative functions would be highly valuable for various tasks, particularly for writing assistance. However, such collection and paring in a principled and automated manner would require high-quality annotated data, which are not available. In this study, we address this shortcoming by creating a manually annotated dataset for detecting communicative functions in sentences. Starting from a seed list of labelled formulaic expressions, we retrieved new sentences from scholarly papers in the ACL Anthology and asked multiple human evaluators to label communicative functions. To show the usefulness of our dataset, we conducted a series of experiments that determined to what extent sentence representations acquired by recent models, such as word2vec and BERT, can be employed to detect communicative functions in sentences.