X-ParEval: A Multilingual Metric for Paraphrase Evaluation

Overview & Synopsis

Paraphrase Evaluation is the task of assigning a score that represents how well a pair of texts paraphrase each other. This task is helpful to assess other tasks like Paraphrase Generation or Paraphrase Compilation. Despite its usefulness, there has been a lack of standardization which complicates comparability between studies. Some of the used approaches are language-dependent. Furthermore, most of the common approaches tend to omit some elements of good paraphrasing. In this thesis, we propose X-ParEval, a multilingual metric for Paraphrase Evaluation that relies on embedding-based semantic similarity while penalizing surface similarity. Our metric outperforms other used metrics in terms of correlation with human judges and works well with various languages. Finally, by providing a publicly available library to compute the score, we hope to encourage and facilitate comparability on future works that require Paraphrase Evaluation.