Training Multilingual and Adversarial Attack-Robust Models for Hate Detection on Social Media

Social media provide plenty of textual information in various languages. This information can contain or provoke hatred towards different social or religious groups. In this paper, we study methods to process short text messages in English, Hindi, and Russian and identify such intolerance with cross-lingual Transformer models. Moreover, these models can be easily adapted to analyze other languages. We fine-tuned these models with several training techniques to build accurate hate speech detectors that are robust to adversarial attacks. Additional preprocessing was carried out for all datasets to improve the quality of model training. Also, for one of the training datasets, we applied the text attack algorithm that replaces some words with synonyms. For some languages, such an attack can greatly reduce the quality of the model. Experiment results show that mixing adversarial examples to a training dataset and combining deep models to randomized ensembles allows not only to reduce test error on attacked data for languages from the dataset (Hindi, Russian) but also to achieve better accuracy in other languages. © 2022 Elsevier B.V.. All rights reserved.

Авторы
Ryzhova A. , Devyatkin D. , Volkov S. , Budzko V.
Сборник материалов конференции
Издательство
Elsevier B.V.
Номер выпуска
C
Язык
Английский
Страницы
196-202
Статус
Опубликовано
Том
213
Год
2022
Организации
  • 1 Federal Research Center computer Science and Control RAS, Vavilova St, build 2, Moscow, 119333, Russian Federation
  • 2 Peoples' Friendship University of Russia (RUDN University), 6 Miklukho-Maklaya St, Moscow, 117198, Russian Federation
  • 3 National Research Nuclear University MEPHI, Kashirskoe ave., 31, Moscow, 115409, Russian Federation
Ключевые слова
adversarial text attack; cross-lingual transformers; deep learning; Hate speech detection; social media
Цитировать
Поделиться

Другие записи

Kudelin A., Popova E.
Электронный научно-образовательный журнал "История". Институт всеобщей истории РАН, Государственный академический университет гуманитарных наук, ООО "Интеграция: Образование и Наука". Том 13. 2022.