Development of a Mechanism for Recognizing the Emotional State Based on the Unconscious Movements of the Subject

The task of assessing emotional state is not easy for a human and especially challenged for an automated system. It requires not only detection but also complex analysis of various factors. The effectiveness of existing algorithms based on such modalities as text, audio, video, physiological characteristics, etc., depends on the subject’s race, language, and other affiliations, which makes it difficult to study in case of lack or complete absence of such data. The paper deals with the creation of a mechanism capable of detecting the emotional state of a person based on his interaction with a computer via a mouse. To solve the problem, we collected a dataset using a web application developed in the Python programming language. The application is focused on collecting data about the subject’s cursor movement such as distance travelled, maximum deviation from the line that connects the start and end points of the movement, the time the subject interacted with the computer mouse during the session and the maximum speed of the cursor movement. Trained classifiers for emotion analysis based on human control of the computer mouse on the created dataset and analyzed the results. © 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.