Advances in Cognitive Sciences

fa بازشناسی حالات هیجانی با استفاده از شبکه عصبی کانولوشنال فازی مبتنی بر الکتروانسفالوگرافی در باندهای مختلف Recognition of emotional states using fuzzy convolutional neural network based on electroencephalography in different bands مدل سازی شناختی، پردازش سیگنال و تصویربرداری مغز پژوهشي اصیل Research <div style="text-align: justify;">مقدمه: احساسات پدیده&lrm;های متغیر با زمانی هستند که به عنوان پاسخی به محرک‌ها ایجاد می‌شوند. در راستای تشخیص احساس به صورت پیوسته می&lrm;توان از پاسخ سیگنال‌های مغزی و حالت‌های چهره به محرک ویدیویی استفاده کرد. به این صورت که مجموعه&lrm;ای از فیلم&lrm;های محرک برای بینندگان به نمایش گذاشته می&lrm;شود و همزمان سیگنال‌های مغزی و حالت&lrm;های چهره&rlm; آنها به ‌طور پیوسته ضبط می&lrm;گردد و سطح ظرفیت آنها (احساسات منفی تا مثبت) ثبت می&rlm;‌شود. روش کار: هدف از این پژوهش، شناخت احساسات انسانی با استفاده از تحلیل سیگنال‌های الکتروانسفالوگرافی بود. در این مطالعه، برای تشخیص احساسات با استفاده از شبکه عصبی کانولوشنال فازی که ویژگی‌های بهینه و موثر را خود از سیگنال الکتروانسفالوگرافی انتخاب می‌کند جهت تشخیص و بازشناسی حالات هیجانی افراد مختلف ارائه می‌شود. در روش پیشنهادی ابتدا سیگنال الکتروانسفالوگرافی به باندهای مختلف آلفا، بتا و گاما تجزیه شده و سپس عمل تشخیص هوشمند انجام خواهد شد. یافته‌ها: نتایج آزمایشات نشان می‌دهد که حالت آرامش و خستگی در باند آلفا بهتر و به ‌ترتیب با دقت 2/94 درصد و 8/78 درصد بازشناسی می‌شود. در باند گاما شادی بهتر و با دقت 2/92 درصد شناسایی می‌شود و در نهایت در باند بتا، ترس با دقت 3/92 درصد بازشناسی خواهد شد. نتیجه‌گیری: دیده می‌شود که مدل پیشنهادی با استفاده از شبکه عصبی کانولوشنال از دقت بالایی در بازشناسی احساسات برخوردار است همچنین استفاده از منطق فازی در روش پیشنهادی دقت بازشناسی را در کلیه باندها بالا برده است.  </div> Introduction Emotion is a psychological experience characterized by intense mental activity that includes coordinated features such as knowledge, expression, response, inclination, and action. Two types of physiological changes related to emotions: the first is related to the peripheral (external) nervous system, and the other is related to the central nervous system. Facial expressions, speech, body posture, and physiological activities can recognize emotions. These techniques are based on externally expressed emotions, which may not be affected by internal emotions, while brain signals reveal these hidden studies and provide emotional patterns (1). In recent years, emotion classification has been the focus of many researchers. Emotions play an essential role in human life. Currently, thanks to emotion recognition systems, doctors and psychologists are able to diagnose and treat people's mental disorders such as depression, autism, and the like. Therefore, emotions play an essential role in daily life and contribute to physical and mental health. Emotional states can be detected by EEG. Retrieving efficient information from EEG sensors is a complex and challenging task. Therefore, deep learning methods for EEG signal analysis, which have shown promising efficiency, have attracted more and more attention. Many researchers also emphasize learning automatic features as a motivation for using deep learning approaches (1, 2). EEG is the non-invasive way of recording the brain's electrical activity by installing surface electrodes on the head. In general, in an EEG system, the electrical effect of brain neuron activity is transmitted to the device through electrodes installed on the head, and after amplification and noise removal, it is recorded and displayed as a time signal. The recorded signal can be analyzed directly or after computer processing by a doctor or neuroscientist. With the help of EEG, it is possible to determine the amount of brain activity and identify the areas involved in the brain. As a result, checking and analyzing the recorded signal through EEG has an influential role in a wide range of diagnostic and research applications (3). Long-term EEG signal analysis in emotion recognition is very time-consuming and tiring, and it may lead to wrong key point recognition, so various algorithms based on feature extraction have been presented for emotion recognition. The extracted features can explain the complexity and non-linearity of EEG signals (1). By carefully observing the EEG signals, it is possible to recognize the differences in people's emotional states. In addition, analyzing these signals in different situations can help diagnose people's mental abnormalities. Methods This study used an optimal method based on a fuzzy convolutional neural network (CNN) that extracts the optimal features to detect and recognize the emotional states of different people in different bands, including alpha, beta, and gamma. The implementation of the proposed method includes three main phases: 1) Data selection and pre-processing, 2) Convolutional-fuzzy neural network design, and 3) Results analysis and emotion recognition. The CNN generally consists of three parts: the convolutional layer, the integration layer, and the fully connected layer. In the proposed model, a fuzzy layer was used in order to improve the accuracy of the extracted features between the integration layer and the fully connected layer. In general, considering that the recognition of boundaries between classes in classifications still faces insufficient uncertainty, using a fuzzy layer should improve classification accuracy. Unlike classical classification, fuzzy classification means neighboring classes have a continuous border with overlapping regions. The classified object is characterized by its degree of belonging to different classes. The purpose of combining CNN and fuzzy logic is that the proposed method can act more like humans with cognitive uncertainties and have the possibility of processing uncertain information. Results Data were selected from the GAMEEMO database and recorded with a 14-channel Emotive Epoch device (15). Data sets of signals were recorded in five states of happiness, fear, relaxation, and satisfaction. This study conducted experiments on four states of happiness, fear, relaxation, and boredom. The proposed model was evaluated based on accuracy criteria, precision, and recall. In order to validate the proposed method, k-fold and k=11 were used, which gave the best results. This type of partitioning, divided the data into k-1 training sets and one testing set. In this type of validation, the data were divided into K subsets. From these K subsets, each time, one was used for validation, and another K-1 was used for training. This procedure is repeated K times, and all data were used precisely once for training and once for validation (16-18). The experimental results revealed that relaxation is better identified than other emotions in the alpha band. In the alpha band, relaxation is better identified, so the result of the proposed method was also in this direction. In the beta band, a boring state and then fear were recognized well than other emotions. The beta band was activated when someone paid particular attention to an activity. In the gamma band, happiness and then fear were recognized better than other emotions. The beta band was activated when someone was engaged in a brain attention activity, so the result of the proposed method is also acceptable. Conclusion In this study, an optimal method based on the neural-fuzzy network that extracts the optimal features was presented to detect and recognize the emotional states of different people in different bands, including alpha, beta, and gamma. In the proposed model, a CNN was used, and a fuzzy layer was added in order to improve the classification accuracy between the integration layer and the fully connected layer. The results of the proposed method showed that the relaxation mode performed better in the alpha band with an accuracy of 94.2%. In the gamma band, happiness was recognized with 92.2% accuracy, and finally, in the beta band, fear was recognized with 92.4% accuracy, and boring state with 95.5% accuracy. It was also observed that fuzzy logic in the proposed method had increased the recognition accuracy in all bands. Ethical Consideration Compliance with ethical guidelines This article is taken from the master's thesis of the first author. Presenting the report or announcing the research results implies observing the material, spiritual rights, and ethical principles of the relevant elements (the subject, the researcher, the research, and the relevant organization). The research was conducted on the database of standard signals that were already recorded, so doing it did not cause any physical or psychological harm to the subjects. Authors’ contributions Mahdi Bolouri: This article was extracted from the master's thesis of the first author, responsible for project implementation, sample collection, holding analysis sessions and review of the results, and initial writing of the article. Elham Askari: Corresponding author and mentor of the implementation stages of the research, and she was also in charge of revising the article. Funding The first author funded this study. Acknowledgments The authors are grateful for the material and spiritual support of the Islamic Azad University, Fouman, and Shaft Branch in conducting this research with Code 162416439. Conflict of Interest The author declares no conflict of interest.   تشخیص احساسات, الکتروانسفالوگرافی, استخراج ویژگی, شبکه عصبی مصنوعی Emotion, Electroencephalography, Convolutional neural network, Fuzzy logic 102 114 http://icssjournal.ir/browse.php?a_code=A-10-1254-3&slc_lang=fa&sid=1 Mahdi Bolouri مهدی بلوری 100319475328460014522 100319475328460014522 No Master Student of Computer Engineering, Department of Computer Engineering, Fouman and Shaft Branch, Islamic Azad University, Fouman, Iran دانشجوی کارشناسی ارشد گروه کامپیوتر، واحد فومن و شفت، دانشگاه آزاد اسلامی، فومن، ایران Elham Askari الهام عسکری Askary.elham@gmail.com 100319475328460014523 100319475328460014523 Yes Assistant Professor of Computer Engineering, Department of Computer Engineering, Fouman and Shaft Branch, Islamic Azad University, Fouman, Iran استادیار گروه کامپیوتر، واحد فومن و شفت، دانشگاه آزاد اسلامی، فومن، ایران