Fri, Apr 19, 2024
| فارسی
Volume 24, Issue 4 (Winter 2023)
Advances in Cognitive Sciences 2023, 24(4): 102-114 |
Back to browse issues page
Download citation:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
Bolouri M, Askari E. Recognition of emotional states using fuzzy convolutional neural network based on electroencephalography in different bands. Advances in Cognitive Sciences 2023; 24 (4) :102-114
URL: http://icssjournal.ir/article-1-1460-en.html
URL: http://icssjournal.ir/article-1-1460-en.html
1- Master Student of Computer Engineering, Department of Computer Engineering, Fouman and Shaft Branch, Islamic Azad University, Fouman, Iran
2- Assistant Professor of Computer Engineering, Department of Computer Engineering, Fouman and Shaft Branch, Islamic Azad University, Fouman, Iran
2- Assistant Professor of Computer Engineering, Department of Computer Engineering, Fouman and Shaft Branch, Islamic Azad University, Fouman, Iran
Abstract: (523 Views)
Introduction
Emotion is a psychological experience characterized by intense mental activity that includes coordinated features such as knowledge, expression, response, inclination, and action. Two types of physiological changes related to emotions: the first is related to the peripheral (external) nervous system, and the other is related to the central nervous system. Facial expressions, speech, body posture, and physiological activities can recognize emotions. These techniques are based on externally expressed emotions, which may not be affected by internal emotions, while brain signals reveal these hidden studies and provide emotional patterns (1).
In recent years, emotion classification has been the focus of many researchers. Emotions play an essential role in human life. Currently, thanks to emotion recognition systems, doctors and psychologists are able to diagnose and treat people's mental disorders such as depression, autism, and the like. Therefore, emotions play an essential role in daily life and contribute to physical and mental health. Emotional states can be detected by EEG. Retrieving efficient information from EEG sensors is a complex and challenging task. Therefore, deep learning methods for EEG signal analysis, which have shown promising efficiency, have attracted more and more attention. Many researchers also emphasize learning automatic features as a motivation for using deep learning approaches (1, 2).
EEG is the non-invasive way of recording the brain's electrical activity by installing surface electrodes on the head. In general, in an EEG system, the electrical effect of brain neuron activity is transmitted to the device through electrodes installed on the head, and after amplification and noise removal, it is recorded and displayed as a time signal. The recorded signal can be analyzed directly or after computer processing by a doctor or neuroscientist. With the help of EEG, it is possible to determine the amount of brain activity and identify the areas involved in the brain. As a result, checking and analyzing the recorded signal through EEG has an influential role in a wide range of diagnostic and research applications (3).
Long-term EEG signal analysis in emotion recognition is very time-consuming and tiring, and it may lead to wrong key point recognition, so various algorithms based on feature extraction have been presented for emotion recognition. The extracted features can explain the complexity and non-linearity of EEG signals (1). By carefully observing the EEG signals, it is possible to recognize the differences in people's emotional states. In addition, analyzing these signals in different situations can help diagnose people's mental abnormalities.
Methods
This study used an optimal method based on a fuzzy convolutional neural network (CNN) that extracts the optimal features to detect and recognize the emotional states of different people in different bands, including alpha, beta, and gamma. The implementation of the proposed method includes three main phases: 1) Data selection and pre-processing, 2) Convolutional-fuzzy neural network design, and 3) Results analysis and emotion recognition.
The CNN generally consists of three parts: the convolutional layer, the integration layer, and the fully connected layer. In the proposed model, a fuzzy layer was used in order to improve the accuracy of the extracted features between the integration layer and the fully connected layer. In general, considering that the recognition of boundaries between classes in classifications still faces insufficient uncertainty, using a fuzzy layer should improve classification accuracy. Unlike classical classification, fuzzy classification means neighboring classes have a continuous border with overlapping regions. The classified object is characterized by its degree of belonging to different classes. The purpose of combining CNN and fuzzy logic is that the proposed method can act more like humans with cognitive uncertainties and have the possibility of processing uncertain information.
Results
Data were selected from the GAMEEMO database and recorded with a 14-channel Emotive Epoch device (15). Data sets of signals were recorded in five states of happiness, fear, relaxation, and satisfaction. This study conducted experiments on four states of happiness, fear, relaxation, and boredom. The proposed model was evaluated based on accuracy criteria, precision, and recall. In order to validate the proposed method, k-fold and k=11 were used, which gave the best results. This type of partitioning, divided the data into k-1 training sets and one testing set. In this type of validation, the data were divided into K subsets. From these K subsets, each time, one was used for validation, and another K-1 was used for training. This procedure is repeated K times, and all data were used precisely once for training and once for validation (16-18).
The experimental results revealed that relaxation is better identified than other emotions in the alpha band. In the alpha band, relaxation is better identified, so the result of the proposed method was also in this direction. In the beta band, a boring state and then fear were recognized well than other emotions. The beta band was activated when someone paid particular attention to an activity. In the gamma band, happiness and then fear were recognized better than other emotions. The beta band was activated when someone was engaged in a brain attention activity, so the result of the proposed method is also acceptable.
Conclusion
In this study, an optimal method based on the neural-fuzzy network that extracts the optimal features was presented to detect and recognize the emotional states of different people in different bands, including alpha, beta, and gamma. In the proposed model, a CNN was used, and a fuzzy layer was added in order to improve the classification accuracy between the integration layer and the fully connected layer. The results of the proposed method showed that the relaxation mode performed better in the alpha band with an accuracy of 94.2%. In the gamma band, happiness was recognized with 92.2% accuracy, and finally, in the beta band, fear was recognized with 92.4% accuracy, and boring state with 95.5% accuracy. It was also observed that fuzzy logic in the proposed method had increased the recognition accuracy in all bands.
Ethical Consideration
Compliance with ethical guidelines
This article is taken from the master's thesis of the first author. Presenting the report or announcing the research results implies observing the material, spiritual rights, and ethical principles of the relevant elements (the subject, the researcher, the research, and the relevant organization). The research was conducted on the database of standard signals that were already recorded, so doing it did not cause any physical or psychological harm to the subjects.
Authors’ contributions
Mahdi Bolouri: This article was extracted from the master's thesis of the first author, responsible for project implementation, sample collection, holding analysis sessions and review of the results, and initial writing of the article. Elham Askari: Corresponding author and mentor of the implementation stages of the research, and she was also in charge of revising the article.
Funding
The first author funded this study.
Acknowledgments
The authors are grateful for the material and spiritual support of the Islamic Azad University, Fouman, and Shaft Branch in conducting this research with Code 162416439.
Conflict of Interest
The author declares no conflict of interest.
Emotion is a psychological experience characterized by intense mental activity that includes coordinated features such as knowledge, expression, response, inclination, and action. Two types of physiological changes related to emotions: the first is related to the peripheral (external) nervous system, and the other is related to the central nervous system. Facial expressions, speech, body posture, and physiological activities can recognize emotions. These techniques are based on externally expressed emotions, which may not be affected by internal emotions, while brain signals reveal these hidden studies and provide emotional patterns (1).
In recent years, emotion classification has been the focus of many researchers. Emotions play an essential role in human life. Currently, thanks to emotion recognition systems, doctors and psychologists are able to diagnose and treat people's mental disorders such as depression, autism, and the like. Therefore, emotions play an essential role in daily life and contribute to physical and mental health. Emotional states can be detected by EEG. Retrieving efficient information from EEG sensors is a complex and challenging task. Therefore, deep learning methods for EEG signal analysis, which have shown promising efficiency, have attracted more and more attention. Many researchers also emphasize learning automatic features as a motivation for using deep learning approaches (1, 2).
EEG is the non-invasive way of recording the brain's electrical activity by installing surface electrodes on the head. In general, in an EEG system, the electrical effect of brain neuron activity is transmitted to the device through electrodes installed on the head, and after amplification and noise removal, it is recorded and displayed as a time signal. The recorded signal can be analyzed directly or after computer processing by a doctor or neuroscientist. With the help of EEG, it is possible to determine the amount of brain activity and identify the areas involved in the brain. As a result, checking and analyzing the recorded signal through EEG has an influential role in a wide range of diagnostic and research applications (3).
Long-term EEG signal analysis in emotion recognition is very time-consuming and tiring, and it may lead to wrong key point recognition, so various algorithms based on feature extraction have been presented for emotion recognition. The extracted features can explain the complexity and non-linearity of EEG signals (1). By carefully observing the EEG signals, it is possible to recognize the differences in people's emotional states. In addition, analyzing these signals in different situations can help diagnose people's mental abnormalities.
Methods
This study used an optimal method based on a fuzzy convolutional neural network (CNN) that extracts the optimal features to detect and recognize the emotional states of different people in different bands, including alpha, beta, and gamma. The implementation of the proposed method includes three main phases: 1) Data selection and pre-processing, 2) Convolutional-fuzzy neural network design, and 3) Results analysis and emotion recognition.
The CNN generally consists of three parts: the convolutional layer, the integration layer, and the fully connected layer. In the proposed model, a fuzzy layer was used in order to improve the accuracy of the extracted features between the integration layer and the fully connected layer. In general, considering that the recognition of boundaries between classes in classifications still faces insufficient uncertainty, using a fuzzy layer should improve classification accuracy. Unlike classical classification, fuzzy classification means neighboring classes have a continuous border with overlapping regions. The classified object is characterized by its degree of belonging to different classes. The purpose of combining CNN and fuzzy logic is that the proposed method can act more like humans with cognitive uncertainties and have the possibility of processing uncertain information.
Results
Data were selected from the GAMEEMO database and recorded with a 14-channel Emotive Epoch device (15). Data sets of signals were recorded in five states of happiness, fear, relaxation, and satisfaction. This study conducted experiments on four states of happiness, fear, relaxation, and boredom. The proposed model was evaluated based on accuracy criteria, precision, and recall. In order to validate the proposed method, k-fold and k=11 were used, which gave the best results. This type of partitioning, divided the data into k-1 training sets and one testing set. In this type of validation, the data were divided into K subsets. From these K subsets, each time, one was used for validation, and another K-1 was used for training. This procedure is repeated K times, and all data were used precisely once for training and once for validation (16-18).
The experimental results revealed that relaxation is better identified than other emotions in the alpha band. In the alpha band, relaxation is better identified, so the result of the proposed method was also in this direction. In the beta band, a boring state and then fear were recognized well than other emotions. The beta band was activated when someone paid particular attention to an activity. In the gamma band, happiness and then fear were recognized better than other emotions. The beta band was activated when someone was engaged in a brain attention activity, so the result of the proposed method is also acceptable.
Conclusion
In this study, an optimal method based on the neural-fuzzy network that extracts the optimal features was presented to detect and recognize the emotional states of different people in different bands, including alpha, beta, and gamma. In the proposed model, a CNN was used, and a fuzzy layer was added in order to improve the classification accuracy between the integration layer and the fully connected layer. The results of the proposed method showed that the relaxation mode performed better in the alpha band with an accuracy of 94.2%. In the gamma band, happiness was recognized with 92.2% accuracy, and finally, in the beta band, fear was recognized with 92.4% accuracy, and boring state with 95.5% accuracy. It was also observed that fuzzy logic in the proposed method had increased the recognition accuracy in all bands.
Ethical Consideration
Compliance with ethical guidelines
This article is taken from the master's thesis of the first author. Presenting the report or announcing the research results implies observing the material, spiritual rights, and ethical principles of the relevant elements (the subject, the researcher, the research, and the relevant organization). The research was conducted on the database of standard signals that were already recorded, so doing it did not cause any physical or psychological harm to the subjects.
Authors’ contributions
Mahdi Bolouri: This article was extracted from the master's thesis of the first author, responsible for project implementation, sample collection, holding analysis sessions and review of the results, and initial writing of the article. Elham Askari: Corresponding author and mentor of the implementation stages of the research, and she was also in charge of revising the article.
Funding
The first author funded this study.
Acknowledgments
The authors are grateful for the material and spiritual support of the Islamic Azad University, Fouman, and Shaft Branch in conducting this research with Code 162416439.
Conflict of Interest
The author declares no conflict of interest.
Type of Study: Research |
Received: 2022/08/27 | Accepted: 2022/12/28 | Published: 2023/02/19
Received: 2022/08/27 | Accepted: 2022/12/28 | Published: 2023/02/19
Send email to the article author
| Rights and permissions | |
 |
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License. |
