010-53352947

Sharing | A multimodal approach to measure the impact of service robots with different levels of anthropomorphism on users' emotional experiences



01

Research Background




Service robots are robots that perform useful tasks for humans or devices. When interacting with a service robot, its appearance makes the first impression on the user (K. Li et al., 2020) [1]. It is not difficult to imagine the following scenario: when you are choosing a service robot in a shopping mall, a robot with a characteristic appearance (e.g., facial aspect ratio) will attract your attention, make you feel interested and excited, and create a desire to obtain further information (Song and Luximon, 2021) [2]. In general, the appearance of service robots can be classified into four categories: anthropomorphic, animalistic, caricatured, and functional, and according to Fong, Nourbakhsh, and Dautenhahn's classification of social robots (2003) [3], anthropomorphic robots justify the design of the robot's appearance and behavior through intra-human attribution to conform to human expectations and promote human-robot interaction (Chiang, Trimi and Lo, 2022) [4]. Previous research has provided evidence that the anthropomorphic appearance of a product elicits positive emotions and increases its attractiveness to customers (Cao et al., 2022) [5]. However, there are conflicting views on the effects of anthropomorphic appearance of robots, with a meta-analysis of the effects of anthropomorphic design features on human-related outcomes showing that anthropomorphic appearance of robots positively affects perception (Roesler, Manzey and Onnasch, 2021) [6]. In contrast, research represented by the Valley of Terror phenomenon has shown that very high levels of anthropomorphic appearance elicit feelings of fear, mistrust, and other oddities in users. And emotional responses are complex, consisting of at least three aspects: self-report, expressive behavior, and physiological responses (Gross and Levenson, 1993) [7]. The effect of anthropomorphic appearance of robots on emotion has not been sufficiently studied through psychophysiological measures. It is therefore of interest to investigate the effect of anthropomorphization of service robots on users' emotional experience.



02

Research Methodology







1. Experimental design

Using a within-subjects experimental design, service robots were anthropomorphized with three levels of appearance: high (AH), medium (AM), and low (AL) levels. To test whether these robots have a significant effect on subjects' mood.

In addition to this, four dependent variables were chosen as multimodal measures: subjective emotional experience, eye tracking, physiological activity, and attitudes toward robot acceptance.




2. Recruitment of subjects

Fifty-two undergraduate students from different majors were recruited through social media, and all participants received 20 RMB in return. None had participated in the same study or used the service robot used in the experiment, and they did not take stimulants (e.g., caffeine in coffee or tea) and tranquilizers (e.g., alcohol or sleeping pills) prior to the experiment. These criteria were confirmed for all students by subjective reports, but two participants with severe myopia were excluded because their eye-tracking data could not be calibrated.

Finally, a total of 50 students (29 males and 21 females) completed the experiment (age range = 19-29, mean age = 23.75, SD = 4.14). They all signed an informed consent form before the experiment. The study was approved by the ethics committee.




3. Experimental flow

To assess the subjects' initial mood, participants were first asked to complete the PANAS scale, followed by being fitted with an eye-tracker and physiological recorder by the primary subject.

Twenty-one robotic images were presented one by one in the center of the display in a randomized order during the formal trial phase, and each image was displayed for 15 s. When all images had been presented, the electrodes were removed and the eye-tracker was taken off. Participants then filled out a questionnaire that included the SAM scale and an attitude scale.

At the end, they were thanked and asked how they felt about the three types of robots during the interview, the whole process is as follows.



03

Results







1. Subjective emotional experiences and attitudes

1

The AM robot elicited more significant pleasure and arousal than the AL and AH robots.

2

Subjects were significantly more receptive to AM than to AL and AH.




2. Eye movement results

The results of the one-way ANOVA showed significant differences in subjects' pupil diameter, number of gaze sessions, number of sweeps, and sweep speed across the three conditions (AL, AM, and AH). Specifically, the anthropomorphic robot at the AM level resulted in subjects having larger pupil diameters, more gaze counts, more frequent sweeps, and faster sweep speeds compared to the AL and AH conditions.

In addition, hotspot maps and gaze trajectory maps provided visual evidence for exploring participants' gaze and scanning conditions. As shown in the figure below, the distribution of participants' visual attention is represented in red and green, respectively. The hotspot map showed that facial features were more attractive to participants, followed by the body. In addition to this, the order of visual attention was marked by the numbers on the circles on the gaze trajectory graph. The gaze trajectory graph shows that the participants noticed the facial features first, followed by the body and other parts of the body.




3. Physiological results

An ANOVA of the EMG signals showed significant differences in subjects' EMG levels across the three conditions. Specifically, the AM robot elicited more significant RMS and MPF than the AL and AH robots.

The EDA signal ANOVA showed that anthropomorphism had a significant effect on SCL, with the AM robot inducing greater SCL than the AH and AL robots.

The results of the ANOVA of HRV signals showed that anthropomorphism had a significant effect on mean HR. The results of pairwise comparisons showed that the mean heart rate (MeanHR) induced by the AM robot was greater than that of AH and AL.



04

talk over




The purpose of this study is to understand how the anthropomorphic appearance of a service robot affects users' emotional experiences and attitudes. We used multimodal measures to analyze subjective emotional experience (pleasure, arousal), eye movements, physiological arousal (fEMG, EDA, and HRV), and attitudes to provide holistic insights into emotional responses in human-robot interaction.

This study also has some limitations and shortcomings, and in order to narrow down the scope of the study, we selected service robots and clustered them according to different degrees of anthropomorphism. It is worth noting that according to the definition given in ISO 8373:2012, service robots can also be subdivided into personal service robots and professional service robots, which are used in different work scenarios and have different functions. Therefore, future research should classify anthropomorphic service robots based on their usage attributes (personal or professional).

In addition, the participants of the experiment were young college students, which limits the generalizability of the findings to some extent. In fact, middle-aged and old-aged groups are equally noteworthy in terms of their emotional experiences, especially the elderly group, who may rely more on service robots to assist in their daily lives due to mobility problems and cognitive decline. Therefore, it would be an important research direction to include the elderly in the scope of the study to explore their emotional experiences and attitudinal changes when interacting with service robots.



05

Conclusion




The main objective of this study is to investigate the effects of anthropomorphic appearance of service robots on users' subjective emotional experience, attitude, eye movements and physiological arousal.

First, the findings confirm that multimodal measurement indices can be used to measure the emotional impact of a robot's anthropomorphic appearance, which include: subjective emotional experiences (pleasure and arousal), eye movement (situation) data, and physiological activity indicators.

Second, the study demonstrated that emotional experiences and attitudes changed nonlinearly as the anthropomorphic visual features of the robots increased. Specifically, the AM robot gave subjects a more positive emotional experience than the AH and AL robots. (Third, the AM robot significantly contributed to more positive subjective emotional experiences and attitudes than the AH and AL robots.)

Finally, measures of eye movements and physiological arousal further confirmed the positive effects of a robot with a moderately anthropomorphic appearance.


Literature citation


1. Zhang, Y., Cao, Y., Proctor, R. W., & Liu, Y. (2023). Emotional experiences of service robots' anthropomorphic appearance: a multimodal measurement method. Ergonomics, 66(12), 2039 -2057. https://doi.org/10.1080/00140139.2023.2182751

 

2. li, K., J. Zhang, L. Wang, M. Zhang, J. Li, and S. Bao. 2020. "A Review of the Key Technologies for sEMG-Based Human-Robot Interaction Systems." Biomedical Signal Processing and Control 62 (9): 1-17. doi:10.1016/j.bspc.2020.102074

 

3. song, Y., and Y. Luximon. 2021. "The Face of Trust: the Effect of Robot Face Ratio on Consumer Preference." computers in Human Behavior 116 (3): 1-9. doi:10.1016/j.chb.2020.106620.

 

4.Chiang, A.H., S. Trimi, and Y.J. Lo. 2022. "Emotion and Service Quality of Anthropomorphic Robots. "Technological Forecasting and Social Change 177 (2): 1-11. doi:10.1016/j.techfore.2022.121550.

 

5. Cao, Y., R.W. Proctor, Y. Ding, V.G. Duffy, Y. Zhang, and X. Zhang. 2022. "Is an Anthropomorphic App Icon More Attractive than a Non-Anthropomorphic One? A Case Study Using Multimodal Measurement." International Journal of Mobile Communication 20 (4): 419-439. 10035579. International Journal of Mobile Communication 20 (4): 419-439. 10035579.

 

6. roesler, E., D. Manzey, and L. Onnasch. 2021. "A Meta-Analysis on the Effectiveness of Anthropomorphism in Human-Robot Interaction. " Science Robotics 6 (58): 1-10. doi:10.1126/scirobotics.abj5425.

 

7.Gross, J.J., and R.W. Levenson. 1993. "Emotional Suppression: Physiology, Self-Report, and Expressive Behavior." Journal of Personality and Social Psychology 64 (6): 970-986. doi:10.1037/0022-3514.64.6.970.

Company Profile

Ltd., invested by Zhongke (Guangdong) Science Group and relying on Guangdong Human Factors Technology Research Institute and Wuhan Human Factors Engineering Technology Research Institute, is a new high-tech enterprise based on the direction of psychological human factors, driving human factors, biomechanics, user experience, virtual reality and other aspects of the production, research and development, sales and technical services in one of the National High-tech Enterprises, science and technology-based small and medium-sized enterprises, Beijing "innovative" small and medium-sized enterprises, Zhongguancun High-tech Enterprise list. The company has been selected as a national high-tech enterprise, science and technology-based small and medium-sized enterprise, Beijing "innovative" small and medium-sized enterprise and Zhongguancun high-tech enterprise.
Hengzheng Technology has been serving scientific research institutes for a long time, doing its part for China's scientific research, and maintaining good cooperative relations with the Chinese Psychological Association, Architectural Society of China, Chinese Society of Ergonomics, Chinese Society of Technology and Economics, Chinese Society of Management Engineering, Chinese Society of Automotive Engineering and other societies! At present, it is the deputy secretary-general unit of Engineering Construction Management Committee of China Society for Technical Economics, and the deputy secretary-general unit of Human-Computer Interaction Committee of China Society for Human Ergonomics!
Hengzhi Technology independently researches and develops driving human factors system, virtual reality graphical editing software, optical environment psychological assessment system, psychological and human factors experimental teaching system, human stress testing system, virtual reality interactive system, multimodal data acquisition software, etc. Meanwhile, as the general agent of Poland Cortivision near infrared, Russia Mitsar EEG and Germany Eyelogic eye movement instrument in China, Italy BTS surface electromyography and other biomechanical and gait analysis scientific research products. At the same time, as the general agent of Poland Cortivision NIR, Russia Mitsar EEG and Germany Eyelogic eye-tracker in China, Italy BTS surface electromyography and other biomechanics and gait analysis scientific research products, Canada AdHawk Mindlink high sampling spectacle eye-tracker, Germany QuaeroSys tactile stimulation system, the Netherlands Noldus behavioral sciences, Sweden Tobii eye-tracker, the Netherlands MindMedia physiology and biofeedback, the U.S. Biopac physiological, the United States ETT smell / taste stimulation. ETT olfactory/taste stimulator and other products agent. We have served Tsinghua University, Peking University, Beijing Normal University, Northeast Normal University, Yanshan University, Xi'an University of Architecture and Technology, Northwest Agriculture and Forestry University, Shenzhen University of Technology, Xi'an University of Science and Technology, Shanghai Jiaotong University, Xinjiang Normal University, Qiyuan Laboratory, China Electronics Technology Group 27, China Electronics Technology Group 28, Huawei Technologies, Mobilink Weather, NetEase, Aerospace Academy II, and so on. Thousands of colleges and universities, research institutes and enterprises and institutions continue to carry out in-depth cooperation in talent cultivation, production and research cooperation, and transformation of achievements.

Follow us at

en_USEnglish