虚拟现实环境中的注意状态识别 - 北京恒挚科技有限公司

Researchers from Poland have published an article in the journal Sensors that explores the practical combination of virtual reality (VR) and near-infrared technology. They designed an integrated experimental platform containing CW-fNIRS and head-mounted display (HMD) technologies and conducted experiments in virtual reality using a variant of the classic n-back task (2-back version). The results confirm that by combining fNIRS and HMD technologies, experimenters can effectively transfer experimental cognitive processes to a controlled VR environment.

Research Background

Functional near-infrared spectroscopy (fNIRS) is one of the newest and fastest functional neuroimaging techniques that uses near-infrared light at wavelengths of 650-950 nm to measure hemodynamic responses in cortical brain regions. NIR systems typically use a laser or LED to emit continuous wave (CW) NIR light.

VR technology has been used for research and training of attentional skills, and there have been many case studies and papers combining VR and NIR technology.

The feasibility of modulating attentional states through training procedures in combination with VR and FNIRS techniques has been demonstrated. Identifying attentional states can be used to monitor the engagement of subjects in the task. The user's experience of interacting with the system can be measured by visual scales and questionnaires to measure their satisfaction.

The hypothesis of this study.

The difference in hemodynamic activity (changes in HbO/HbR concentrations) between enhanced attentional engagement (n-back task) and relaxation states in the DLFPC and MFG regions would be significantly higher than the chance threshold levels in the study group.

The user experience of interaction with the system (fNIRS+HMD) will be higher than the mean of the satisfaction assessment scale.

Subjects

Twelve subjects (10 females), aged 21-34 years (M = 24.82; SD = 4.38) participated in the experiment. All participants in the experiment were right-handed with normal bare and corrected visual acuity.

Equipment

NIR signals were recorded in a dual wavelength (760 and 850 nm) fnirs system ((Cortivision sp. z o.o., Lublin, Poland). A total of 16 LED sources and 10 detectors were used.

Data processing for the online phase was performed on OpenViBE 3.1.0 (Inria Hybrid Team, Rennes, France) using a custom Python script.

Develop VR scenes using the Unity3D engine and display them on Oculus Quest (Facebook Technologies, Menlo Park, CA, USA).

The results were statistically analyzed using JASP software.

The experimental setup consisted of wireless integrated VR goggles and a wearable fNIRS device. During data recording, participants were seated and the devices were wirelessly connected.

Subjective satisfaction assessment

The Visual Analogue Scale (VAS) was used to assess the overall subjective satisfaction at the end of the experiment in both groups. The assessment was done on an 11-point scale".

A modified version of the Quebec Assistive Technology User Evaluation (eQUEST 2.0) was used to measure participant satisfaction with the use of the system.

Experimental procedure

The study consisted of three parts.

Phase 1 (Tutorial): You need to determine if the current fruit has been seen before two fruits. When the mage throws the energy ball represents relaxation.

Phase 2: The offline phase has the same tasks as the tutorial.

Stage 3: Subjects were asked to do the same task only in the brain. The direction of the object's motion was directly controlled by the categorical output signal.

In the experiment, tasks and rests were alternated sequentially. In each "2-back" task block, the fruit list was randomly divided into 3 targets and 7 non-targets.

Data Collection

NIR data were collected by the Cortivision PHOTON CAP system. Ten channels were used. In addition, two short distance channels were placed at positions F3 and F4 (see Figure 3). For all data channels, the distance between the light source and the detector was kept at about 30 mm. For the short channels, the distance was fixed at 10 mm. The placement positions of the light sources and detectors were selected based on predefined positions in the software. The three regions of interest (MFG, L-DLPFC, R-DLPFC) were automatically translated from brain region anatomy to 10-5 international system positions.

Signal Processing

Converts raw light intensity to optical density. A baseline was recorded during the first 5 seconds of the treatment recording. The optical density was then converted to oxy-deoxyhemoglobin (HbO/HbR) concentration. Stimulus-based segmentation was then performed based on VR application flow markers. Low-pass filtering was performed. Generalized linear model (GLM) was used for feature extraction and further analysis.

The obtained dataset containing the extracted features is divided into five parts for five times cross-validation. Each repeated cross-validation consists of the following steps.

Z-score normalization: z-score normalization of the training and test sets.

The SVM classifier was used to distinguish between two classes: "relaxed" and "2-back tasks".

Results

Classification accuracy

The results of the one-sample nonparametric Wilcoxon signed rank test showed a classification accuracy of M=88.58 and SD=8.49 for the calibration phase, significantly higher than the upper probability limit of 95% calculated by Müller-Putz et al. for the two BCI categories (T=78, p<0.01, rrb=1). However, we did not observe significant differences in the online condition (M=61, SD=14.89, T=6, p=0.107). Also, both conditions were significantly higher than the chance level of 50% (calibrated: t=78, p<0.001, rrb=1; online: t=31, p<0.001, rrb=0.72). The mean classification accuracy results for all participants in the calibration and online sessions are given in Table.

User satisfaction

Overall user satisfaction with the system's interactions averaged 6 out of 10, which is close to the highest rating of "very satisfied" out of 11. The survey found that the highest scoring aspects of the interaction were security and ease of use. For the rest of the system, ratings averaged 5 or higher, including adjustability, size, reliability, weight, and comfort.

Conclusion

Classification of hemodynamic changes according to the region of interest allows identification of the degree of attentional enhancement and relaxation.

In the experiment, overall user satisfaction was higher for head-mounted display devices and mobile fNIRS interactions.

In addition, the signal processing approach in open source real-time data processing software could provide a framework for future brain-computer interface and neurofeedback research.

Portable fNIRS and virtual reality systems are feasible to study cognitive processes under more natural conditions.

References:

Zapała, D., Augustynowicz, P., & Tokovarov, M. (2022). Recognition of Attentional States in VR Environment: an fNIRS Study. sensors, 22(9), 3133.

Company Profile

Beijing Hengzheng Technology Co.(Guangdong)Science group investment, relying on the Guangdong Institute of Human Factors Technology and Wuhan Institute of Human Factors Engineering TechnologyThe technical background ofIt is a new type ofofWith the psychological human factors, driving human factors, biologicalObject mechanics, user experience, virtual reality and other directions based on the production, research and development, sales, technical services in oneofHigh and NewTechnology-based enterprise, has been successfully selected to the list of high-tech enterprises in Zhongguancun.

Independently developed by Hengzhi Technologyofdriving human factors system, virtual reality graphical editing software, light environment psychological assessment system, psychological and human factors experimental teaching system.It has entered the domestic market.

Hengzhi TechnologyAs PolandCortivisionNIR, RussiaMitsarGeneral Manager of EEG ChinaItalyBTSSurface electromyography and other biologyPhysical mechanics and gait analysisof domesticProducts General AgentBusiness, NetherlandsNoldusBehavioral Sciences, SwedenTobiiEye-tracking device, NetherlandsMindMediaPhysiology and Biofeedback, USABiopacPhysiology, United StatesETTSense of smell/Taste stimulator and other productsof domestic concessionsAgentBusiness.The high-technology products operated byServedinIncluding Tsinghua University, Beijing Normal University, Northeast Normal University, Yanshan University, Qiyuan Laboratory, Xi'an University of Architecture and Technology, Northwest Agriculture and Forestry University, Shenzhen University of Technology, Xi'an University of Science and Technology, Shanghai University, Aerospace Second Institute, ,China Electronics Technology Group27theand28Institute, etc.Top universities and highest level research units in China, , ,Also forMozi, NetEase, Huawei Technologiesproviding technical support.In talent training, scientific research cooperation, and transformation of resultsWithin the field of technologyOngoing deepdegreeCooperation.

This article comes from the WeChat public number: EVERLOYAL

Attentional state recognition in virtual reality environments

Partners

University

Scientific Research Institutes:

Company:

Contact information

Follow us