Study of Visual Hybrid Brain-Computer Interface Based on Wearable Magnetoencephalogram

doi:10.11938/cjmr20243096

Abstract

Abstract:

The emerging wearable magnetoencephalography technology lays the foundation for brain-computer interface to provide high-quality data. To explore the feasibility of applying wearable magnetoencephalography in visual hybrid brain-computer interface, a SSVEF-Alpha hybrid brain-computer interface is designed based on steady-state visual evoked field and Alpha wave, and the performance is compared with different classification models. The results show that based on the user-dependent training method, the average classification accuracy of hybrid brain-computer interface is (93.29±1.69)%, the information transmission rate can reach 86.81 bits/min. And the user-independent training method with short data length shows superiority over the training-free method. This study verifies the effectiveness of visual hybrid brain-computer interface and provides a reference example for further development and design of brain-computer interface products of wearable magnetoencephalography.

Key words: brain-computer interface, wearable magnetoencephalogram, steady-state visual evoked field, Alpha wave, classification accuracy, information transmission rate

CLC Number:

O441.5
Q64

WANG Chenxu, GUO Xu, WANG Hui, ZHANG Xin, CHANG Yan, GUO Qingqian, HU Tao, FENG Xiaoyu, YANG Xiaodong. Study of Visual Hybrid Brain-Computer Interface Based on Wearable Magnetoencephalogram[J]. Chinese Journal of Magnetic Resonance, 2024, 41(4): 405-417.

Figures/Tables 9

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Table 1

Fig. 5

Table 2

Fig. 6

Table 3

References 35

[1]	BENABID A L, COSTECALDE T, ELISEYEV A, et al. An exoskeleton controlled by an epidural wireless brain-machine interface in a tetraplegic patient: a proof-of-concept demonstration[J]. Lancet Neurol, 2019, 18(12): 1112-1122. doi: S1474-4422(19)30321-7 pmid: 31587955
[2]	ZHAI W W, YANG Y N, LU S Y, et al. Brain computer interface system research of upper limb rehabilitation training robot[J]. Journal of Biomedical Engineering Research, 2019, 38(3):269-274.
	翟文文, 杨玉娜, 鲁守银, 等. 上肢康复训练机器人的脑机接口系统研究[J]. 生物医学工程研究, 2019, 38(3): 269-274.
[3]	ANUMANCHIPALLI G K, CHARTIER J, CHANG E F. Speech synthesis from neural decoding of spoken sentences[J]. Nature, 2019, 568(7753): 493-498.
[4]	LEEB R, FRIEDMAN D, MüLLER-PUTZ G R, et al. Self-paced (asynchronous) BCI control of a wheelchair in virtual environments: a case study with a tetraplegic[J]. Comput Intell Neurosci, 2007, 2007: 79642.
[5]	ZRENNER C, DESIDERI D, BELARDINELLI P, et al. Real-time EEG-defined excitability states determine efficacy of TMS-induced plasticity in human motor cortex[J]. Brain Stimul, 2018, 11(2): 374-389. doi: S1935-861X(17)30972-5 pmid: 29191438
[6]	LIU H, DU Y X, PENG J, et al. A review of brain-computer interface development[J]. 2011, 24(5): 116-119.
	刘辉, 杜玉晓, 彭杰, 等. 脑-机接口技术发展[J]. 电子科技, 2011, 24(5): 116-119.
[7]	拉杰什P.N.拉奥. 脑机接口导论[M]. 张莉, 陈民铀, 译. 脑机接口导论, 2016.
[8]	VANSTEENSEL M J, PELS E G M, BLEICHNER M G, et al. Fully implanted brain-computer interface in a locked-in patient with ALS[J]. N Engl J Med, 2016, 375(21): 2060-2066.
[9]	PANDARINATH C, NUYUJUKIAN P, BLABE C H, et al. High performance communication by people with paralysis using an intracortical brain-computer interface[J]. Elife, 2017, 6: e18554.
[10]	KRUSIENSKI D J, SHIH J J. Control of a brain-computer interface using stereotactic depth electrodes in and adjacent to the hippocampus[J]. J Neural Eng, 2011, 8: 025006.
[11]	LOPES DA SILVA F. EEG and MEG: Relevance to neuroscience[J]. Neuron, 2013, 80(5): 1112-1128. doi: 10.1016/j.neuron.2013.10.017 pmid: 24314724
[12]	HEDRICH T, PELLEGRINO G, KOBAYASHI E, et al. Comparison of the spatial resolution of source imaging techniques in high-density EEG and MEG[J]. NeuroImage, 2017, 157: 531-544. doi: S1053-8119(17)30491-3 pmid: 28619655
[13]	IIVANAINEN J, STENROOS M, PARKKONEN L. Measuring MEG closer to the brain: Performance of on-scalp sensor arrays[J]. NeuroImage, 2017, 147: 542-553. doi: S1053-8119(16)30770-4 pmid: 28007515
[14]	BOTO E, HOLMES N, LEGGETT J, et al. Moving magnetoencephalography towards real-world applications with a wearable system[J]. Nature, 2018, 555(7698): 657-661.
[15]	ZHANG S L, CAO N. A synthetic optically pumped gradiometer for magnetocardiography measurements[J]. Chin Phys B, 2020, 29(4): 040702.
[16]	TIERNEY T M, HOLMES N, MELLOR S, et al. Optically pumped magnetometers: From quantum origins to multi-channel magnetoencephalography[J]. NeuroImage, 2019, 199: 598-608. doi: S1053-8119(19)30455-0 pmid: 31141737
[17]	CHEN C Q, ZHANG X, GUO Q Q, et al. Moving wearable magnetoencephalography measurement study based on optically-pumped magnetometer[J]. Chinese J Magn Reson, 2022, 39(3): 337-344.
	陈春巧, 张欣, 郭清乾, 等. 基于原子磁力计的穿戴式脑磁图动态测量研究[J]. 波谱学杂志, 2022, 39(3): 337-344. doi: 10.11938/cjmr20222975
[18]	SUN W, WANG H, ZHANG Y, et al. Optimal design for quantification of gas concentration based olfactory stimulator[J]. Chinese J Magn Reson, 2021, 38(1):12-21.
	孙韦, 王慧, 张寅, 等. 基于气体浓度定量的嗅觉刺激器优化设计[J]. 波谱学杂志, 2021, 38(1): 12-21.
[19]	REGAN D. Steady-state evoked potentials[J]. J Opt Soc Am, 1977, 67(11): 1475-1489. pmid: 411904
[20]	ZHIGALOV A, HERRING J D, HERPERS J, et al. Probing cortical excitability using rapid frequency tagging[J]. NeuroImage, 2019, 195: 59-66. doi: S1053-8119(19)30256-3 pmid: 30930309
[21]	BRICKWEDDE M, SCHMIDT M D, KRÜGER M C, et al. 20 Hz steady-state response in somatosensory cortex during induction of tactile perceptual learning through LTP-like sensory stimulation[J]. Front Hum Neurosci, 2020, 14: 257. doi: 10.3389/fnhum.2020.00257 pmid: 32694988
[22]	DEWAN E M. Occipital Alpha rhythm eye position and lens accommodation[J]. Nature, 1967, 214(5092): 975-977.
[23]	LI L, ZHANG J X. Study of the alpha wave differences between eyes-closed and eyes-open resting states[J]. Journal of University of Electronic Science and Technology of China, 2010, 39(3): 450-453.
	李凌, 张金香. 闭眼与开眼静息状态下脑电α波的差异研究[J]. 电子科技大学学报, 2010, 39(3): 450-453.
[24]	KIRKUP L, SEARLE A, CRAIG A, et al. EEG-based system for rapid on-off switching without prior learning[J]. Med Biol Eng Comput, 1997, 35(5): 504-509. pmid: 9374055
[25]	WITTEVRONGEL B, HOLMES N, BOTO E, et al. Practical real-time MEG-based neural interfacing with optically pumped magnetometers[J]. BMC Biol, 2021, 19(1): 158. doi: 10.1186/s12915-021-01073-6 pmid: 34376215
[26]	LI X, CHEN J, SHI N, et al. A hybrid steady-state visual evoked response-based brain-computer interface with MEG and EEG[J]. Expert Syst Appl, 2023, 223: 119736.
[27]	RAVI A, BENI N H, MANUEL J, et al. Comparing user-dependent and user-independent training of CNN for SSVEP BCI[J]. J Neural Eng, 2020, 17(2): 026028.
[28]	ZHAO D, WANG T, TIAN Y, et al. Filter bank convolutional neural network for SSVEP classification[J]. IEEE Access, 2021, 9: 147129-147141.
[29]	BRAINARD D H. The psychophysics toolbox[J]. Spat Vis, 1997, 10(4): 433-436.
[30]	VAPNIK V. Statistical learning theory[M]. New York: Wiley, 1998.
[31]	NAKANISHI M, WANG Y, WANG Y T, et al. A comparison study of canonical correlation analysis based methods for detecting steady-state visual evoked potentials[J]. PLoS One, 2015, 10(10): e0140703.
[32]	TANAKA H, KATURA T, SATO H. Task-related component analysis for functional neuroimaging and application to near-infrared spectroscopy data[J]. NeuroImage, 2013, 64: 308-327. doi: 10.1016/j.neuroimage.2012.08.044 pmid: 22922468
[33]	LAWHERN V J, SOLON A J, WAYTOWICH N R, et al. EEGNet: a compact convolutional neural network for EEG-based brain-computer interfaces[J]. J Neural Eng, 2018, 15(5): 056013.
[34]	WOLPAW J R, RAMOSER H, MCFARLAND D J, et al. EEG-based communication: improved accuracy by response verification[J]. IEEE Trans Rehabil Eng, 1998, 6(3): 326-333. doi: 10.1109/86.712231 pmid: 9749910
[35]	ZHANG Y, VALSECCHI M, GEGENFURTNER K R, et al. The time course of chromatic adaptation in human early visual cortex revealed by SSVEPs[J]. J Vis, 2023, 23(5): 17.

分类方法	数据长度
分类方法	1 s	2 s	3 s	4 s	5 s
UD-SVM(2)	94.13±1.26	94.50±1.45	95.63±1.45	95.38±1.66	96.38±1.42
UD-TRCA(2)	52.88±2.47	52.38±2.31	57.75±2.62	54.50±2.37	57.50±2.58
UD-EEGNet(2)	95.63±1.59	97.50±1.07	93.88±1.97	93.00±2.22	93.75±2.24
UI-SVM(2)	81.63±3.67	82.50±3.95	83.50±3.90	83.13±3.86	83.50±4.07
UI-TRCA(2)	49.13±1.56	51.50±1.72	55.38±2.27	54.00±1.72	55.63±1.72
UI-EEGNet(2)	83.13±3.28	85.25±3.35	85.13±3.40	85.50±3.35	86.13±3.38
CCA(4)	49.44±2.50	75.81±3.16	87.13±2.37	92.00±1.73	93.56±1.64
UD-TRCA(4)	89.19±1.81	96.25±1.09	97.50±0.88	98.19±0.85	99.00±0.43
UD-EEGNet(4)	88.56±2.07	88.69±2.60	95.69±1.28	96.75±1.19	97.63±1.07
UI-TRCA(4)	56.44±2.47	59.06±3.16	61.69±3.92	63.63±4.20	67.31±4.02
UI-EEGNet(4)	67.50±2.99	79.31±3.27	82.25±3.56	85.25±3.48	88.13±3.23
UD-TRCA(6)	65.00±1.66	71.71±1.23	73.92±1.29	73.88±1.29	75.33±1.18
UD-EEGNet(6)	81.10±2.16	88.04±1.48	90.42±1.31	91.67±1.50	93.29±1.69
UI-TRCA(6)	37.54±1.74	40.29±2.57	44.08±2.95	45.71±3.05	48.54±3.21
UI-EEGNet(6)	52.67±2.33	62.37±3.00	73.63±3.02	78.71±2.73	81.75±2.75

分类方法	数据长度
分类方法	1 s	2 s	3 s	4 s	5 s
UD-SVM(2)	40.65	20.78	14.82	10.95	9.30
UD-TRCA(2)	0.14	0.05	0.35	0.09	0.20
UD-EEGNet(2)	44.45	24.94	13.35	9.51	7.95
UI-SVM(2)	18.71	9.93	7.08	5.18	4.25
UI-TRCA(2)	0.01	0.02	0.17	0.07	0.11
UI-EEGNet(2)	20.71	11.89	7.87	6.04	5.03
CCA(4)	11.92	24.55	24.84	22.07	18.64
UD-TRCA(4)	80.06	51.30	35.83	27.61	22.84
UD-EEGNet(4)	78.34	39.34	33.50	26.13	21.60
UI-TRCA(4)	19.29	11.25	8.65	7.17	6.84
UI-EEGNet(4)	34.51	28.10	20.88	17.44	15.43
UD-TRCA(6)	50.29	32.06	23.03	17.25	14.48
UD-EEGNet(6)	86.81	53.37	38.14	29.66	24.89
UI-TRCA(6)	10.80	6.78	5.93	4.95	4.69
UI-EEGNet(6)	29.28	22.68	22.80	20.15	17.71

数据长度	专用分类方法			通用分类方法
数据长度	Alpha波	SSVEF信号	SSVEF-Alpha 混合信号	Alpha波	SSVEF信号	SSVEF-Alpha 混合信号
1 s	UD-EEGNet(2)	UD-TRCA(4)	UD-EEGNet(6)	UI-EEGNet(2)	UI-EEGNet(4)	UI-EEGNet(6)
2 s	UD-EEGNet(2)	UD-TRCA(4)	UD-EEGNet(6)	UI-EEGNet(2)	UI-EEGNet(4)	UI-EEGNet(6)
3 s	UD-SVM(2)	UD-TRCA(4)	UD-EEGNet(6)	UI-EEGNet(2)	CCA(4)	UI-EEGNet(6)
4 s	UD-SVM(2)	UD-TRCA(4)	UD-EEGNet(6)	UI-EEGNet(2)	CCA(4)	UI-EEGNet(6)
5 s	UD-SVM(2)	UD-TRCA(4)	UD-EEGNet(6)	UI-EEGNet(2)	CCA(4)	UI-EEGNet(6)