基于可穿戴式脑磁图的视觉混合脑机接口研究
Study of Visual Hybrid Brain-Computer Interface Based on Wearable Magnetoencephalogram
通讯作者: * Tel: 18900616030, E-mail:xiaodongyang@sibet.ac.cn.
收稿日期: 2024-02-2 网络出版日期: 2024-03-11
基金资助: |
|
Corresponding authors: * Tel: 18900616030, E-mail:xiaodongyang@sibet.ac.cn.
Received: 2024-02-2 Online: 2024-03-11
新兴的可穿戴式脑磁图技术为脑机接口(BCI)提供高质量数据奠定了基础.为探究可穿戴式脑磁图应用于视觉混合BCI上的可行性,本文基于稳态视觉诱发场(SSVEF)和Alpha波设计了SSVEF-Alpha混合BCI,并在不同分类模型上进行了性能对比.结果表明,基于用户依赖(UD)的训练方法,混合BCI 6分类平均分类准确率为(93.29±1.69)%,信息传输速率可达86.81 bits/min,且使用短数据长度进行用户独立(UI)的训练方法比免训练的方法更具优越性.本研究验证了视觉混合BCI的有效性,为进一步开发设计可穿戴式脑磁图的BCI应用产品提供参考范例.
关键词:
The emerging wearable magnetoencephalography technology lays the foundation for brain-computer interface to provide high-quality data. To explore the feasibility of applying wearable magnetoencephalography in visual hybrid brain-computer interface, a SSVEF-Alpha hybrid brain-computer interface is designed based on steady-state visual evoked field and Alpha wave, and the performance is compared with different classification models. The results show that based on the user-dependent training method, the average classification accuracy of hybrid brain-computer interface is (93.29±1.69)%, the information transmission rate can reach 86.81 bits/min. And the user-independent training method with short data length shows superiority over the training-free method. This study verifies the effectiveness of visual hybrid brain-computer interface and provides a reference example for further development and design of brain-computer interface products of wearable magnetoencephalography.
Keywords:
本文引用格式
王晨旭, 郭旭, 王慧, 张欣, 常严, 郭清乾, 胡涛, 冯晓宇, 杨晓冬.
WANG Chenxu, GUO Xu, WANG Hui, ZHANG Xin, CHANG Yan, GUO Qingqian, HU Tao, FENG Xiaoyu, YANG Xiaodong.
引言
脑机接口(brain-computer interfaces,BCI)旨在为大脑和外部设备(计算机或其他电子设备)之间建立直接通信,而不依赖于任何肌肉或外周神经.BCI技术的应用有助于患有肌萎缩性侧索硬化、脑卒中、脑瘫、脊髓损伤等“闭锁综合征”的病患恢复日常交流和生活的能力.近年来,随着BCI技术的不断发展,其已从医学领域小众应用[1⇓-3]逐步走向虚拟现实、人工智能、航空航天等非医学领域[4⇓⇓-7].BCI技术取得成功的一个关键是能够依托高新技术设备获取高质量的脑神经活动测量数据,如皮层脑电(electrocorticography,ECoG)[8]、微电极阵列[9]或深度电极[10]等;然而,目前主流的BCI技术需要在人脑中侵入式的植入电极,具有一定安全风险,也从本质上极大限制了终端用户和应用场景.此外,常见的非侵入式脑电图(electroencephalogram,EEG)技术虽可以从头皮无创地记录脑电信号,但其信号经过脑脊液、头骨和皮肤的传导后而严重失真,导致信息含量较低,且脑电图在使用时需借助凝胶或生理盐水来降低头皮和电极之间的阻抗,因此数据采集前期准备时间较长.
与前述技术手段不同,非侵入式脑磁图(magnetoencephalography,MEG)技术探测到的脑磁信号受不均匀导电性扭曲得很少,且对肌肉活动产生的伪迹敏感性较低.因此,MEG不仅能够检测到脑神经活动引起的微小磁场变化,而且与EEG相比,MEG能够提供更高空间分辨率的信息[11,12].传统MEG测量使用的传感器主要是超导量子干涉仪(superconductor quantum interference devices,SQUID),而一种新型原子磁力计——无自旋交换弛豫(spin-exchange relaxation-free,SERF)光泵磁强计(optically-pumped magnetometer,OPM),作为新一代量子弱磁传感技术,与SQUID技术相比,具有小型化、紧贴头皮、无需液氦、室温检测等优势[13⇓⇓-16],将OPM传感器以阵列形式安装在脑磁头盔上,能够实现可穿戴式脑磁图.进一步地,借助磁屏蔽房和匀场补偿线圈,极大程度上降低了外界磁场的干扰,使得OPM传感器对受试者动态脑磁测量下产生的伪影不敏感,进而实现受试者在自然状态下视觉、听觉、触觉、嗅觉等脑磁信号的探测[17,18].
稳态视觉诱发电位或场(steady-state visual evoked potential/field,SSVEP/SSVEF)是BCI应用中常见的类型之一,其特点是人脑在特定频率重复闪烁的视觉刺激下,将诱发出与闪烁刺激具有相同基频及其谐波的神经元信号[19⇓-21].Alpha波是BCI早期研究常见的类型,是一种与视觉皮层闲散节律相对应的自发节律信号,频率为8~13 Hz,在人清醒闭目时出现,睁眼或接受其它刺激时消失[22,23].两种神经活动均在枕部最为明显,且几乎不需要用户训练.1997年,悉尼科技大学的Kirkup等[24]利用Alpha波的特性,设计了一种无需用户训练的快速开关系统,有助于肢体残疾人士更好地控制周围环境.2021年,Wittevrongel等[25]利用OPM技术,设计了基于SSVEF的“思维拼写”应用程序,并达到了97.7%的平均准确率.2023年,清华大学的Li等[26]提出一种基于EEG和SQUID的混合BCI系统,将分类准确率从50%提高到95%,突显了混合BCI方法的优越性.
1 实验方法
1.1 受试者
共有20名健康受试者参加了脑磁图实验(7名男性和13名女性,年龄22~29岁),均为右利手,没有已知的神经或精神障碍病史.所有受试者进行实验时,身体上或体内均不携带金属,视力正常或矫正视力正常.实验前所有受试者均签署了知情同意书,并支付了被试费.
1.2 实验设计
受试者佩戴气动耳机坐在磁屏蔽室内,距离投影屏幕50 cm,他们依据耳机内的声音提示及投影屏幕上的画面提示完成相应任务.本研究共计设定了6类指令,包括注视投影屏幕上带方向箭头的4个圆形闪烁刺激和不注视屏幕保持睁、闭眼.每个圆形闪烁刺激在水平和垂直维度上的视角为7.44˚,方向箭头在水平或垂直维度上的视角为1.3˚,刺激界面设置如图1(a)所示,视角的计算公式为:
(1)式中M为视窗大小,即投影屏幕上显示的图像大小,D为观察距离,即受试者与投影屏幕之间的距离,R为计算所得弧度,(2)式中angle为视角,即将R进行弧度到角度的换算.根据闪烁频率的不同,4个闪烁刺激被放置在投影屏幕的顶部(8.6 Hz)、左侧(12 Hz)、右侧(15 Hz)、底部(20 Hz),在刺激过程中,所有目标都以预定的频率同时闪烁,为避免视觉疲劳,以绿色作为闪烁刺激.视觉刺激使用ProPixx投影仪产生,分辨率为1 920×1 080像素,刷新率为60 Hz.视觉刺激使用光学镜片系统投射到屏幕上.刺激程序基于心理物理学工具箱Psychtoolbox Version3(PTB-3)开发[29],刺激器经过测试,能够确保稳定的刺激频率.
每位受试者接受一次实验需完成20个block,每个block中6类任务以随机顺序执行1次,共计进行120次试验(20个block×6类任务).每一次试验开始前都有2 s的任务提示(方向箭头变红和语音提醒),以便受试者依据指令做好准备,试验持续5 s后休息3 s,然后进行下一次试验.实验设计如图1(b)所示.
图1
图1
(a)刺激界面设置;(b)SSVEF-Alpha混合BCI实验设计
Fig. 1
(a) Stimulus interface setting; (b) Experimental design of SSVEF-Alpha hybrid brain computer interface
1.3 脑磁数据采集
图2
图2
(a)自主搭建的可穿戴式脑磁图系统;(b)OPM传感器阵列排布示意图
Fig. 2
(a) Self-built wearable MEG system; (b) Schematic diagram of OPM sensor array arrangement
柔性脑磁头盔表面共排布60个插槽用于安装OPM传感器,头盔可根据受试者头围尺寸选择不同型号,使传感器紧贴头皮.6个OPM传感器各作为一个独立单元安装在受试者大脑的枕骨区,覆盖了预估SSVEF及Alpha波最强的头皮区域.图2(b)为OPM传感器阵列排布(数字1~6为OPM传感器编号)相对于受试者大脑表面的示意图.本实验中,各受试者均在自然状态下进行脑磁信号探测,测量了垂直于头皮表面的磁场分量,并以1 024 Hz的采样率同步记录OPM传感器阵列原始数据以及各次试验开始时刻.
1.4 数据分析
BCI技术的关键步骤为信号处理和解码,即采集到的脑信号经过一系列的预处理和分类识别算法,提取出有效信息,并解码得出该信号最大概率的类别.本研究就涉及到的脑磁数据预处理和4种分类识别算法(基于共空间模式的支持向量机、典型相关分析、任务相关成分分析、EEGNet——一种紧凑型卷积神经网络)分别作详细介绍.
1.4.1 数据预处理
实验过程中收集到的原始脑磁数据使用四阶零相位巴特沃斯滤波器在6~80 Hz之间进行滤波,并在 25 Hz、50 Hz、75 Hz处进行陷波滤波,以滤除工频干扰及设备固有干扰.相对于t=0 s的试验开始时刻,提取其-1~5 s之间的数据作为1个数据段,并标记相应指令的类别.各数据段通过减去其-1~0 s的平均信号,对单个试验进行基线校正.然后选用不同的窗宽,以t=0 s为起始点对基线校正后的数据段进行分割:1 s、2 s、3 s、4 s、5 s,以模拟受试者接受不同时长的实验.
1.4.2 基于共空间模式的支持向量机
共空间模式(Common Spatial Pattern,CSP)算法适用于2分类,其基本原理是利用矩阵的对角化,找到一组最优空间滤波器进行投影,使得类内间距尽可能小,类间间距尽可能大,即两类信号的方差值差异最大化,从而得到较高区分度的特征向量.
支持向量机(Support Vector Machine,SVM)由Vapnik[30]提出,能够通过非线性映射的方法将低维空间线性不可分的样本映射到高维特征空间,通过构建特征空间中的最优分割超平面,使得不同类别的数据尽可能明显的间隔出来,具有良好的实际应用能力.本研究使用CSP算法进行8维特征提取,采用Matlab软件自带的fitcsvm函数训练相关参数,实现数据2分类.
1.4.3 典型相关分析
典型相关分析(Canonical Correlation Analysis,CCA)[31]是一种多元统计分析方法,用于衡量两组多维变量之间的潜在相关性.先前的研究表明,CCA在识别SSVEP信号方面具有优越的性能,且无需训练数据.CCA是将多维的矩阵X和Y线性变换为一维的
其中上标T表示矩阵的转置运算,(1)式可用奇异值分解(SVD)或其他方式求解.
在本研究SSVEF识别中,
1.4.4 任务相关成分分析
任务相关成分分析(Task-Related Component Analysis,TRCA)[32]是一种最大化任务期间的再现性来提取任务相关成分的方法.TRCA针对信号的空间信息,着重于空间滤波,用于提高信号的信噪比.脑磁信号
所有可能的试验组合总结为:
为了获得有限解,
约束优化问题可以求解为:
本研究进行信号识别时,分别使用受试者各类信号的训练数据,通过TRCA得到对应的空间滤波器
1.4.5 EEGNet
EEGNet[33]是一种紧凑的卷积神经网络,它使用时间卷积和深度卷积分别充当带通频率滤波器和空间滤波器,以降低数据的维度,然后使用可分离卷积和逐点卷积来总结特征映射并进行最优混合.该模型有几个参数:F1、F2分别控制要学习的时间滤波器和点滤波器的数量,D控制每个时间卷积内要学习的空间滤波器的数量,dropout层按照一定的概率从卷积神经网络中随机丢弃神经元,帮助正则化,防止过拟合.本研究应用EEGNet网络模型时,先对每个数据段下采样至128 Hz.使用GitHub(
EEGNet模型训练过程中需要对参数更新迭代,使用Adam优化器,最小化分类交叉熵损失函数,学习率设置为0.000 1,批大小(batch size)为64,运行500个训练迭代(epoch)并执行验证停止,选择最低验证集损失的模型.
2 实验结果
2.1 时频分析
图3
图3
(a)脑磁信号功率谱密度;(b)脑地形图的功率分布图
Fig. 3
(a) The power spectral density of brain magnetic signal; (b) The power distribution map of brain topographic map
2.2 分类准确率
本研究分别对Alpha波、SSVEF信号以及SSVEF-Alpha混合信号进行2分类、4分类、6分类准确率计算.免训练的方法不需要收集任何训练数据,作为一个通用的模型,可以立刻开始信号分类,稳态视觉信号分类最广泛使用的免训练方法是CCA,其也作为分类算法的基线算法.UD的方法需要依据每个受试者的训练数据,生成适用于个人的专用模型,UI的方法则依据多个受试者的训练数据,生成一个通用的模型,以便应用于未训练过的用户.3种分类方法示意图如图4所示.在这3种模型实际使用过程中,UI模型仅次于免训练的模型,因为它不需要再对新的受试者收集任何训练数据.
图4
本研究对于UD的方法,使用单个受试者的数据训练模型,并在同一受试者的数据上测试模型,每位受试者的数据集进行10折交叉验证.对于UI的方法,采用留一法对模型进行训练和测试,即20位受试者的数据集,通过组合19位受试者的数据来训练,剩余1位受试者的数据进行测试.为区分使用不同分类算法的多分类结果,以“分类方法-分类算法(分类数)”对分类方法重新命名.例如:UD-SVM(2)为使用SVM分类算法,依据个人的训练数据生成用户依赖的模型后,使用该模型对测试数据进行2分类。
2.2.1 Alpha波2分类
表1 不同数据长度时分类准确率(平均正确率±标准误)/%
Table 1
分类方法 | 数据长度 | ||||
---|---|---|---|---|---|
1 s | 2 s | 3 s | 4 s | 5 s | |
UD-SVM(2) | 94.13±1.26 | 94.50±1.45 | 95.63±1.45 | 95.38±1.66 | 96.38±1.42 |
UD-TRCA(2) | 52.88±2.47 | 52.38±2.31 | 57.75±2.62 | 54.50±2.37 | 57.50±2.58 |
UD-EEGNet(2) | 95.63±1.59 | 97.50±1.07 | 93.88±1.97 | 93.00±2.22 | 93.75±2.24 |
UI-SVM(2) | 81.63±3.67 | 82.50±3.95 | 83.50±3.90 | 83.13±3.86 | 83.50±4.07 |
UI-TRCA(2) | 49.13±1.56 | 51.50±1.72 | 55.38±2.27 | 54.00±1.72 | 55.63±1.72 |
UI-EEGNet(2) | 83.13±3.28 | 85.25±3.35 | 85.13±3.40 | 85.50±3.35 | 86.13±3.38 |
CCA(4) | 49.44±2.50 | 75.81±3.16 | 87.13±2.37 | 92.00±1.73 | 93.56±1.64 |
UD-TRCA(4) | 89.19±1.81 | 96.25±1.09 | 97.50±0.88 | 98.19±0.85 | 99.00±0.43 |
UD-EEGNet(4) | 88.56±2.07 | 88.69±2.60 | 95.69±1.28 | 96.75±1.19 | 97.63±1.07 |
UI-TRCA(4) | 56.44±2.47 | 59.06±3.16 | 61.69±3.92 | 63.63±4.20 | 67.31±4.02 |
UI-EEGNet(4) | 67.50±2.99 | 79.31±3.27 | 82.25±3.56 | 85.25±3.48 | 88.13±3.23 |
UD-TRCA(6) | 65.00±1.66 | 71.71±1.23 | 73.92±1.29 | 73.88±1.29 | 75.33±1.18 |
UD-EEGNet(6) | 81.10±2.16 | 88.04±1.48 | 90.42±1.31 | 91.67±1.50 | 93.29±1.69 |
UI-TRCA(6) | 37.54±1.74 | 40.29±2.57 | 44.08±2.95 | 45.71±3.05 | 48.54±3.21 |
UI-EEGNet(6) | 52.67±2.33 | 62.37±3.00 | 73.63±3.02 | 78.71±2.73 | 81.75±2.75 |
图5
图5
(a) Alpha波平均分类准确率;(b) SSVEF信号平均分类准确率;(c) SSVEF-Alpha混合信号平均分类准确率. 图中黑色加粗实线分别为各分类方法对应的偶然性水平线(计算方法:
Fig. 5
(a) Average classification accuracy of Alpha wave; (b) Average classification accuracy of SSVEF signals; (c) Average classification accuracy of SSVEF-Alpha mixed signals. The black bold solid lines in the figure are the accidental horizontal lines corresponding to each classification method respectively (calculation method:
2.2.2 SSVEF信号4分类
2.2.3 SSVEF-Alpha混合信号6分类
2.3 统计学分析
为评估同一范式下不同分类方法的性能,采用双因素方差分析对分类准确率的结果进行了评估.设定分类准确率为因变量,数据长度、分类方法为固定因子,零假设为所有分类方法及数据长度的分类准确率相同,同时设定95%的置信区间用于比较和分析,并进一步进行Bonferroni事后多重比较.
对于Alpha波的分类准确率,双因素方差分析显示分类方法(p<0.05)存在显著影响,而数据长度(p=0.390)则不存在显著影响.Bonferroni事后多重比较表明,UD-SVM(2)与UD-EEGNet(2)、UD-TRCA(2)与UI-TRCA(2)、UI-SVM(2)与UI-EEGNet(2)分类方法相较,差异均无统计学意义(p>0.05).各数据长度两两之间相较,分类准确率的差异也均无统计学意义(p>0.05),结合图5(a)可知数据长度对分类准确率的影响不大,分类准确率的高低依赖于分类方法.
对于SSVEF信号的分类准确率,双因素方差分析显示分类方法和数据长度均存在显著影响(p<0.05).Bonferroni事后多重比较表明,CCA(4)与UI-EEGNet(4)、UD-TRCA(4)与UD-EEGNet(4)分类方法相较,差异无统计学意义(p>0.05).对同一分类方法不同数据长度之间两两相较,数据长度大于2 s后,分类准确率的差异无统计学意义(p>0.05),结合图5(b)可知同一分类方法2 s后的分类准确率趋于稳定.为比较1 s数据长度时UI模型与免训练CCA模型的性能,对CCA(4)、UI-TRCA(4)、UI-EEGNet(4)等分类方法做单因素方差分析,结果表明,CCA(4)与UI-EEGNet(4)分类方法相较,UI-EEGNet(4)更具优越性(p<0.05).
对于SSVEF-Alpha混合信号的分类准确率,双因素方差分析显示分类方法和数据长度均存在显著影响(p<0.05).Bonferroni事后多重比较表明,UD-TRCA(6)与UI-EEGNet(6)分类算法相较,差异无统计学意义(p>0.05).数据长度3 s、4 s、5 s之间两两相比,差异也无统计学意义(p>0.05),结合图5(c)可知UD、UI结合不同的分类算法可以展现出相似的分类性能,同一分类方法的分类准确率在3 s后趋于稳定.
2.4 信息传输速率
除分类准确率外,信息传输速率(ITR)[34]也是评价BCI性能的重要参数,用于表示每分钟传输数据信息的数目,它考虑了分类准确率、刺激目标数和数据长度之间的关系.ITR(bits/min)公式为:
表2 不同数据长度信息传输速率/(bits/min)
Table 2
分类方法 | 数据长度 | ||||
---|---|---|---|---|---|
1 s | 2 s | 3 s | 4 s | 5 s | |
UD-SVM(2) | 40.65 | 20.78 | 14.82 | 10.95 | 9.30 |
UD-TRCA(2) | 0.14 | 0.05 | 0.35 | 0.09 | 0.20 |
UD-EEGNet(2) | 44.45 | 24.94 | 13.35 | 9.51 | 7.95 |
UI-SVM(2) | 18.71 | 9.93 | 7.08 | 5.18 | 4.25 |
UI-TRCA(2) | 0.01 | 0.02 | 0.17 | 0.07 | 0.11 |
UI-EEGNet(2) | 20.71 | 11.89 | 7.87 | 6.04 | 5.03 |
CCA(4) | 11.92 | 24.55 | 24.84 | 22.07 | 18.64 |
UD-TRCA(4) | 80.06 | 51.30 | 35.83 | 27.61 | 22.84 |
UD-EEGNet(4) | 78.34 | 39.34 | 33.50 | 26.13 | 21.60 |
UI-TRCA(4) | 19.29 | 11.25 | 8.65 | 7.17 | 6.84 |
UI-EEGNet(4) | 34.51 | 28.10 | 20.88 | 17.44 | 15.43 |
UD-TRCA(6) | 50.29 | 32.06 | 23.03 | 17.25 | 14.48 |
UD-EEGNet(6) | 86.81 | 53.37 | 38.14 | 29.66 | 24.89 |
UI-TRCA(6) | 10.80 | 6.78 | 5.93 | 4.95 | 4.69 |
UI-EEGNet(6) | 29.28 | 22.68 | 22.80 | 20.15 | 17.71 |
图6
图6
(a) Alpha波信息传输速率(UD-TRCA(2)与UI-TRCA(2)存在重叠);(b) SSVEF信号信息传输速率;(c) SSVEF-Alpha混合信号信息传输速率
Fig. 6
(a) The information transmission rate of Alpha wave (The lines of UD-TRCA(2) and UI-TRCA(2) are partly overlaid); (b) The information transmission rate of SSVEF signals; (c) The information transmission rate of SSVEF-Alpha mixed signals
综上所述,以分类方法是否对新受试者通用为条件,对单一范式BCI及混合范式BCI不同数据长度最优的分类方法(高准确率、高信息传输速率)进行总结,如表3所示.
表3 不同数据长度最优分类方法
Table 3
数据长度 | 专用分类方法 | 通用分类方法 | |||||
---|---|---|---|---|---|---|---|
Alpha波 | SSVEF信号 | SSVEF-Alpha 混合信号 | Alpha波 | SSVEF信号 | SSVEF-Alpha 混合信号 | ||
1 s | UD-EEGNet(2) | UD-TRCA(4) | UD-EEGNet(6) | UI-EEGNet(2) | UI-EEGNet(4) | UI-EEGNet(6) | |
2 s | UD-EEGNet(2) | UD-TRCA(4) | UD-EEGNet(6) | UI-EEGNet(2) | UI-EEGNet(4) | UI-EEGNet(6) | |
3 s | UD-SVM(2) | UD-TRCA(4) | UD-EEGNet(6) | UI-EEGNet(2) | CCA(4) | UI-EEGNet(6) | |
4 s | UD-SVM(2) | UD-TRCA(4) | UD-EEGNet(6) | UI-EEGNet(2) | CCA(4) | UI-EEGNet(6) | |
5 s | UD-SVM(2) | UD-TRCA(4) | UD-EEGNet(6) | UI-EEGNet(2) | CCA(4) | UI-EEGNet(6) |
3 讨论
本研究通过SSVEF-Alpha混合BCI实验对比了SSVEF和Alpha节律的信号特征,并进行了分类识别研究.在此基础上,进一步验证了混合BCI应用于可穿戴式脑磁图的可行性.判断BCI性能的两个重要指标分别为分类准确率和信息传输速率,越高的分类准确率代表BCI的可行性越高,越高的信息传输速率代表BCI的实用性越强.
利用多种分类方法对受试者的单一范式信号和联合范式信号进行分类准确率计算,从多分类的结果来看,分类准确率仍有很大的上升空间,不同范式下的信号都有其最适合的分类方法,但基于深度学习的EEGNet在多分类中均展现出较为优越的性能.TRCA算法不能有效进行2分类,可能是Alpha波涉及频率较多,任务期间单频率再现性较差,从而无法有效进行空间滤波,进而也限制了TRCA算法6分类准确率的上限.总体结果表明,由于个体间存在差异,与UI和免训练CCA相比,UD具有更高的分类准确性,但这需要每位受试者都完成一次训练,可能会导致受试者依从性差等问题.UI与免训练CCA在短数据长度分类时相比有更佳的表现,未来的研究可以探索使用更多受试者的数据,根据信号的特点对算法进行调整或采用其他算法建立通用的模型,进一步提升BCI系统的可行性.从统计分析的结果来看,Alpha波2分类,不同数据长度在同一分类算法下的分类准确率差距不大,而SSVEF信号在同一分类算法下,数据长度2 s后的分类准确率差距不大,这可能是Alpha波自发节律信号可以在较短的时间内达到稳定状态,而SSVEF诱发信号则需要随着刺激时间的延长逐步达到稳定[35].从信息传输速率的结果来看,SSVEF-Alpha混合BCI将诱发信号与自发信号相结合,能够实现更高的通信指标,将两种范式组合为并行范式或通过检测Alpha波设计异步范式[2],可以在多任务情况下提高实际应用的价值.同时,本研究仍然存在一定的局限性.首先,SSVEF刺激范式中的部分刺激频率与Alpha波频段存在重合,使得联合刺激信号解码容易产生混淆.其次,本研究仅在枕骨区排布了6个传感器进行实验分析,在后续研究中可增加传感器的数量和扩大传感器的排布范围,并进一步分析传感器数量对BCI性能的影响.最后,基于可穿戴式脑磁图的SSVEF-Alpha混合信号分类识别目前仍处于离线阶段,缺乏进一步的验证,后续计划进行在线实验,测试其实际使用的有效性.
4 结论
本研究对基于可穿戴式脑磁图的SSVEF-Alpha混合BCI进行分类识别研究.使用多种分类方法对脑磁信号进行分类准确率和信息传输速率计算,首先验证了单一范式BCI在可穿戴脑磁图上的可行性,其次证明了与单一范式BCI相比,混合范式BCI配合有效的分类方法可以有效的节约刺激诱发时间,提高系统的信息传输速率,最后表明了短数据长度下用户独立模型与免训练模型相较更具优越性,为构建更为实用、高效的通用BCI解码模型奠定了实验基础.上述分析结果为可穿戴式脑磁图BCI系统应用者在SSVEF-Alpha混合范式的分类方法选择上,提供了比较清晰的理论方法及系统的实验参考,具有较好的参考价值和应用潜力.
利益冲突
无
参考文献
An exoskeleton controlled by an epidural wireless brain-machine interface in a tetraplegic patient: a proof-of-concept demonstration
[J]. ,DOI:S1474-4422(19)30321-7 PMID:31587955 [本文引用: 1]
Approximately 20% of traumatic cervical spinal cord injuries result in tetraplegia. Neuroprosthetics are being developed to manage this condition and thus improve the lives of patients. We aimed to test the feasibility of a semi-invasive technique that uses brain signals to drive an exoskeleton.We recruited two participants at Clinatec research centre, associated with Grenoble University Hospital, Grenoble, France, into our ongoing clinical trial. Inclusion criteria were age 18-45 years, stability of neurological deficits, a need for additional mobility expressed by the patient, ambulatory or hospitalised monitoring, registration in the French social security system, and signed informed consent. The exclusion criteria were previous brain surgery, anticoagulant treatments, neuropsychological sequelae, depression, substance dependence or misuse, and contraindications to magnetoencephalography (MEG), EEG, or MRI. One participant was excluded because of a technical problem with the implants. The remaining participant was a 28-year-old man, who had tetraplegia following a C4-C5 spinal cord injury. Two bilateral wireless epidural recorders, each with 64 electrodes, were implanted over the upper limb sensorimotor areas of the brain. Epidural electrocorticographic (ECoG) signals were processed online by an adaptive decoding algorithm to send commands to effectors (virtual avatar or exoskeleton). Throughout the 24 months of the study, the patient did various mental tasks to progressively increase the number of degrees of freedom.Between June 12, 2017, and July 21, 2019, the patient cortically controlled a programme that simulated walking and made bimanual, multi-joint, upper-limb movements with eight degrees of freedom during various reach-and-touch tasks and wrist rotations, using a virtual avatar at home (64·0% [SD 5·1] success) or an exoskeleton in the laboratory (70·9% [11·6] success). Compared with microelectrodes, epidural ECoG is semi-invasive and has similar efficiency. The decoding models were reusable for up to approximately 7 weeks without recalibration.These results showed long-term (24-month) activation of a four-limb neuroprosthetic exoskeleton by a complete brain-machine interface system using continuous, online epidural ECoG to decode brain activity in a tetraplegic patient. Up to eight degrees of freedom could be simultaneously controlled using a unique model, which was reusable without recalibration for up to about 7 weeks.French Atomic Energy Commission, French Ministry of Health, Edmond J Safra Philanthropic Foundation, Fondation Motrice, Fondation Nanosciences, Institut Carnot, Fonds de Dotation Clinatec.Copyright © 2019 Elsevier Ltd. All rights reserved.
Brain computer interface system research of upper limb rehabilitation training robot
[J]. ,
上肢康复训练机器人的脑机接口系统研究
[J]. , 2019,
Speech synthesis from neural decoding of spoken sentences
[J]. ,
Self-paced (asynchronous) BCI control of a wheelchair in virtual environments: a case study with a tetraplegic
[J]. ,
Real-time EEG-defined excitability states determine efficacy of TMS-induced plasticity in human motor cortex
[J]. ,DOI:S1935-861X(17)30972-5 PMID:29191438 [本文引用: 1]
Rapidly changing excitability states in an oscillating neuronal network can explain response variability to external stimulation, but if repetitive stimulation of always the same high- or low-excitability state results in long-term plasticity of opposite direction has never been explored in vivo.Different phases of the endogenous sensorimotor μ-rhythm represent different states of corticospinal excitability, and repetitive transcranial magnetic stimulation (rTMS) of always the same high- vs. low-excitability state results in long-term plasticity of different direction.State-dependent electroencephalography-triggered transcranial magnetic stimulation (EEG-TMS) was applied to target the EEG negative vs. positive peak of the sensorimotor μ-rhythm in healthy subjects using a millisecond resolution real-time digital signal processing system. Corticospinal excitability was indexed by motor evoked potential amplitude in a hand muscle.EEG negative vs. positive peak of the endogenous sensorimotor μ-rhythm represent high- vs. low-excitability states of corticospinal neurons. More importantly, otherwise identical rTMS (200 triple-pulses at 100 Hz burst frequency and ∼1 Hz repetition rate), triggered consistently at this high-excitability vs. low-excitability state, leads to long-term potentiation (LTP)-like vs. no change in corticospinal excitability.Findings raise the intriguing possibility that real-time information of instantaneous brain state can be utilized to control efficacy of plasticity induction in humans.Copyright © 2017 Elsevier Inc. All rights reserved.
A review of brain-computer interface development
[J].
脑-机接口技术发展
[J]. ,
Fully implanted brain-computer interface in a locked-in patient with ALS
[J]. ,
High performance communication by people with paralysis using an intracortical brain-computer interface
[J]. ,
Control of a brain-computer interface using stereotactic depth electrodes in and adjacent to the hippocampus
[J]. ,
EEG and MEG: Relevance to neuroscience
[J]. ,DOI:10.1016/j.neuron.2013.10.017 PMID:24314724 [本文引用: 1]
To understand dynamic cognitive processes, the high time resolution of EEG/MEG is invaluable. EEG/MEG signals can play an important role in providing measures of functional and effective connectivity in the brain. After a brief description of the foundations and basic methodological aspects of EEG/MEG signals, the relevance of the signals to obtain novel insights into the neuronal mechanisms underlying cognitive processes is surveyed, with emphasis on neuronal oscillations (ultra-slow, theta, alpha, beta, gamma, and HFOs) and combinations of oscillations. Three main functional roles of brain oscillations are put in evidence: (1) coding specific information, (2) setting and modulating brain attentional states, and (3) assuring the communication between neuronal populations such that specific dynamic workspaces may be created. The latter form the material core of cognitive functions. Copyright © 2013 Elsevier Inc. All rights reserved.
Comparison of the spatial resolution of source imaging techniques in high-density EEG and MEG
[J]. ,DOI:S1053-8119(17)30491-3 PMID:28619655 [本文引用: 1]
The present study aims at evaluating and comparing electrical and magnetic distributed source imaging methods applied to high-density Electroencephalography (hdEEG) and Magnetoencephalography (MEG) data. We used resolution matrices to characterize spatial resolution properties of Minimum Norm Estimate (MNE), dynamic Statistical Parametric Mapping (dSPM), standardized Low-Resolution Electromagnetic Tomography (sLORETA) and coherent Maximum Entropy on the Mean (cMEM, an entropy-based technique). The resolution matrix provides information of the Point Spread Functions (PSF) and of the Crosstalk functions (CT), this latter being also called source leakage, as it reflects the influence of a source on its neighbors.The spatial resolution of the inverse operators was first evaluated theoretically and then with real data acquired using electrical median nerve stimulation on five healthy participants. We evaluated the Dipole Localization Error (DLE) and the Spatial Dispersion (SD) of each PSF and CT map.cMEM showed the smallest spatial spread (SD) for both PSF and CT maps, whereas localization errors (DLE) were similar for all methods. Whereas cMEM SD values were lower in MEG compared to hdEEG, the other methods slightly favored hdEEG over MEG. In real data, cMEM provided similar localization error and significantly less spatial spread than other methods for both MEG and hdEEG. Whereas both MEG and hdEEG provided very accurate localizations, all the source imaging methods actually performed better in MEG compared to hdEEG according to all evaluation metrics, probably due to the higher signal-to-noise ratio of the data in MEG.Our overall results show that all investigated methods provide similar localization errors, suggesting very accurate localization for both MEG and hdEEG when similar number of sensors are considered for both modalities. Intrinsic properties of source imaging methods as well as their behavior for well-controlled tasks, suggest an overall better performance of cMEM in regards to spatial resolution and spatial leakage for both hdEEG and MEG. This indicates that cMEM would be a good candidate for studying source localization of focal and extended generators as well as functional connectivity studies.Copyright © 2017 Elsevier Inc. All rights reserved.
Measuring MEG closer to the brain: Performance of on-scalp sensor arrays
[J]. ,DOI:S1053-8119(16)30770-4 PMID:28007515 [本文引用: 1]
Optically-pumped magnetometers (OPMs) have recently reached sensitivity levels required for magnetoencephalography (MEG). OPMs do not need cryogenics and can thus be placed within millimetres from the scalp into an array that adapts to the individual head size and shape, thereby reducing the distance from cortical sources to the sensors. Here, we quantified the improvement in recording MEG with hypothetical on-scalp OPM arrays compared to a 306-channel state-of-the-art SQUID array (102 magnetometers and 204 planar gradiometers). We simulated OPM arrays that measured either normal (nOPM; 102 sensors), tangential (tOPM; 204 sensors), or all components (aOPM; 306 sensors) of the magnetic field. We built forward models based on magnetic resonance images of 10 adult heads; we employed a three-compartment boundary element model and distributed current dipoles evenly across the cortical mantle. Compared to the SQUID magnetometers, nOPM and tOPM yielded 7.5 and 5.3 times higher signal power, while the correlations between the field patterns of source dipoles were reduced by factors of 2.8 and 3.6, respectively. Values of the field-pattern correlations were similar across nOPM, tOPM and SQUID gradiometers. Volume currents reduced the signals of primary currents on average by 10%, 72% and 15% in nOPM, tOPM and SQUID magnetometers, respectively. The information capacities of the OPM arrays were clearly higher than that of the SQUID array. The dipole-localization accuracies of the arrays were similar while the minimum-norm-based point-spread functions were on average 2.4 and 2.5 times more spread for the SQUID array compared to nOPM and tOPM arrays, respectively.Copyright © 2016 The Authors. Published by Elsevier Inc. All rights reserved.
Moving magnetoencephalography towards real-world applications with a wearable system
[J]. ,
A synthetic optically pumped gradiometer for magnetocardiography measurements
[J]. ,
Optically pumped magnetometers: From quantum origins to multi-channel magnetoencephalography
[J]. ,DOI:S1053-8119(19)30455-0 PMID:31141737 [本文引用: 1]
Optically Pumped Magnetometers (OPMs) have emerged as a viable and wearable alternative to cryogenic, superconducting MEG systems. This new generation of sensors has the advantage of not requiring cryogenic cooling and as a result can be flexibly placed on any part of the body. The purpose of this review is to provide a neuroscience audience with the theoretical background needed to understand the physical basis for the signal observed by OPMs. Those already familiar with the physics of MRI and NMR should note that OPMs share much of the same theory as the operation of OPMs rely on magnetic resonance. This review establishes the physical basis for the signal equation for OPMs. We re-derive the equations defining the bounds on OPM performance and highlight the important trade-offs between quantities such as bandwidth, sensor size and sensitivity. These equations lead to a direct upper bound on the gain change due to cross-talk for a multi-channel OPM system.Copyright © 2019 The Authors. Published by Elsevier Inc. All rights reserved.
Moving wearable magnetoencephalography measurement study based on optically-pumped magnetometer
[J]. ,
基于原子磁力计的穿戴式脑磁图动态测量研究
[J]. ,DOI:10.11938/cjmr20222975 [本文引用: 2]
脑磁图作为一种无创的脑功能成像技术,依靠超高的时间及空间溯源分辨率,在脑科学研究和临床应用领域中有着极其重要的价值.本文介绍了自主搭建的基于原子磁力计的穿戴式脑磁图系统,通过设计匀场补偿线圈组并结合参考传感器阵列,实现被试头部运动区域内剩磁在±1 nT以内,保证动态测量过程中传感器输出维持在动态范围以内;同时提出了一种虚拟合成梯度去噪方法,显著抑制了环境共模噪声;最终在被试者头部自然运动状态下,成功检测到高信噪比的α节律信号与听觉诱发磁场信号,证实了该系统的有效性,为穿戴式脑磁图应用推广提供更多的可能性.
Optimal design for quantification of gas concentration based olfactory stimulator
[J]. ,
基于气体浓度定量的嗅觉刺激器优化设计
[J]. , 2021,
Steady-state evoked potentials
[J]. ,The advantages of steady-state EP recording include (1) speed in assessing sensory function in normal and sick infants (e.g., in amblyopia) and in sick adults (e.g., in multiple sclerosis); (2) monitoring certain activities of sensory pathways that do not intrude into conscious perception; (3) rapidly assessing sensory function when a large number of subjects must be tested (e.g., in refraction); (4) objective measurement at very high suprathreshold levels where psychophysical methods are difficult or ineffective; (5) rapidly assessing sensory function in normal subjects when EP variability and nonstationarity preculde lengthy experiments; and (6) proving a speedy objective equivalent to behavioral test in animals.
Probing cortical excitability using rapid frequency tagging
[J]. ,DOI:S1053-8119(19)30256-3 PMID:30930309 [本文引用: 1]
Frequency tagging has been widely used to study the role of visual selective attention. Presenting a visual stimulus flickering at a specific frequency generates so-called steady-state visually evoked responses. However, frequency tagging is mostly done at lower frequencies (<30 Hz). This produces a visible flicker, potentially interfering with both perception and neuronal oscillations in the theta, alpha and beta band. To overcome these problems, we used a newly developed projector with a 1440 Hz refresh rate allowing for frequency tagging at higher frequencies. We asked participants to perform a cued spatial attention task in which imperative pictorial stimuli were presented at 63 Hz or 78 Hz while measuring whole-head magnetoencephalography (MEG). We found posterior sensors to show a strong response at the tagged frequency. Importantly, this response was enhanced by spatial attention. Furthermore, we reproduced the typical modulations of alpha band oscillations, i.e., decrease in the alpha power contralateral to the attentional cue. The decrease in alpha power and increase in frequency tagged signal with attention correlated over subjects. We hereby provide proof-of-principle for the use of high-frequency tagging to study sensory processing and neuronal excitability associated with attention.Copyright © 2019 The Authors. Published by Elsevier Inc. All rights reserved.
20 Hz steady-state response in somatosensory cortex during induction of tactile perceptual learning through LTP-like sensory stimulation
[J]. ,DOI:10.3389/fnhum.2020.00257 PMID:32694988 [本文引用: 1]
The induction of synaptic plasticity requires the presence of temporally patterned neural activity. Numerous cellular studies in animals and brain slices have demonstrated that long-term potentiation (LTP) enhances synaptic transmission, which can be evoked by high-frequency intermittent stimulation. In humans, plasticity processes underlying perceptual learning can be reliably induced by repetitive, LTP-like sensory stimulation. These protocols lead to improvement of perceptual abilities parallel to widespread remodeling of cortical processing. However, whether maintained rhythmic cortical activation induced by the LTP-like stimulation is also present during human perceptual learning experiments, remains elusive. To address this question, we here applied a 20 Hz intermittent stimulation protocol for 40 min to the index-, middle- and ring-fingers of the right hand, while continuously recording EEG over the hand representation in primary somatosensory cortex in young adult participants. We find that each train of stimulation initiates a transient series of sensory-evoked potentials which accumulate after about 500 ms into a 20 Hz steady-state response persisting over the entire period of the 2-s-train. During the inter-train interval, no consistent evoked activity can be detected. This response behavior is maintained over the whole 40 min of stimulation without any indication of habituation. However, the early stimulation evoked potentials (SEPs) and the event-related desynchronization (ERD) during the steady-state response change over the 40 min of stimulation. In a second experiment, we demonstrate in a separate cohort of participants that the here-applied pneumatic type of stimulation results in improvement of tactile acuity as typically observed for electrically applied 20 Hz intermittent stimulation. Our data demonstrate that repetitive stimulation using a 20 Hz protocol drives rhythmic activation in the hand representation of somatosensory cortex, which is sustained during the entire stimulation period. At the same time, cortical excitability increases as indicated by altered ERD and SEP amplitudes. Our results, together with previous data underlining the dependence of repetitive sensory stimulation effects on NMDA-receptor activation, support the view that repetitive sensory stimulation elicits LTP-like processes in the cortex, thereby facilitating perceptual learning processes.Copyright © 2020 Brickwedde, Schmidt, Krüger and Dinse.
Occipital Alpha rhythm eye position and lens accommodation
[J]. ,
Study of the alpha wave differences between eyes-closed and eyes-open resting states
[J]. ,
闭眼与开眼静息状态下脑电α波的差异研究
[J]. ,
EEG-based system for rapid on-off switching without prior learning
[J]. ,Details are reported of an EEG-based system that permits a person rapidly and reliably to switch on and off electrical devices without prior learning. The system detects and utilises increases in the amplitude of the alpha component of the EEG spectrum that occur when people close their eyes for more than 1 s. In addition to conventional signal-processing elements, the system incorporates a module for suppressing switching at the output of the system when predetermined noise threshold levels (such as those due to sources of EMG) are exceeded. This work indicates that a majority, perhaps in excess of 90%, of the adult population can demonstrate the control necessary to operate an electrical device or appliance using this system. It is indicated that multi-level switching and quasi-continuous control options are feasible with further development of the system. This work has implications for the design of a system that could be used, for example, to assist the infirm or severely physically disabled to effect greater control over their environment.
Practical real-time MEG-based neural interfacing with optically pumped magnetometers
[J]. ,DOI:10.1186/s12915-021-01073-6 PMID:34376215 [本文引用: 1]
Brain-computer interfaces decode intentions directly from the human brain with the aim to restore lost functionality, control external devices or augment daily experiences. To combine optimal performance with wide applicability, high-quality brain signals should be captured non-invasively. Magnetoencephalography (MEG) is a potent candidate but currently requires costly and confining recording hardware. The recently developed optically pumped magnetometers (OPMs) promise to overcome this limitation, but are currently untested in the context of neural interfacing.In this work, we show that OPM-MEG allows robust single-trial analysis which we exploited in a real-time 'mind-spelling' application yielding an average accuracy of 97.7%.This shows that OPM-MEG can be used to exploit neuro-magnetic brain responses in a practical and flexible manner, and opens up new avenues for a wide range of new neural interface applications in the future.© 2021. The Author(s).
A hybrid steady-state visual evoked response-based brain-computer interface with MEG and EEG
[J]. ,
Comparing user-dependent and user-independent training of CNN for SSVEP BCI
[J]. ,
Filter bank convolutional neural network for SSVEP classification
[J]. ,
A comparison study of canonical correlation analysis based methods for detecting steady-state visual evoked potentials
[J]. ,
Task-related component analysis for functional neuroimaging and application to near-infrared spectroscopy data
[J]. ,DOI:10.1016/j.neuroimage.2012.08.044 PMID:22922468 [本文引用: 1]
Reproducibility of experimental results lies at the heart of scientific disciplines. Here we propose a signal processing method that extracts task-related components by maximizing the reproducibility during task periods from neuroimaging data. Unlike hypothesis-driven methods such as general linear models, no specific time courses are presumed, and unlike data-driven approaches such as independent component analysis, no arbitrary interpretation of components is needed. Task-related components are constructed by a linear, weighted sum of multiple time courses, and its weights are optimized so as to maximize inter-block correlations (CorrMax) or covariances (CovMax). Our analysis method is referred to as task-related component analysis (TRCA). The covariance maximization is formulated as a Rayleigh-Ritz eigenvalue problem, and corresponding eigenvectors give candidates of task-related components. In addition, a systematic statistical test based on eigenvalues is proposed, so task-related and -unrelated components are classified objectively and automatically. The proposed test of statistical significance is found to be independent of the degree of autocorrelation in data if the task duration is sufficiently longer than the temporal scale of autocorrelation, so TRCA can be applied to data with autocorrelation without any modification. We demonstrate that simple extensions of TRCA can provide most distinctive signals for two tasks and can integrate multiple modalities of information to remove task-unrelated artifacts. TRCA was successfully applied to synthetic data as well as near-infrared spectroscopy (NIRS) data of finger tapping. There were two statistically significant task-related components; one was a hemodynamic response, and another was a piece-wise linear time course. In summary, we conclude that TRCA has a wide range of applications in multi-channel biophysical and behavioral measurements.Copyright © 2012 Elsevier Inc. All rights reserved.
EEGNet: a compact convolutional neural network for EEG-based brain-computer interfaces
[J]. ,
EEG-based communication: improved accuracy by response verification
[J]. ,DOI:10.1109/86.712231 PMID:9749910 [本文引用: 1]
Humans can learn to control the amplitude of electroencephalographic (EEG) activity in specific frequency bands over sensorimotor cortex and use it to move a cursor to a target on a computer screen. EEG-based communication could provide a new augmentative communication channel for individuals with motor disabilities. In the present system, each dimension of cursor movement is controlled by a linear equation. While the intercept in the equation is continually updated, it does not perfectly eliminate the impact of spontaneous variations in EEG amplitude. This imperfection reduces the accuracy of cursor movement. We evaluated a response verification (RV) procedure in which each outcome is determined by two opposite trials (e.g., one top-target trial and one bottom-target trial). Success, or failure, on both is required for a definitive outcome. The RV procedure reduces errors due to imperfection in intercept selection. Accuracy for opposite-trial pairs exceeds that predicted from the accuracies of individual trials, and greatly exceeds that for same-trial pairs. The RV procedure should be particularly valuable when the first trial has >2 possible targets, because the second trial need only confirm or deny the outcome of the first, and it should be applicable to nonlinear as well as to linear algorithms.
The time course of chromatic adaptation in human early visual cortex revealed by SSVEPs
[J]. ,
/
〈 | 〉 |