基于新一代脑磁图的语义视听单试次检测
Semantic Audiovisual Single-trial Detection Based on the New Generation of Magnetoencephalography
通讯作者: *Tel: 18900616030, E-mail:xiaodong.yang@sibet.ac.cn.
收稿日期: 2024-01-4
基金资助: |
|
Corresponding authors: *Tel: 18900616030, E-mail:xiaodong.yang@sibet.ac.cn.
Received: 2024-01-4
为解码人脑在语义情境下的视听双模态与单模态中的响应差异,本研究设计了相关任务范式并应用新一代脑磁图(OPM-MEG)结合机器学习方法对采集信号从行为学响应、事件相关场(ERF)和单试次检测3个角度进行分析.结果显示单模态语义响应主要集中在枕叶,而双模态语义响应主要集中在顶叶.同时,双模态下的被试响应速率及单试次检测准确率显著高于单模态.此外,支持向量机(SVM)在4种机器学习模型中显示出了最佳分类效能,在被试内分类平均准确率可达75.16%,被试间平均准确率达80.56%.结果表明基于OPM-MEG结合机器学习为实现解码语义情境下的视听双模态与单模态响应差异提供了一条新的有效途径.
关键词:
In order to decode the difference between audiovisual bimodal and unimodal responses of the human brain in semantic context, this study designed a related task paradigm and applied a new generation magnetoencephalogram combined with the machine learning model to analyze the collected signals from three perspectives: behavioral response, event-related field (ERF) and single-trial detection. Results show that the unimodal semantic response was mainly concentrated in the occipital cortex, while the bimodal semantic response was mainly concentrated in the parietal cortex. At the same time, respondents' response rate and the detection accuracy of single-trial in bimodal mode were significantly higher than that in unimodal mode. Moreover, the support vector machine (SVM) showed the best classification performance among the four machine learning models, with an average classification accuracy of 75.16% for within-subject classification and 80.56% for between-subject classification. This research concludes that the combination of optically pumped magnetometer-magnetoencephalography (OPM-MEG) and machine learning model provides an efficient approach to decode the difference between audiovisual bimodal and unimodal responses of the human brain in semantic context.
Keywords:
本文引用格式
郭旭, 王晨旭, 张欣, 常严, 崔峰, 郭清乾, 胡涛, 杨晓冬.
GUO Xu, WANG Chenxu, ZHANG Xin, CHANG Yan, CUI Feng, GUO Qingqian, HU Tao, YANG Xiaodong.
引言
脑磁图(MEG)是一种非侵入性神经成像技术,可以在人头皮外检测到大脑的磁场[1].脑磁图(MEG)与脑电图(EEG)不同,因脑磁信号在传递过程中受头皮、颅骨、脑脊液等介质干扰较小,信号不易变得模糊扭曲,所以对比于脑电图而言,在时间和空间上都有着较高的分辨率[2].随着无自旋交换弛豫(spin-exchange relaxation-free,SERF)原子磁力计(optically-pumped magnetometer,OPM)的快速发展[2,3],对比基于超导量子干涉仪(super conducting quantum interference devices,SQUID)的传统脑磁图(SQUID-MEG)而言,其无需依靠超低温环境就能实现较高的探测灵敏度(15 fT/Hz),通过将OPM传感器阵列排布在头部测量帽上可实现新一代可穿戴脑磁图仪——OPM-MEG[4⇓-6].OPM-MEG相比SQUID-MEG更接近大脑,因此可以获得更高的信号振幅[7,8],允许受试者有更大的活动范围来进行长时间连续的单词或短语实验刺激,更加适用于开展基于行为学任务下的语言实验研究[2].
目前开展语言任务实验最常采取的现代技术为事件相关电位/场(event-related potentials/fields,ERP/ERF),通过测得时间锁定的大脑反应,获得特定感觉、认知或运动事件的直接结果(脑磁图中称为事件相关场ERF,脑电图中称为事件相关电位ERP).语义相关的ERF成分主要包括M400(包括P400和N400)、M600(包括P600和N600)和M300(包括P300和N300),其中M300为早期语言认知常见的ERF成分,反映大脑皮质对语言的认知加工过程,可作为体现人类语言功能的特异性波形.在目前脑机接口(BCI)研究与应用中,大多数研究者认为基于事件相关电位/场的BCI系统(ERP/F-BCI)更加高效且稳定[9].ERP/F-BCI系统中,受试者可以对基于视觉或听觉通道的刺激进行选择性注意来实现对目标字符或任务的选择[10].但大多数对于语义相关的ERP/F-BCI研究都是基于视觉或听觉的单通道刺激,Geuze等[11]使用L2正则化逻辑回归算法作为分类器,对基于视觉呈现的相关和不相关单词的语义启动(Semantic Priming)范式进行单次试验检测,并发现所有参与者都能有效检测到语义相关的ERP成分N400和P300.Tanaka等[12]记录了18名被试听含有语义或语法异常的口语句子时的大脑活动,并利用多层感知机(MLP)检测N400和P600准确率为59.5%.
近年来,研究者也开始对视听混合刺激范式进行研究:Mortier等利用5个目标数字与5个视觉或听觉干扰数字来构建视听混合刺激[13];Pires等利用7个视觉单词与其口语发音相结合构建视听混合刺激[14]. 上述研究均得到视听双通道混合刺激比单一视觉或听觉刺激具有更好性能的结论,但其探索设计的BCI范式都是基于脑电信号在刺激后300 ms所诱发出的与早期语言认知相关的P300成分,未能检测出语义相关的其他ERP成分,例如N400和P600.Chang等[15]利用汉语高频名词设计了一种新的基于视觉条件下的词图匹配语义判断任务,整合了图片与词语,能有效的诱发出N400波形,Lu等[16]也从行为学的角度对视听通道的图片和词汇语义加工进行比较研究,结果表明:视听双通道的整合效应对图片和词汇语义信息加工均有促进作用,且证实了词汇语义信息的加工要比图片语义更为缓慢和复杂.然而对于图片语义加工,在脑机接口中视听整合效应是否会有同样的促进效应,依然是一个需要验证的问题.
因M400和M600受背景噪音、人为因素和试验差异的强烈影响,所以对诱发M400和M600的范式仍需进行进一步探索.本研究的目的是探究基于新一代脑磁图的听觉、视觉以及视听结合的词图匹配实验能否诱发语义相关ERF成分.假设3组实验结果均能诱发M400和M600相关的ERF成分,试图从脑磁图中检测出这种语义启动信号.结合4种机器学习分类器,并在被试间与被试内视听双通道条件下,进一步探究词图匹配的分类准确率是否高于视听单通道条件下的分类准确率.
1 材料与方法
1.1 参与者
本研究共招募15例健康成年人,其中男性5例,女性10例,平均年龄(24.8±3.2)岁.所有被试均为右利手,裸视或矫正视力正常,无精神疾病史,母语为汉语,大学本科及以上文化水平.所有被试实验前均签署受试者知情同意书,并进行蒙特利尔(MoCA)认知评估,评分结果均大于26分,平均得分28.47±1.53分,实验结束后按约支付一定报酬.
1.2 实验设计
3组语言任务改编自Chang等[15]所设计的词图匹配范式.从《基础名词认知卡片大全》中,挑选日常生活中出现频率较高的120张彩色卡通认知图片,图片尺寸为6英寸(15.24 cm×10.16 cm),图片内事物名称均为双字词.语言任务基于MATLAB平台的Psychtoolbox3工具包进行设计,并利用DATAPixx硬件工具箱将任务画面传输至屏蔽房中投影屏幕呈现.书写字符以100磅(3.53 cm×3.53 cm)黑色宋体字呈现在被试前方约70 cm距离的白色背景屏幕中央,屏幕尺寸为15.6英寸(34.54 cm×19.43 cm),分辨率为1 920×1 080,刷新率60 Hz.彩色图片呈现大小约占白色背景屏幕1/4,双字词语呈现大小约占白色背景屏幕1/27.
3组语言实验任务呈现顺序如图1所示,首先呈现图片1 500 ms,紧接着屏幕白屏500 ms,随后1 500 ms内,视觉条件下仅投影屏幕呈现汉语词语文字,听觉条件下仅耳麦以口语的方式播报词语语音而屏幕画面呈现白屏,视听同步条件下屏幕呈现汉语词语文字,同时耳麦以口语的方式播报相同词语语音,最后呈现十字加号1 800~2 200 ms(平均2 000 ms),在出现十字加号时,要求被试对3种条件下出现的词语与图片的语义进行判断,若图片与词语语义相同则按下手柄中的红色按钮,反之则按绿色按钮.词汇与图片均随机呈现,每种条件下词图语义一致和不一致情况出现次数均为60次,共进行120次词图语义判断.在实验之前,所有被试都需要进行预实验,以确保他们完全理解实验规则.连续两个刺激条件之间有3 min的休息时间.
图1
1.3 数据采集与预处理
数据采集工作在一个内部尺寸为1.75 m×1.95 m×2.23 m的磁屏蔽室内进行,其可屏蔽地球磁场至10 nT以内,确保传感器处于正常工作状态.室内放置一台与外界操作刺激电脑相连接的投影屏幕,用于呈现视觉刺激,投影屏幕前放置一台集传导声音刺激的空气耳机和OPM传感器于一体的智能化座椅,可任意调节座椅高度确保被试视线与投影屏幕中央在同一水平线.本研究使用了由8通道OPM传感器(Gen-2.0 QZFM,QuSpin Inc.Colorado,USA)组成的阵列.将这些传感器布置在一个由弹性塑料制成的柔性头盔上,其可被伸展以适用于任何头型.脑磁数据通过模拟信号接入两张16位的NI(National Instruments,USA)数据采集卡进行获取,采样率为1 024 Hz.根据之前研究[12,17,18],分次测量了左右额颞叶、顶叶和枕叶感兴趣区的信号,在左右额颞叶各放置3个OPM传感器,枕叶和顶叶各放置1个OPM传感器,共采用8个OPM传感器测量头部脑磁信号(如图2,图中采用数字1~8对OPM传感器进行编号).实验时,要求被试佩戴好插有OPM传感器的柔性脑磁帽,舒适地坐在屏蔽室内座椅,并根据屏蔽室内投影屏幕指示完成实验.
图2
OPM-MEG的数据处理使用MATLAB R2022b(The MathWorks,Inc.),利用巴特沃斯滤波器对原始信号进行1~80 Hz的带通滤波,降采样至256 Hz,并对数据进行去趋势.按照刺激十字加号呈现时间的-300~ 1 000 ms对去趋势后数据进行分段,以-300~0 ms为基准进行基线校正.
1.4 数据分析
1.4.1 行为学数据分析
对被试按键准确率(Accuracy,ACC)和反应时间(Reaction time,RT)分别进行重复测量方差分析(Analysisofvariance,ANOVA),采用模态(听觉、视觉和视听同步)作为被试内变量,并进行事后检验(t-test,Bonferroni校正).使用Mauchly检验评估球形假设,并在需要时使用Greenhouse-Geisser非球形校正来进行p值校正,Bonferroni校正用于多重成对比较.
1.4.2 ERF分析
因语言相关ERF成分主要在6 Hz左右[12],为进一步提高信噪比,使用巴特沃斯零相正反向滤波器对信号进一步进行1~8 Hz带通滤波,此步骤仅为更好的可视化分析ERF,并未将滤波后数据进行后续的分类检测.基于之前语义与语法相关的M400和M600响应研究,M400的峰值潜伏期大约在刺激呈现后300 ~500 ms,M600的峰值潜伏期大约在刺激呈现后500~800 ms[19,20],所以选择100~300 ms、300~500 ms和500~800 ms 3个时间窗口进行分析,计算所有被试3个窗口下的平均振幅,并将传感器位置划分3个感兴趣区域(ROIs):左边(OPM-1,OPM-3,OPM-6),中间(OPM-4,OPM-8),右边(OPM-2,OPM-5,OPM-7).针对于视觉、听觉和视听同步3种模态,分别在每个时间窗内对每个模态下匹配与不匹配两种条件和3个感兴趣区域进行双因素重复测量方差分析,检查条件间的主效应以及条件和区域之间的相互作用.使用Bonferroni方法对条件和区域之间的相互作用进行事后多重比较,显著性水平设置为0.05,如果p值小于临界α水平(0.05),则得出两种条件下的数据具有显著性差异的结论.
1.4.3 特征提取与分类
对于分类模型,使用支持向量机(SVM)[23]、随机森林(RF)[24]、线性判别分析(LDA)[25]和深度学习中的长短期记忆模型(LSTM)[26],所有的分类模型都是在MATLAB中实现的.首先在15名被试数据内,对每名被试数据采用十折交叉验证进行被试内数据的分类检测,训练集与测试集比率为9:1,即将每名被试数据样本随机分成10份,每次采用其中9份108次作为训练,1份12次作为测试,重复进行10次网络结构训练,计算十折交叉验证的平均分类精度作为最终精度;然后,进行被试间数据分类,即将15名被试数据按照4:1划分为训练集和测试集,即12例被试数据在4种分类模型中进行训练,随后在3名被试数据上进行测试.线性SVM算法使用fitcsvm()函数实现,使用“一对一”分类.LDA算法使用classify()函数实现;RF算法采用Abhishek Jaiantilal开发的randomforest-matlab开源工具箱中classRF_train()函数实现;LSTM模型包含一个输入层,一个LSTM层,一个ReLU激活函数和一个用于多分类的softmax激活函数作为输出层,其中LSTM层的隐含单元个数为128.
2 结果
2.1 行为结果
3种模态条件下,参与者执行行为任务均具有较高的准确性,ACC的平均值(±标准差)在视觉条件下为(97.72±2.28)%,在听觉条件下为(95.94±4.06)%,在视听条件下为(98.11±1.89)%.RT的平均值(±标准差)在视觉条件下为(490.01±162.17)ms,在听觉条件下为(447.47±195.08)ms,在视听双模态条件下为(424.16±189.86)ms.3种模态间ACC和RT经重复测量ANOVA所得F检验的统计量对应的p值均小于显著性水平0.05,表明3种模态间ACC和RT均具有显著的主效应[ACC:F(2,28)= 3.83,p= 0.04;RT:F(2,28)= 7.19,p= 0.004].事后检验进一步表明,被试在视听条件下相比于视觉[F(1,14)= 19.87,p= 0.015]和听觉[F(1,14)= 14.43,p= 0.03]条件能更快地做出判断.
2.2 ERF分析结果
图3为3种模态下ERF的平均值和3个时间窗内差异波(词图语义不匹配-词图语义匹配)地形图.
图3
图3
3种模态下所有被试总的叠加平均图(左侧)和差异波(不匹配-匹配)地形图(右侧)
Fig. 3
The total superimposed average diagram of all subjects in three modes(on the left)and the differential wave (mismatch-match) topographic map (on the right)
3种模态在每个时间窗内重复测量方差分析结果如表1所示,3种模态下的词图匹配刺激所诱发出的大脑响应具有显著性差异.在视觉和听觉单模态条件下,300~500 ms时间窗内条件与区域都有显著的交互效应,并进一步事后检验表明,二者均在中间和左边两个区域表现出显著性差异,其中在听觉模态下,条件与区域之间在500~800 ms时间窗内同样表现出显著的交互效应;在视听双模态条件下,100~300 ms和500~800 ms时间窗内条件间都具有显著的主体内效应,而条件与区域之间未发现显著的交互效应.
表1 重复测量方差分析结果
Table 1
模态 | 时间窗t/ms | 重复测量方差分析 | ||||||
---|---|---|---|---|---|---|---|---|
主体内效应检验 显著性p值 | 条件与区域交互效应 显著性p值 | 成对比较 | ||||||
区域 | 平均值差值 匹配-不匹配 | 显著性p值 | 差值的95%置信区间 | |||||
下限 | 上限 | |||||||
视觉 | 100~300 | 不显著 | 不显著 | |||||
300~500 | 0.016 | 0.007 | 左边 | 0.21 | 0.029 | 0.02 | 0.39 | |
中间 | 0.49 | 0.009 | 0.14 | 0.83 | ||||
500~800 | 不显著 | 不显著 | ||||||
听觉 | 100~300 | 不显著 | 不显著 | |||||
300~500 | 不显著 | 0.003 | 左边 | 0.2 | 0.013 | 0.05 | 0.36 | |
中间 | 0.19 | 0.014 | 0.08 | 0.58 | ||||
500~800 | 0.001 | 0.004 | 左边 | 0.3 | 0.007 | 0.09 | 0.51 | |
中间 | 0.6 | 0.001 | 0.3 | 0.9 | ||||
视听 同步 | 100~300 | 0.044 | 不显著 | |||||
300~500 | 不显著 | 不显著 | ||||||
500~800 | 0.02 | 不显著 |
2.3 单试次检测
图4显示了在3种模态下,每个分类器在单个被试数据中的测试集分类准确率,所有分类精度均为10次运行的平均精度.所有被试数据分类准确率均大于机会水平50%,对比3种不同模态下的所有被试平均准确率,在视听双模态中SVM和LDA都表现出较好的分类结果,其中SVM的15例被试平均分类准确率要略高于LDA,平均值为75.16%,范围在59.97%~96.64%之间.
图4
图4
3种模态下每个被试数据在4种分类器中的测试集分类准确率(左侧)和所有被试测试集平均分类准确率(右侧)
Fig. 4
On the left is the classification accuracy of test set in four classifiers for each subject data in three modes; on the right is the average classification accuracy of all test sets
图5(a)、(c)显示了在3种模态下,每个分类器在15例被试内数据测试集的平均分类准确率(图中*表示p<0.05,**表示p<0.01,***表示p<0.001).图5(b)、(d)显示了在3种模态下,每个分类器在被试间数据中的测试集分类准确率.ANOVAs结果显示3种模态准确率在两种分类检测中均具有显著性差异.且视听双模态条件下,4种分类器在被试内和被试间数据的测试集平均分类准确率均高于视觉和听觉单模态条件下的分类准确率.对于4种分类算法在同一模态条件下的分类准确率,ANOVAs结果未发现显著性差异.但从图5(a)、(b)中可以看到,在视听双模态条件下SVM平均分类准确率都略高于其他分类器的平均分类水平.其在被试间分类检测的平均准确率为80.56%,与以往语义相关单试次检测相比都有着相似或更好的结果[11,12].
图5
图5
(a) 3种模态下所有被试内数据分类准确率;(b) 3种模态下被试间分类准确率;(c) 4种分类器在3种模态下的被试内数据分类水平;(d) 4种分类器在3种模态下的被试间数据分类水平
Fig. 5
(a) Accuracy of data classification in all subjects under three modes; (b) classification accuracy among subjects in three modes; (c) the data classification level of the four classifiers in three modes; (d) the classification level of data among subjects under three modes of four classifiers
3 讨论
以往的研究主要是基于脑电针对于诱发P300范式进行视听双模态和单模态的对比研究[27⇓-29],本研究的主要目的是基于新一代脑磁图针对于诱发语义特异性波形M400和M600的词图匹配范式开展实验研究,借助OPM-MEG高时空分辨率探究词图匹配语义启动范式在脑机接口中视听双模态对比单模态是否具有明显的优越性.实验范式设计采用的是双字词语刺激,主要是在词汇水平上诱发出语义启动响应,其中M400通常作为词汇语义层面的ERF指标,M600则作为句法层面的ERF指标,因此在研究过程中M400为主要分析的ERF成分.本研究从行为学和ERF角度分析,从行为学的结果来看,视听同步刺激下促进大脑语义认知,被试能更快速准确的进行词图语义判断.受试者在做出行为判断时有两种结果:判断错误与判断正确,每种语义判断结果都与大脑各个区域活动相关联,目前研究也未能对大脑行为判断结果做出准确具体的解释,有研究发现大脑前额叶皮质可能参与大脑认知行为判断,然而尚需进一步的实验验证[15,30],而本研究主要是讨论语义启动视听双模态刺激的优越性,未具体考虑被试行为按键错误数据段的大脑响应机制,加之语言加工是一个极为复杂的过程,目前尚无实验可完整的呈现语言处理的全过程,这也将在下一步研究中进一步设计语义干扰相关范式深入探讨语言加工过程[18,31].
ERF分析结果显示,视听双模态和单模态条件下的词图匹配均能诱发出明显的语义特异性波形M400和M600,这为一直以来开展基于P300成分识别目标刺激的ERP/F-BCI提供了新思路,但M400和M600对比于P300潜伏期更长,随着潜伏期的增长,大脑参与的语义处理的区域更多,这也意味着需要更加全面的了解各个脑区的认知活动,本研究利用8通道OPM传感器分布在左右额颞叶、顶叶以及枕叶,共同探讨语义视听双模态和单模态条件下大脑响应差异,在刺激后1 000 ms内,视觉和听觉单模态条件下在枕叶以及左右额颞叶诱发出更大的语义启动响应,而视听双模态条件下语义启动响应主要集中于顶部中央区域,已有研究发现人类语言产生最早开始于大脑皮质额下回的Broca区和颞上回处的Wernicke区,二者通过弓状束相连接,是语言处理的核心区域[18],枕叶是参与早期视觉语言加工有关的重要区域[32],顶叶在处理短期空间记忆信息中占主导作用[33],根据Peter Hagoort提出的关于语言加工的记忆-整合-控制模型,表明视听同步刺激也可能促进语言加工处理,加速激活中央顶叶区域,让语言加工更快进入记忆存储阶段并进行词汇信息提取,将语音和语义信息整合到整个语言的全部表征中,从而控制行为快速地进行词图语义判断[34].
分类性能分析结果表明,所有的分类结果都显著高于机会水平,视听双模态条件下的词图匹配范式表现出良好的性能.从单个被试内数据和被试间数据的分类结果来看[图5(a)、(b)],视听双通道刺激条件下分类准确率均显著高于单通道刺激的分类准确率.对比以往的研究,基于新一代脑磁图所观察到的词图匹配诱发反应能产生更高的分类准确度,这也有部分原因是各个语义启动范式所诱发大脑响应的强度有所差异,这将在下一步研究中探索不同语义启动范式诱发响应的差异.从分类器性能来看[图5(c)、(d)],每个分类器对视听条件下的分类水平都高于仅视觉或听觉的分类水平,在视觉和视听同步条件下,SVM在被试间数据的分类水平都优于其他分类器,而在听觉条件下,RF表现出更好的分类性能.在4种分类器中,深度学习模型LSTM往往需要较大的数据量支撑,可能由于本研究数据量未能满足模型发挥最优性能的需求,导致LSTM未能表现出比传统机器学习更加明显的优势,在下一步研究中将进一步扩大样本数据量来验证模型的分类性能.
综上所述,本研究利用新一代脑磁图对语义视听不同模态进行对比研究,无论从行为学角度还是单试次分类水平角度,视听双模态词图匹配范式都有着更好的性能,也表明,OPM-MEG从数据采集到数据处理已是一项成熟的技术,并能为脑科学研究提供新的研究手段和更高质量的数据.
4 结论
视听双模态和单模态条件下的词图匹配均能诱发出明显的语义特异性波形M400和M600,并且利用新一代脑磁图,在被试间与被试内数据进行单试次检测,证实了视听双模态刺激条件要比单模态条件下有着更好的分类准确率,分类结果表明,M400和M600也可作为ERP/F-BCI研究的新方向.其次,视听同步刺激的词图匹配范式更加适用于临床失语症、脑卒中等语言障碍人群进行康复训练,本研究为揭示语言认知障碍类疾病的预防、发展和康复治疗提供了新的思路和方法,也揭示了OPM-MEG在神经科学研究中的潜力和优势,为MEG技术的发展开辟了新的可能性.
利益冲突
无
参考文献
Magnetoencephalography—theory, instrumentation, and applications to noninvasive studies of the working human brain
[J]. ,
Research progress of magnetoencephalography in the functional mechanism of bilingual brain
[J]. ,
双语脑功能机制的脑磁图研究进展
[J]. ,
Progress in biomagnetic signal measurements with ultra-sensitive atomic magnetometers
[J]. ,
超灵敏原子磁力计在生物磁应用中的研究进展
[J]. ,
A new generation of magnetoencephalography: Room temperature measurements using optically-pumped magnetometers
[J]. ,DOI:S1053-8119(17)30041-1 PMID:28131890 [本文引用: 1]
Advances in the field of quantum sensing mean that magnetic field sensors, operating at room temperature, are now able to achieve sensitivity similar to that of cryogenically cooled devices (SQUIDs). This means that room temperature magnetoencephalography (MEG), with a greatly increased flexibility of sensor placement can now be considered. Further, these new sensors can be placed directly on the scalp surface giving, theoretically, a large increase in the magnitude of the measured signal. Here, we present recordings made using a single optically-pumped magnetometer (OPM) in combination with a 3D-printed head-cast designed to accurately locate and orient the sensor relative to brain anatomy. Since our OPM is configured as a magnetometer it is highly sensitive to environmental interference. However, we show that this problem can be ameliorated via the use of simultaneous reference sensor recordings. Using median nerve stimulation, we show that the OPM can detect both evoked (phase-locked) and induced (non-phase-locked oscillatory) changes when placed over sensory cortex, with signals ~4 times larger than equivalent SQUID measurements. Using source modelling, we show that our system allows localisation of the evoked response to somatosensory cortex. Further, source-space modelling shows that, with 13 sequential OPM measurements, source-space signal-to-noise ratio (SNR) is comparable to that from a 271-channel SQUID system. Our results highlight the opportunity presented by OPMs to generate uncooled, potentially low-cost, high SNR MEG systems.Copyright © 2017 The Authors. Published by Elsevier Inc. All rights reserved.
Detection and analysis of MEG signals in occipital region with double-channel OPM sensors
[J]. ,
Moving wearable magnetoencephalography measurement study based on optically-pumped magnetometer
[J]. ,
基于原子磁力计的穿戴式脑磁图动态测量研究
[J]. ,DOI:10.11938/cjmr20222975 [本文引用: 1]
脑磁图作为一种无创的脑功能成像技术,依靠超高的时间及空间溯源分辨率,在脑科学研究和临床应用领域中有着极其重要的价值.本文介绍了自主搭建的基于原子磁力计的穿戴式脑磁图系统,通过设计匀场补偿线圈组并结合参考传感器阵列,实现被试头部运动区域内剩磁在±1 nT以内,保证动态测量过程中传感器输出维持在动态范围以内;同时提出了一种虚拟合成梯度去噪方法,显著抑制了环境共模噪声;最终在被试者头部自然运动状态下,成功检测到高信噪比的α节律信号与听觉诱发磁场信号,证实了该系统的有效性,为穿戴式脑磁图应用推广提供更多的可能性.
A synthetic optically pumped gradiometer for magnetocardiography measurements
[J]. ,
Nuclear magnetic resonance and imaging of hyperpolarized 3He using high-Tc superconducting quantum interference device in microtesla magnetic fields
[J]. ,
采用超导量子干涉组件在微特斯拉磁场下获取超极化3He的核磁共振波谱和影像(英文)
[J]. ,该文以超导量子干涉元件研究光激发氦三极化气体的低磁场磁共振与造影术. 使用圆偏极化雷色光将氦三气体极化, 超导量子干涉元件磁共振与造影之测量是以磁通耦合方式来进行, 超导量子干涉元件是用铋锶钙铜氧高溫超导罐来隔离环境噪音. 此方法测得的磁共振信号与影像相较于直接将样品置于杜瓦瓶下方有较高的信噪比, 当样品无法靠近感测元件时, 此装置具高信噪比特色. 磁通耦合方式的超导量子干涉元件低磁场磁共振与造影术及其氦三极化气体的肺部造影在学术与应用上是相当有趣.
Research on cognitive mechanism and brain-computer interface application in visual-auditory crossmodal stimuli
[J]. ,
基于视听交互刺激的认知机理与脑机接口范式研究进展
[J]. ,
Detecting semantic priming at the single-trial level
[J]. ,
Electroencephalogram-based single-trial detection of language expectation violations in listening to speech
[J]. ,
Classification of targets and distractors in an audiovisual attention task based on electroencephalography
[J]. ,
Visuo-auditory stimuli with semantic, temporal and spatial congruence for a P300-based BCI: An exploratory test with an ALS patient in a completely locked-in state
[J]. ,
Design and pilot study of word-picture matching semantic judgment task based on chinese high frequency nouns
[J]. ,
基于汉语高频名词下词图匹配语义判断任务的设计与运用范式
[J]. ,DOI:10.3969/j.issn.1006-9771.2018.08.010 [本文引用: 3]
目的 探索基于汉语高频名词的词图匹配语义判断任务的设计思路,以及该任务下的功能磁共振成像(fMRI)和事件相关电位(ERP)实验研究方法,为该领域研究提供范式。方法 阐述汉语高频名词的词图匹配判断任务的材料来源和设计流程,并纳入1例健康受试者,进行词图匹配语义判断任务下的fMRI、ERP实验,将任务下采集的fMRI/ERP数据进行分析,判断该任务在fMRI/ERP实验中的可行性。结果 fMRI结果显示,与注视“+”时相比,受试者在词图呈现时右侧额中回激活增高;与词图匹配条件相比,受试者在词图不匹配条件下右侧颞中回激活增高。ERP结果显示,词图匹配条件下受试者左侧额颞区激活较明显,词图不匹配条件下右侧额叶区激活较明显。结论 基于本实验设计,采用fMRI、ERP技术均可以获得语言相关脑区激活效应,提示本任务可用于探索汉语语言的脑加工机制。
A comparative study of pictorial and lexical semantic processing based on visual- auditory modality
[J]. ,
基于视听通道的图片和词汇语义加工比较研究
[J]. ,
Probabilistic language pathways based HARDI tractography
[J]. ,
基于高夹角分辨率扩散磁振造影神经径路追踪的人脑语言机率路径图谱
[J]. ,大脑皮质内部的联系神经束, 对于大脑皮质之间的信息传递担任非常重要的角色. 传统的语言模型理论提出人类的2个主要语言中枢分别位于大脑皮质的左侧额下回的布罗卡区域(Broca’s area, BA44 and BA45)以及颞上回处的维尼基区域(Wernicke’s area, BA22), 而联系这2个区域的纤维束, 也就是弓状束(arcuate fasciculus). 另外, 近期研究也发现下顶叶(inferior parietal cortex, BA39 and BA40)在语音处理历程具重要性. 扩散磁振造影(Diffusion MRI)可以提供大脑白质精细的组织结构, 配合神经径路追踪(tractography)便能撷取出复杂的神经纤维连结路径. 该研究利用扩散磁振影像中的高夹角分辨率扩散磁振造影(high angular resolution diffusion imaging)与神经径路追踪技术, 呈现与语言相关的大脑机率神经连结路径(probabilistic language pathway).
The correlation between the structural properties of the arcuate bundle and the performance of language comprehension is analyzed by limiting spherical convolution
[J]. ,
应用限制球形卷积解析弓状束的结构特性与语言理解表现的相关性
[J]. ,DOI:10.11938/cjmr20160209 [本文引用: 3]
通过限制球形卷积(constrained spherical deconvolution,CSD)和神经纤维追踪技术(tractography),以了解大脑左右半球的弓状束(arcuate fasciculus,AF)神经结构完整性与语言理解能力的相关性.该文使用磁共振成像(MRI)仪对14例大脑左半球罹患肿瘤的右利手患者进行了术前术后扫描(每次扫描均伴随一次波士顿失语症测试),并用分析软件DSI Studio计算术前术后左右半球弓状束的四项扩散指标,并进行了比较.结果发现左脑弓状束有两项指标在术前术后有非常显著的差异(p<0.01),而右脑四项指标均无显著改变(p>0.05).另外,该文还将扩散指标与患者相对应的语言测试所反映的语言理解能力进行了相关性分析,发现无论术前术后,左半球弓状束与语言理解能力相关系数r介于0.6~0.8之间,而右半球则仅介于0.3~0.4之间.上述结果表明,语言理解能力与左侧弓状束密切相关(p<0.01),而与右侧弓状束相关性不大,此结果与以往研究者对大脑侧化的认知相符.
Native and nonnative speakers' processing of a miniature version of Japanese as revealed by ERPs
[J]. ,Several event-related potential (ERP) studies in second language (L2) processing have revealed a differential vulnerability of syntax-related ERP effects in contrast to purely semantic ERP effects. However, it is still debated to what extent a potential critical period for L2 acquisition, as opposed to the attained proficiency level in the L2, contributes to the pattern of results reported in previous ERP studies. We studied L2 processing within the model of a miniature version of a natural language, namely Japanese, specifically constructed to assure high proficiency of the learners. In an auditory ERP experiment, we investigated sentence processing of the "Mini-Japanese" in Japanese native speakers and German volunteers before and after training. By making use of three different types of violation, namely, word category, case, and classifier violations, native and nonnative ERP patterns were compared. The three types of violation elicited three characteristic ERP patterns in Japanese native speakers. The word category violation elicited an anteriorly focused, broadly distributed early negativity followed by a P600, whereas the case violation evoked a P600 which was preceded by an N400. The classifier violation led solely to a late left distributed negativity with an anterior focus. Although the P600 was similar for Japanese natives and learners, the N400 and the anterior negativities were not present in the learner group. The differences across groups suggest deviant neural processes in on-line syntactic and thematic processing in the L2 learners despite high behavioral skills.
The neural mechanisms of word order processing revisited: electrophysiological evidence from Japanese
[J]. ,DOI:10.1016/j.bandl.2008.06.003 PMID:18667231 [本文引用: 1]
We present two ERP studies on the processing of word order variations in Japanese, a language that is suited to shedding further light on the implications of word order freedom for neurocognitive approaches to sentence comprehension. Experiment 1 used auditory presentation and revealed that initial accusative objects elicit increased processing costs in comparison to initial subjects (in the form of a transient negativity) only when followed by a prosodic boundary. A similar effect was observed using visual presentation in Experiment 2, however only for accusative but not for dative objects. These results support a relational account of word order processing, in which the costs of comprehending an object-initial word order are determined by the linearization properties of the initial object in relation to the linearization properties of possible upcoming arguments. In the absence of a prosodic boundary, the possibility for subject omission in Japanese renders it likely that the initial accusative is the only argument in the clause. Hence, no upcoming arguments are expected and no linearization problem can arise. A prosodic boundary or visual segmentation, by contrast, indicate an object-before-subject word order, thereby leading to a mismatch between argument "prominence" (e.g. in terms of thematic roles) and linear order. This mismatch is alleviated when the initial object is highly prominent itself (e.g. in the case of a dative, which can bear the higher-ranking thematic role in a two argument relation). We argue that the processing mechanism at work here can be distinguished from more general aspects of "dependency processing" in object-initial sentences.
ERP effects of listening to speech compared to reading: the P600/SPS to syntactic violations in spoken sentences and rapid serial visual presentation
[J]. ,In this study, event-related brain potential effects of speech processing are obtained and compared to similar effects in sentence reading. In two experiments sentences were presented that contained three different types of grammatical violations. In one experiment sentences were presented word by word at a rate of four words per second. The grammatical violations elicited a Syntactic Positive Shift (P600/SPS), 500 ms after the onset of the word that rendered the sentence ungrammatical. The P600/SPS consisted of two phases, an early phase with a relatively equal anterior-posterior distribution and a later phase with a strong posterior distribution. We interpret the first phase as an indication of structural integration complexity, and the second phase as an indication of failing parsing operations and/or an attempt at reanalysis. In the second experiment the same syntactic violations were presented in sentences spoken at a normal rate and with normal intonation. These violations elicited a P600/SPS with the same onset as was observed for the reading of these sentences. In addition two of the three violations showed a preceding frontal negativity, most clearly over the left hemisphere.
EEG decoding of spoken words in bilingual listeners: from words to language invariant semantic-conceptual representations
[J]. ,DOI:10.3389/fpsyg.2015.00071 PMID:25705197 [本文引用: 1]
Spoken word recognition and production require fast transformations between acoustic, phonological, and conceptual neural representations. Bilinguals perform these transformations in native and non-native languages, deriving unified semantic concepts from equivalent, but acoustically different words. Here we exploit this capacity of bilinguals to investigate input invariant semantic representations in the brain. We acquired EEG data while Dutch subjects, highly proficient in English listened to four monosyllabic and acoustically distinct animal words in both languages (e.g., "paard" - "horse"). Multivariate pattern analysis (MVPA) was applied to identify EEG response patterns that discriminate between individual words within one language (within-language discrimination) and generalize meaning across two languages (across-language generalization). Furthermore, employing two EEG feature selection approaches, we assessed the contribution of temporal and oscillatory EEG features to our classification results. MVPA revealed that within-language discrimination was possible in a broad time-window (similar to 50-620 ms) after word onset probably reflecting acoustic-phonetic and semantic-conceptual differences between the words. Most interestingly, significant across-language generalization was possible around 550-600 ms, suggesting the activation of common semantic-conceptual representations from the Dutch and English nouns. Both types of classification, showed a strong contribution of oscillations below 12 Hz, indicating the importance of low frequency oscillations in the neural representation of individual words and concepts. This study demonstrates the feasibility of MVPA to decode individual spoken words from EEG responses and to assess the spectro-temporal dynamics of their language invariant semantic-conceptual representations. We discuss how this method and results could be relevant to track the neural mechanisms underlying conceptual encoding in comprehension and production.
Application of radiomics based on new support vector machine in the classification of hepatic nodules
[J]. ,
基于新型支持向量机的影像组学在肝脏结节分类中的应用
[J]. ,DOI:10.11938/cjmr20212916 [本文引用: 1]
肝癌是最常见的恶性肿瘤之一,亚洲地区最为常见的肝癌演变过程为肝炎-肝硬化结节-异型增生结节-肝细胞性肝癌.判断肝脏结节在演变过程所处分期,并采取干预措施,对降低肝癌的发生率非常关键.本文针对影像组学提出了更精确的支持向量机(SVM)分类算法——LFOA-F-SVM,用于对120名患者的腹部动态增强磁共振图像的肝脏结节进行四分类.该算法利用了考虑半径与几何间距的F-SVM,并结合莱维飞行策略(LF)的果蝇优化算法(FOA)寻求超参.为了验证方法的有效性,本文另外添加了5个UCI分类数据集(心脏、帕金森疾病、虹膜、葡萄酒和动物园),并与SVM、PSO-SVM、FOA-SVM、F-SVM进行比较.结果表明,在6个分类数据集(包括肝脏结节数据集和5个UCI分类数据集)中,相对于其他分类算法,LFOA-F-SVM的分类准确率最高,在肝脏结节数据集中的四分类精确率和查全率也较高.
Prediction of preoperative T staging of rectal cancer based on radiomics
[J]. ,
基于影像组学的直肠癌术前T分期预测
[J]. ,DOI:10.11938/cjmr20212908 [本文引用: 1]
直肠癌T分期对患者的术前评估有重要作用.然而,传统的放射科医生根据患者磁共振图像直接判断分期的方法效果欠佳.本文提出使用影像组学的方法预测直肠癌T分期,首先获取105例直肠癌患者影像数据,根据病理报告中的T分期结果将T1、T2期患者划分为未突破肌层组,将T3、T4期患者分为突破肌层组,整理数据得到未突破肌层组31例,突破肌层组74例.在患者的轴向位T2WI图像中勾画病灶区域,并在病灶上使用pyradiomics工具包提取影像组学特征,使用最小绝对值收敛和选择算子(LASSO)对高维特征做特征选择,得到与T分期高度相关的特征数据,使用随机森林、支持向量机(SVM)、逻辑回归、梯度提升树(GBDT)分别建模,进行交叉验证调参,评估模型性能.每层图像提取100维特征,经LASSO特征选择后得到7个与T分期高度相关的特征,使用4种模型分别建模,其中SVM算法表现最优,平均受试者操作特征曲线下面积(AUC)、准确率、灵敏度、特异度分别为0.968 5、0.886 4、0.962 5、0.899 2,测试集准确率达到了0.904 7.结果表明,使用影像组学方法可以提高直肠癌T分期预测的准确率.
Application analysis of machine learning algorithm in the field of motor imagery brain-computer interface
[J]. ,
机器学习算法在运动想象脑机接口领域的应用分析
[J]. ,
Design of auditory P300-based brain-computer interfaces with a single auditory channel and no visual support
[J]. ,
A comparative analysis of P300 based BCI in visual and auditory domain
[J]. ,
Audio-visual hybrid brain-computer interface based on space-frequency domain features
[J]. ,
基于空-频域特征的视听混合脑机接口
[J]. ,
Correlating behavioral responses to FMRI signals from human prefrontal cortex: examining cognitive processes using task analysis
[J]. ,
Review of “Word Satiation” research paradigms and experimental tasks in language cognitive processing
[J]. ,
语言认知加工“饱和”现象的研究范式及实验任务述评天津师范大学学报
[J]. ,
Cerebral organization of component processes in reading
[J]. ,
Collaborative activity between parietal and dorso-lateral prefrontal cortex in dynamic spatial working memory revealed by fMRI
[J]. ,Functional MRI was used to determine how the constituents of the cortical network subserving dynamic spatial working memory respond to two types of increases in task complexity. Participants mentally maintained the most recent location of either one or three objects as the three objects moved discretely in either a two- or three-dimensional array. Cortical activation in the dorsolateral prefrontal (DLPFC) and the parietal cortex increased as a function of the number of object locations to be maintained and the dimensionality of the display. An analysis of the response characteristics of the individual voxels showed that a large proportion were activated only when both the variables imposed the higher level of demand. A smaller proportion were activated specifically in response to increases in task demand associated with each of the independent variables. A second experiment revealed the same effect of dimensionality in the parietal cortex when the movement of objects was signaled auditorily rather than visually, indicating that the additional representational demands induced by 3-D space are independent of input modality. The comodulation of activation in the prefrontal and parietal areas by the amount of computational demand suggests that the collaboration between areas is a basic feature underlying much of the functionality of spatial working memory.Copyright 2000 Academic Press.
On Broca, brain, and binding: a new framework
[J]. ,In speaking and comprehending language, word information is retrieved from memory and combined into larger units (unification). Unification operations take place in parallel at the semantic, syntactic and phonological levels of processing. This article proposes a new framework that connects psycholinguistic models to a neurobiological account of language. According to this proposal the left inferior frontal gyrus (LIFG) plays an important role in unification. Research in other domains of cognition indicates that left prefrontal cortex has the necessary neurobiological characteristics for its involvement in the unification for language. I offer here a psycholinguistic perspective on the nature of language unification and the role of LIFG.
/
〈 | 〉 |