相依性度量的比较研究

数学物理学报 ›› 2017, Vol. 37 ›› Issue (5): 931-949.

相依性度量的比较研究

蒋杭进^1,2, 尚妍^3,4, 吴琼莉²

1. 中国科学院大学北京 100190;
2. 中国科学院武汉物理与数学研究所武汉 430071;
3. 中国科学院大学经济与管理学院北京 100190;
4. 中国工商银行博士后科研工作站北京 100032

收稿日期:2016-12-26 修回日期:2017-05-17 出版日期:2017-10-26 发布日期:2017-10-26
通讯作者: 吴琼莉,E-mail:wuqiongli@wipm.ac.cn E-mail:wuqiongli@wipm.ac.cn
基金资助:
国家自然科学基金（31600290）

Dependence Measure:A Comparative Study

Jiang Hangjin^1,2, Shan Yan^3,4, Wu Qiongli²

1. University of Chinese Academy of Sciences, Beijing 100190;
2. Wuhan Institute of Physics and Mathematics, Chinese Academy of Sciences, Wuhan 430071;
3. School of Economics and Management, University of Chinese Academy of Sciences, Beijing 100190;
4. Post-Doctoral Research Center, Industrial and Commercial Bank of China, Beijing 100032

Received:2016-12-26 Revised:2017-05-17 Online:2017-10-26 Published:2017-10-26
Supported by:
Supported by the NSFC (31600290)

摘要/Abstract

摘要： 相依性度量在统计分析中起着关键作用，比如fMRI数据分析、变量选择、网络分析、基因数据分析、蛋白质-蛋白质相互作用网络分析.该文综合分析了现有的相依性度量的性质，比如MIC^[1]、MI^[2]、HHG^[3]，并将它们分为三大类：第一类是相关系数的直接推广，第二类相依性度量则基于不同的独立性假设，第三类是基于学习理论的相依性度量.该文还指出并没有一种相依性度量方法是在所有情况下都是最优的，从相依性度量的性质和我们的模拟结果看CDC^[4]是最好的相依性度量.另外我们还提出用CDC-|ρ|或者CDC²-|ρ|²做非线性度量要比MIC-ρ²合理，其中ρ是相关系数.

关键词: 相依性, CDC, 相依性度量, 非线性度量

Abstract: Dependence measure plays a fundamental role in statistical analysis, such as fMRI data analysis, variable selection, network analysis, genetical analysis, PPI network analysis. The aim of this paper is to provide a state of the art on the topic of dependence measure and to show their properties. We classified all these dependence measures into 3 classes:(1) Extension of correlation coefficient; (2) Dependence measures based on independent conditions; (3) Dependence measures in learning framework; As showed in our analysis, there is no such a dependence measure has the best performance over all kinds of functional types. However, according to the properties that a dependence measure should have combing with our results, CDC^[4] is the best one. Furthermore, CDC²-ρ² or CDC-|ρ|is proposed as a measure of non-linearity, which is better than MIC-ρ², as showed in our analysis, where ρ is the Pearson correlation coefficient.

Key words: Dependence, CDC, Dependence measure, Non-linearity measurement

中图分类号:

O212

蒋杭进, 尚妍, 吴琼莉. 相依性度量的比较研究[J]. 数学物理学报, 2017, 37(5): 931-949.

Jiang Hangjin, Shan Yan, Wu Qiongli. Dependence Measure:A Comparative Study[J]. Acta mathematica scientia,Series A, 2017, 37(5): 931-949.

参考文献

[1] Reshef, David N, et al. Detecting novel associations in large data sets. Science, 2011, 334(6062):1518-1524
[2] Steuer R, et al. The mutual information:detecting and evaluating dependencies between variables. Bioinformatics, 2002, 18(suppl 2):S231-S240
[3] Heller R, Heller Y, Gorfine M. A consistent multivariate test of association based on ranks of distances. Biometrika, 2013, 100(2):503-510
[4] Jiang Hangjin, Wu Qiongli. Robust dependence measure for detecting associations in large data set. Submitted to Acta Mathematica Scientia
[5] Rényi A. On measures of dependence. Acta Mathematica Hungarica, 1959, 10(3):441-451
[6] Moon Y, Rajagopalan B, Lall U. Estimation of mutual information using kernel density estimators. Physical Review E, 1995, 52(3):2318-2321
[7] Fernandes A D, Gregory B G. Mutual information is critically dependent on prior assumptions:would the correct estimate of mutual information please identify itself? Bioinformatics, 2010, 26(9):1135-1139
[8] Gretton A, Herbrich R, Smola A. The kernel mutual information. 2003, DOI:10.1109/ICASSP.2003.1202784
[9] Bach F R, Jordan M I. Kernel independent component analysis. Journal of Machine Learning Research, 2003, DOI:10.1109/ICASSP.2003.1202783
[10] Gretton A, et al. Measuring Statistical Dependence with Hilbert-Schmidt Norms//Jain S, et al. Algorithmic Learning Theory. Berlin:Springer, 2005
[11] Schweizer B, Wolff E F. On nonparametric measures of dependence for random variables. The Annals of Statistics, 1981, 9(4):879-885
[12] Székely G J, Rizzo M L. Brownian distance covariance. The Annals of Applied Statistics, 2009, 3(4):1236-1265
[13] Gretton A, et al. A kernel two-sample test. Journal of Machine Learning Research, 2012, 13(1):723-773
[14] Póczos B, Ghahramani Z, Schneider J. Copula-based kernel dependency measures. 2012, arXiv:1206.4682
[15] Yin X R. Canonical correlation analysis based on information theory. Journal of Multivariate Analysis, 2004, 91(2):161-176
[16] Delicado P, Smrekar M. Measuring non-linear dependence for two random variables distributed along a curve. Statistics and Computing, 2009, 19(3):255-269
[17] Lopez-Paz D, Hennig P, Schölkopf B. The randomized dependence coefficient. Advances in Neural Information Processing Systems, 2013:1-9
[18] Linfoot E H. An informational measure of correlation. Information and Control, 1957, 1(1):85-89
[19] Tsallis C. Possible generalization of Boltzmann-Gibbs statistics. Journal of Statistical Physics, 1988, 52(1):479-487
[20] Tan Q H, Jiang H J, Ding Y M. Model selection method based on maximal information coefficient of residuals. Acta Mathematica Scientia, 2014, 34(2):579-592
[21] Gretton A, et al. Kernel methods for measuring independence. Journal of Machine Learning Research, 2005, 6:2075-2129
[22] Bryc W, Dembo A, Kagan A. On the maximum correlation coefficient. Theory of Probability and Its Applications, 2005, 49(1):132-138
[23] Breiman L, Friedman J H. Estimating optimal transformations for multiple regression and correlation. Journal of the American Statistical Association, 1985, 80(391):580-598
[24] Papadatos N, Xifara T. A simple method for obtaining the maximal correlation coefficient and related characterizations. Journal of Multivariate Analysis, 2013, 118:102-114
[25] Dembo A, Kagan A, Shepp L A. Remarks on the maximum correlation coefficient. Bernoulli, 2001, 7(2):343-350
[26] Nelsen R B. An Introduction to Copulas. New York:Springer, 2006
[27] Shannon C E. A mathematical theory of communication. ACM SIGMOBILE Mobile Computing and Communications Review, 2001, 5(1):3-55
[28] Sugiyama M, Yamada M. On kernel parameter selection in hilbert-schmidt independence criterion. IEICE TRANSACTIONS on Information and Systems, 2012, 95(10):2564-2567
[29] Sejdinovic D, et al. Equivalence of distance-based and RKHS-based statistics in hypothesis testing. The Annals of Statistics, 2013, 41(5):2263-2291
[30] Micchelli C A, Xu Y S, Zhang H Z. Universal kernels. Journal of Machine Learning Research, 2006, 7(4):2651-2667