ROBUST DEPENDENCE MEASURE FOR DETECTING ASSOCIATIONS IN LARGE DATA SET

doi:10.1016/S0252-9602(17)30117-0

数学物理学报(英文版) ›› 2018, Vol. 38 ›› Issue (1): 57-72.doi: 10.1016/S0252-9602(17)30117-0

ROBUST DEPENDENCE MEASURE FOR DETECTING ASSOCIATIONS IN LARGE DATA SET

蒋杭进^1,2, 吴琼莉²

1. University of Chinese Academy of Sciences, Beijing 100049, China;
2. Key Laboratory of Magnetic Resonance in Biological Systems, Wuhan Institute of Physics and Mathematics, Chinese Academy of Sciences, Wuhan 430071, China

收稿日期:2017-01-26 出版日期:2018-02-25 发布日期:2018-02-25
通讯作者: Qiongli WU E-mail:wuqiongli@wipm.ac.cn
作者简介:Hangjin JIANG,E-mail:jianghangjin10@mails.ucas.ac.cn
基金资助:
Supported by the National Natural Science Foundation of China (31600290).

ROBUST DEPENDENCE MEASURE FOR DETECTING ASSOCIATIONS IN LARGE DATA SET

Hangjin JIANG^1,2, Qiongli WU²

1. University of Chinese Academy of Sciences, Beijing 100049, China;
2. Key Laboratory of Magnetic Resonance in Biological Systems, Wuhan Institute of Physics and Mathematics, Chinese Academy of Sciences, Wuhan 430071, China

Received:2017-01-26 Online:2018-02-25 Published:2018-02-25
Contact: Qiongli WU E-mail:wuqiongli@wipm.ac.cn
Supported by:
Supported by the National Natural Science Foundation of China (31600290).

摘要/Abstract

摘要：

In this paper, we proposed a new statistical dependency measure for two random vectors based on copula, called copula dependency coefficient (CDC). The CDC is proved to be robust to outliers and easy to be implemented. Especially, it is powerful and applicable to high-dimensional problems. All these properties make CDC practically important in related applications. Both experimental and application results show that CDC is a good robust dependence measure for association detecting.

关键词: CDC, dependence measure, EDC, association, large dataset, robust

Abstract:

Key words: CDC, dependence measure, EDC, association, large dataset, robust

蒋杭进, 吴琼莉. ROBUST DEPENDENCE MEASURE FOR DETECTING ASSOCIATIONS IN LARGE DATA SET[J]. 数学物理学报(英文版), 2018, 38(1): 57-72.

Hangjin JIANG, Qiongli WU. ROBUST DEPENDENCE MEASURE FOR DETECTING ASSOCIATIONS IN LARGE DATA SET[J]. Acta mathematica scientia,Series B, 2018, 38(1): 57-72.

参考文献

[1] Breiman L, Friedman J H. Estimating optimal transformations for multiple regression and correlation. J Am Stat Assoc, 1985, 80(391):580-598
[2] Dembo A, Kagan A, Shepp L A. Remarks on the maximum correlation coefficient. Bernoulli, 2001, 7(2):343-350
[3] Deng H, Wickham H. Density Estimation in R. Electronic Publication, 2011
[4] Fan J Q, Lv J C. Sure independence screening for ultrahigh dimensional feature space. J R Stat Soc B (Statistical Methodology), 2008, 70(5):849-911
[5] Ghahramani Z, Póczos B, Schneider J G. Copula-based kernel dependency measures. Proceedings of the 29th International Conference on Machine Learning (ICML-12), 2012:775-782
[6] Gretton A, Bousquet O, Smola A, Schölkopf B. Measuring statistical dependence with Hilbert-Schmidt norms. International Conference on Algorithmic Learning Theory, 2005:63-77
[7] Heller R, Heller Y, Gorfine M. A consistent multivariate test of association based on ranks of distances. Biometrika, 2013, 100(2):503-510
[8] Hoeffding W. A non-parametric test of independence. Ann Math Stat, 1948, 19(4):546-557
[9] Huang Q M, Zhu Y. Model-free sure screening via maximum correlation. J Multivariate Analysis, 2016, 148:89-106
[10] Jiang H J, Ding Y M. Equitability of Dependence Measure. arxiv stat, 2015
[11] Jiang H J, Shan Y, Wu Q L. Dependence measure:a comparative study. Acta Math Sci, 2017, 37A(5):931-949
[12] Lopez-Paz D, Hennig P, Schölkopf B. The randomized dependence coefficient. Advances in Neural Information Processing Systems, 2013:1-9
[13] Nelsen R B. An Introduction to Copulas. New York:Springer, 1999
[14] Papadatos N, Xifara T. A simple method for obtaining the maximal correlation coefficient and related characterizations. J Multivariate Anal, 2013, 118:102-114
[15] Rényi A. On measures of dependence. Acta Math Hung, 1959, 103:441-451
[16] Reshef D N, Reshef Y A, Finucane H K, et al. Detecting novel associations in large data sets. Science, 2011, 334(6062):1518-1524
[17] Schweizer B, Wolff E F. On nonparametric measures of dependence for random variables. Ann Stat, 1981, 9(4):879-885
[18] Silverman B W. Density Estimation for Statistics and Data Analysis. Boca Raton:CRC press, 1986
[19] Simon N, Tibshirani R. Comment on "detecting novel associations in large data sets" by Reshef et al, Science Dec 16, 2011. Science, 2014
[20] Székely G J, Rizzo M L. Brownian distance covariance. Ann Appl Stat, 2009, 3(4):1236-1265
[21] Tan Q H, Jiang H J, Ding Y M. Model selection method based on maximal information coefficient of residuals. Acta Math Sci, 2014, 34(2):579-592

ROBUST DEPENDENCE MEASURE FOR DETECTING ASSOCIATIONS IN LARGE DATA SET

ROBUST DEPENDENCE MEASURE FOR DETECTING ASSOCIATIONS IN LARGE DATA SET

PDF (PC)

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 6

Metrics

本文评价

推荐阅读 10

[1]	杨青, 朱家砚, 李正帮. MAXIMIN EFFICIENCY ROBUST TEST FOR MULTIPLE NUISANCE PARAMETERS AND ITS STATISTICAL PROPERTIES[J]. 数学物理学报(英文版), 2017, 37(1): 223-234.
[2]	杨建峰, 张润楚, 刘民千. CONSTRUCTION OF OPTIMAL BLOCKING SCHEMES FOR ROBUST PARAMETER DESIGNS[J]. 数学物理学报(英文版), 2013, 33(5): 1431-1438.
[3]	周云华. ROBUST WEAK ERGODICITY AND STABLE ERGODICITY[J]. 数学物理学报(英文版), 2013, 33(5): 1375-1381.
[4]	缪柏其, 吴月华, 刘东海. LIMITING BEHAVIOR OF RECURSIVE M-ESTIMATORS IN MULTIVARIATE LINEAR REGRESSION MODELS AND THEIR ASYMPTOTIC EFFICIENCIES[J]. 数学物理学报(英文版), 2010, 30(1): 319-329.
[5]	秦国友; 朱仲义. ROBUST ESTIMATION IN PARTIAL LINEAR MIXED MODEL FOR LONGITUDINAL DATA[J]. 数学物理学报(英文版), 2008, 28(2): 333-347.
[6]	王恺顺, 魏鸿增. GEOMETRIC CONSTRUCTION OF ASSOCIATION SCHEMES FROM DEGENERATE QUADRICS IN PROJECTIVE SPACES PG(2v +δ+l−1, Fq) FOR q ODD[J]. 数学物理学报(英文版), 2000, 20(1): 35-43.