上海财经大学张耀武教授：高维数据中的非线性关系和独立性检验
报告摘要
Measuring the nonlinear dependence between random vectors and testing for their statistical independence is a fundamental problem in statistics and has attracted increasing attention in recent years. However, most existing works have focused on either fixed or very high-dimensional covariates. To bridge this gap, we offer statistical insights into the behavior of popular independence measurements, such as DC and HSIC, as the dimensions vary at diverse rates. We first show that, under the null hypothesis, the rescaled measurements converge in distribution to the standard normal distribution. Subsequently, we provide a general condition that ensures the tests based on these measurements possess nontrivial power in high dimensions. By decomposing this condition, we illustrate how the ability of the measurements to detect nonlinear dependence changes with increasing dimensions. Moreover, we demonstrate that, depending on the sample size, the covariate dimensions and the dependence structures within covariates, these measurements can capture different types of associations between random vectors. Finally, to enhance the power of the tests in sparse high-dimensional scenarios, we introduce a cross-validation procedure which incorporates feature screening with the independence tests. Extensive numerical studies are conducted to validate our theoretical results.
嘉宾简介
张耀武，上海财经大学信息管理与工程学院常任教授，数据、算法与工程系系主任。主要从事复杂数据关联分析的统计理论、方法和应用研究，代表性研究成果发表在AOS, Biometrika, JOE, POM, JMLR, NeurIPS, ICML等顶级期刊和会议上。担任杉数科技科学家顾问，为京东、顺丰、华为、南航等多个国内著名企业提供数据分析和供应链管理等技术服务。
直播分享时间：2024年5月18日
顶部