[Ml-stat-talks] Seminar TODAY: Jianqing Fan, 12:30 PM, Sherrerd 101
Amir Ali Ahmadi
a_a_a at princeton.edu
Wed Mar 11 10:30:23 EDT 2015
DATE: Wednesday, March 11, 2015
TIME: 12:30 – 1:30 p.m. (Lunch at 12:00 in Sherrerd 125)
LOCATION: Room 101, Sherrerd Hall
SPEAKER: Jianqing Fan
TOPIC: Spurious discoveries, random geometry, and endogeneity for Big Data (abstract below)
Spurious discoveries, random geometry, and endogeneity for Big Data
Over the last two decades, many exciting variable selection methods have been developed for finding a small group of covariates that are associated with the response from a large pool. Can the discoveries by such data mining approaches be spurious? Can our fundamental assumptions on exogeneity of covariates needed for such variable selection be validated with the data? To answer these questions, we need to derive the distributions of the maximum spurious correlations given certain number of predictors. When the covariance matrix of covariates possesses the restricted eigenvalue property, we derive such distributions, using Gaussian approximation and empirical process techniques. However, such a distribution depends on the unknown covariance matrix of the covariate. Hence, we propose a multiplier bootstrap method to approximate the unknown distributions and establish the consistency of such a simple bootstrap approach. The results are further extended to the situation where residuals are from regularized fits. Our approach is then applied to construct the upper confidence limit for the maximum spurious correlation and testing exogeneity of covariates. The former provides a baseline for guiding false discoveries due to data mining and the latter tests whether our fundamental assumptions for high-dimensional model selection are statistically valid. Our techniques and results are illustrated by both numerical examples.
(Based on a joint paper with Qiman Shao and Wenxin Zhou)
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Ml-stat-talks