A Sequential Penalized Likelihood Procedure and EBIC for Feature Selection with Big Data

Abstract

[evdate 2013-12-19] HKUST International Forum on Probability and Statistics. Talk no.4.

Title from slide title.

The Second HKUST International Forum on Probability and Statistics (2013), held 19 December, 2013, at the Hong Kong University of Science and Technology. Co-sponsored by the HKUST Jockey Club Institute for Advanced Study and the Center for Statistical Science.

Abstract: In this talk, the speaker discuss a sequential method for variable selection in ultra-high dimensional feature space - the sequential penalized likelihood cum EBIC approach. In principle, the method selects features by sequentially solving partially penalized likelihood problems where the features selected in earlier steps are not penalized and uses the extended BIC (EBIC) as the stopping rule. Speaker first consider the method in the case of linear models. The asymptotic properties of the method are considered under the setting that the dimension of the feature space is ultra-high and the number of relevant feature diverges. And show that, with probability converging to 1, the method first selects all the relevant features before any irrelevant features can be selected, and that the EBIC decreases until it attains the minimum at the model consisting of exactly all the relevant features and then begins to increase. These results establish the selection consistency of the method. Then the speaker consider the generation of the method to the case of generalized linear models. We also consider the method for models with interactions. The sequential penalized likelihood cum EBIC approach is compared with other methods by simulation studies, which demonstrates its edge over the other methods for feature selection in ultra-high dimensional space.

Duration: 32 min.