Technological innovations have revolutionized the process of scienti�c research and knowledge discovery. Nowadays, massive abundance of data is not uncommon in many scientific areas ranging from genomics to economy. They give rise to many novel opportunities in knowledge discovery but come coupled with many unforeseen computational and statistical challenges, including scalability and storage bottleneck, noise accumulation, spurious correlation, incidental endogeneity, and measurement errors. These challenges are distinguished and require both new computational and new statistical paradigms. My current research lies at the frontier of such big data paradigm and focuses mostly on only one task of variable selection and feature extraction. In this talk I will expose the challenges that arise in this one task and show how optimization, probability and matrix analysis interplay in the new statistical theory.