Many statistical learning problems, where one aims to recover an underlying low-dimensional signal, are based on optimization, e.g., the linear programming approach for recovering a sparse vector. Existing work often either overlooked the high computational cost in solving the optimization problem, or required case-specific algorithm and analysis -- especially for nonconvex problems. This talk addresses the above two issues from a unified perspective of conditioning. In particular, we show that once the sample size exceeds the intrinsic dimension of the signal, (1) a broad range of convex problems and a set of key nonsmooth nonconvex problems are well-conditioned, (2) well-conditioning, in turn, inspires new algorithm designs and ensures the efficiency of many off-the-shelf optimization methods.