High-order numerical methods promise higher fidelity and more predictive power when compared with traditional low-order methods. Furthermore, many properties of these methods make them well-suited for modern computer architectures. However, the use of these methods also introduces several new challenges. For example, in the time-dependent setting, high-order spatial discretizations can result in severe time step restrictions, motivating the use of implicit solvers. The resulting systems are large and often ill-conditioned, posing a challenge for traditional solvers. In this talk, I will discuss the development of efficient solvers and preconditioners designed specifically for high-order finite element and discontinuous Galerkin methods. An entropy-stable sparse line-based discretization will be developed to make these methods suitable for use on GPU- and accelerator-based architectures. These methods will then be applied to relevant problems in compressible flow.