In this talk, I will discuss some models and algorithms from two different fields: (1) machine learning, including logistic regression and deep learning, and (2) numerical PDEs, including multigrid methods. I will explore mathematical relationships between these models and algorithms and demonstrate how such relationships can be used to understand, study and improve the model structures, mathematical properties and relevant training algorithms for deep neural networks. In particular, I will demonstrate how a new convolutional neural network known as MgNet, can be derived by making very minor modifications of a classic geometric multigrid method for the Poisson equation and then explore the theoretical and practical potentials of MgNet.