The values of the famous j-invariant at quadratic irrationalities in the upper half plane are known as singular moduli and are of particular interest in number theory. Recently, Zagier realized the generating series of the traces of the singular moduli as a classical meromorphic modular form of weight 3/2. In this talk, we will discuss this and related results and give a generalization to modular functions on Riemann surfaces of arbitrary genus. Furthermore, we realize a certain generating series of arithmetic intersection numbers and Faltings heights as the derivative of Zagier's Eisenstein series of weight 3/2. This recovers a result of Kudla, Rapoport and Yang. This is joint work with Jan Bruinier.