This talk is intended for those who, like the speaker, have at some point wondered whether there is a theory of three- or higher-dimensional matrices that parallels matrix theory. A $d$-dimensional hypermatrix may be viewed as a coordinate representation of an order-$d$ tensor but we will explain why it is not quite the same. We discuss how notions like rank, norm, determinant, eigen and singular values may be generalized to hypermatrices. We will see that, far from being artificial constructs, these notions have appeared naturally in a wide range of applications and can be enormously useful.