Consider the random bipartite Erdos-Renyi graph $G(n, m, p)$, where each edge with one vertex in $V_{1}=[n]$ and the other vertex in $V_{2}=[m]$ is connected with probability $p$ with $n \geq m$. For the centered and normalized adjacency matrix $H$, it is well known that the empirical spectral measure will converge to the Marchenko-Pastur (MP) distribution. However, this does not necessarily imply that the largest (resp. smallest) singular values will converge to the right (resp. left) edge when $p = o(1)$, due to the sparsity assumption. In Dumitriu and Zhu 2024, it was proved that almost surely there are no outliers outside the compact support of the MP law when $np = \omega(\log(n))$. In this paper, we consider the critical sparsity regime with $np =O(\log(n))$, where we denote $p = b\log(n)/\sqrt{mn}$, $\gamma = n/m$ for some positive constants $b$ and $\gamma$. For the first time in the literature, we quantitatively characterize the emergence of outlier singular values. When $b > b_{\star}$, there is no outlier outside the bulk; when $b^{\star}< b < b_{\star}$, outlier singular values only appear outside the right edge of the MP law; when $b < b^{\star}$, outliers appear on both sides. Meanwhile, the locations of those outliers are precisely characterized by some function depending on the largest and smallest degrees of the sampled random graph. The thresholds $b^{\star}$ and $b_{\star}$ purely depend on $\gamma$. Our results can be extended to sparse random rectangular matrices with bounded entries.