The first-year Wilkinson Microwave Anisotropy Probe (WMAP) set quantitative constraints on the amplitude of any primordial non-Gaussianity. We run a series of dark matter-only (DM-only) N-body simulations with the WMAP constraints to investigate the effect of the presence of primordial non-Gaussianity on large-scale structures. The model parameters can be constrained using the observations of protoclusters associated with Ly alpha emitters at high redshift ( 2 <= z <= 4), assuming that the galaxy velocity bias can be modelled properly. High-redshift structure formation potentially provides a more powerful test of possible primordial non-Gaussianity than does the cosmic microwave background, albeit on smaller scales. Another constraint is given by the local galaxy density probability distribution function (PDF), as mapped by the 2dF Galaxy Redshift Survey (2dFGRS). The PDF of 2dFGRS L-bJ* galaxies is substantially higher than the standard model predictions and requires either a non-negligible bias between galaxy and DM on similar to 12 h(-1) Mpc scales and a stronger non-Gaussianity than that allowed by the WMAP first-year data. The latter interpretation is preferred since second-order bias corrections are negative. With a lower normalization of the power spectrum fluctuations, sigma(8) = 0.74, as favoured by the WMAP three-year data, the discrepancy between the Gaussian model and the data is even larger.