Very important in statistical theory is a statistical distribution called the normal distribution. It’s sometimes called the “bell curve” because of its shape. Looking at the plot of a normal distribution below, the high point around 0 means that’s where the majority of observations/people are, with fewer and fewer folks as you get more extreme in either the positive or negative direction. When the mean value of a normal distribution is 0 and the standard deviation is 1 (as in the below), that is a special form called the standard normal distribution.
It’s common in Item response theory (IRT) item parameter estimation to assume that the latent variable (the trait/concept/construct) being measured follows this standard normal distribution (the assumption of normality). The reason for this assumption is that it makes the calculations for getting items parameters a lot simpler, compared to if you didn’t make the assumption. Unfortunately for statisticians and psychometricians, not everything in the world is normally distributed. For instance, depression does not likely follow a bell-shaped curved like the one presented above – it’s more plausible that most people aren’t depressed, so the high point of the distribution (the bulk of the observations) is off to the low end (left) of the plot, like below.
For something really not normal, take a look at this estimated monstrosity from the analysis of an anxiety scale.
(Original found in Woods & Thissen’s 2006 Psychometrika article).
While research has shown that assuming the standard normal distribution for a population that isn’t standard normally distributed isn’t necessarily the end of the world (De Ayala, 1995; Stone, 1992; Woods & Lin, 2009), if a population distribution is decidedly non-normal and decidedly non-symmetric, parameter estimates can become quite biased/wrong (e.g., Woods, 2015). To avoid this “wrongness,” the obvious answer is to use an estimation method that doesn’t make any assumptions about the latent distribution. There are several different methods that appear promising for this, such as the use of empirical histograms (e.g. Woods, 2007) or Ramsay Curve Item Response Theory (e.g., Woods & Thissen, 2006; Monroe & Cai, 2014). Happily, these methods are becoming available in IRT software programs, such as flexMIRT®’s inclusion of empirical histograms. Even if the latent variable you’re studying ends up being as normal as normal can be, checking the shape of the distribution with one of these newer methods seems like it should become standard practice now that tools are readily at hand.