A Comparative Study of the Normal and Non-Parametric Methods for Estimating IRT Ability Distributions

Many applications of item response theory (IRT) require that the population distribution for a latent ability variable should be specified or estimated. Given IRT item parameters and item response vectors of a sample of examinees from a population, the ability distribution may be estimated using the normal or non-parametric method. The purpose of this study was to use computer simulations to examine the characteristics and relative performance of the Gauss-Hermite quadrature and fixed-points quadrature based normal methods and the non-parametric method for estimating ability distributions. Test type, distribution shape, sample size, and number of ability points were considered as simulation factors. Main results were as follows. First, the two normal methods estimated properly the mean and standard deviation (SD) of the population distribution when the distribution was normal but they estimated the distribution parameters with some bias when the distribution was skewed. In contrast, the non-parametric method estimated the mean and SD without bias regardless of the distribution shape. Second, the standard errors of the mean and SD estimates decreased approximately inversely proportional to the square root of the sample size as the sample size increased. Third, overall, the fixed-points quadrature method estimated the distribution parameters with less bias and more stably than the Gauss-Hermite quadrature method. Fourth, when the number of ability points was 20 or 30, the distribution estimated by the non-parametric method tended to have a choppy shape.