Text

theses

Test speededness is defined as the failure to attempt all items on an assessment within a specified time frame. The presence of speededness is an issue known to undermine assessments (Bejar, 1985). Therefore, researchers have developed several approaches to reduce test speededness, including non-statistical methods (e.g., Evans & Reilly, 1972), as well as probabilistic models, such as augmented item response theory (IRT) models (e.g., Cao & Stokes, 2008). However, an assumption about speededness that is often not discussed in the literature is the relationship between speededness and ability of the examinee in the context of IRT modeling. Previous studies have used modified IRT models to reduce test speededness, but none have evaluated the effect of neglecting the association between speededness and ability. In the same regard, only one model has addressed this association. This study will address four different purposes regarding the relationship be- tween speededness and ability. The first purpose is to propose a new IRT model that associates ability with speededness and to develop an estimation algorithm for the proposed model, which is evaluated in terms of the recovery of model parameters by manipulating certain hyperparameters in the model. The second purpose of this study is to determine the robustness of the proposed IRT model when speededness is not present, and to show the inefficiency of the traditional IRT model when speededness is associated with ability. The third purpose is to examine the impact of ignoring the association between ability and speededness on parameter estimation and to investigate the robustness of the proposed model under conditions when speededness and ability are independent. Lastly, data are generated from an existing model that associated ability and speededness in a different manner to determine how robust the proposed model is under a different speededness schema. These four purposes are used to thoroughly understand the proposed model and its contribution to the research of test speededness. The Markov Chain Monte Carlo (MCMC) Metropolis Hastings algorithm was implemented to estimate model parameters using C++ and R, an object-oriented language and a statistical software, respectively. The results showed that the pro- posed model was able to recover the model parameters accurately under various conditions of known hyperparameters. The proposed model was also able to es- timate model parameters well when ability was not associated with speededness when there were a large amount of respondents and items. In addition, the pro- posed model was also able to estimate model parameters well when speededness was not present when the sample size and the number of items were large. Lastly, when speededness and ability were generated under a different method, the pro- posed model was unable to estimate the model parameters well. In summary, this work allows researchers to further understand the impact of speededness and its association with ability in a variety of conditions.

ETD_8017

Ph.D.

Includes bibliographical references

by Immanuel James Williams

doi:10.7282/T3R78J4S

ETD doctoral

The author owns the copyright to this work.