A joint project of the Graduate School, Peabody College, and the Jean & Alexander Heard Library

Title page for ETD etd-04032013-015440

Type of Document Master's Thesis
Author Nelson, Michael Cader
Author's Email Address michael.nelson@vanderbilt.edu
URN etd-04032013-015440
Title New tools for intervention fidelity assessment: an empirical comparison of explanatory multidimensional IRT and CTT approaches
Degree Master of Science
Department Psychology
Advisory Committee
Advisor Name Title
David Cordray Committee Chair
Andrew Tomarken Committee Member
Bethany Rittle-Johnson Committee Member
Sun-Joo Cho Committee Member
  • MIRT
  • IRT
  • experiments
  • interventions
  • treatment integrity
  • fidelity
  • implementation
  • item response theory
  • RCT
  • randomized controlled trials
Date of Defense 2012-10-30
Availability unrestricted
Intervention fidelity (Nelson, Cordray, Hulleman, Darrow, & Sommer, in press) is the extent to which an intervention has been implemented as planned in the treatment group, and differentiated from the control group, in the context of a randomized controlled experiment (RCT). Education researchers are seeking more and better tools for measuring intervention fidelity, but approaches have varied widely among researchers, and there are few direct comparisons of different analytical methods.

IRT approaches may be especially capable of overcoming difficulties associated with analyzing intervention fidelity data, including skewed distributions, multidimensionality, and poorly-defined constructs. A recent development in explanatory multidimensional IRT (MIRT) is a model that detects group differences and individual differences simultaneously for multidimensional tests (Cho, Athay, & Preacher, in press). Model results then can be compared directly with factorial analysis of variance (ANOVA, Kirk, 1968) results.

The primary goal of this study is to demonstrate parallel analyses of empirical intervention fidelity data with both the traditional ANOVA using total scores and this particular MIRT model. The comparison shows the unique strengths of explanatory MIRT for intervention fidelity analyses, allowing researchers to assess its benefits over classical test theory (CTT) approaches, as well as the feasibility of MIRT analysis for their data. Secondarily, the results of this study show that choice of analytical method for intervention fidelity analysis can lead to somewhat different statistical conclusions. It is recommended that the sources of such deviations be investigated through simulation studies and other methods in the future.

  Filename       Size       Approximate Download Time (Hours:Minutes:Seconds) 
 28.8 Modem   56K Modem   ISDN (64 Kb)   ISDN (128 Kb)   Higher-speed Access 
  mnelson.pdf 477.83 Kb 00:02:12 00:01:08 00:00:59 00:00:29 00:00:02

Browse All Available ETDs by ( Author | Department )

If you have more questions or technical problems, please Contact LITS.