Informative vs uninformative prior distributions with characteristic curve linking methods
Brandon LeBeau, Keyu Chen, Wei Cheng Liu, and Aaron McVay
University of Iowa
- With item response theory (IRT), the ability scale is arbitrarily defined (commonly mean of 0 and sd of 1).
- Linking is useful to help place individual ability and IRT item parameters on the same scale.
- Particularly when two forms are administered to non-equivalent groups.
- Four linking methods are common:
- Stocking Lord
- Random Groups
- Single group with counterbalancing
- Common-item nonequivalent group design
- More details in Kolen & Brennan (2014).
Common-item NEG Design
- The proficiency points and weights can be specified to reflect the ability distribution of the original scale.
- In addition, proficiency points and weights can be specified to reflect the ability distribution of the new scale.
- More details are provided in Kim & Lee (2006).
- To what extent does the prior distribution have an impact on the estimation of the transformation constants?
- To what extent does the relationship from #1 generalize across the simulation conditions?
Simulation Design 2
- The A and B transformation constants were also simulated as a part of the design.
- This was done in an attempt to increase generalizeability of study results.
- Both were simulated from a random uniform distribution.
- A ranged from 0.5 to 1.5 rounded to nearest .05 (21 possibilities)
- B ranged from -2 to 2 rounded to nearest 0.10 (41 possibilities)
- 1000 replications
- A population of 55 items were simulated as Form X from a normal ability distribution.
- Form Y consisted of common items from Form X (transformed based on A and B parameters).
- Additional items were simulated to fill out Form Y.
- Form Y was calibrated with Bilog-MG using a 3PL IRT model.
- Transformation constants were computed from calibrated Form Y item parameters and population Form X item parameters.
- An R package, plink, was used.
- Bias in the transformation constants (A and B) were explored descriptively and inferentially:
|Ability Dist:Prior Dist
|Ability Dist:A Pop
|Ability Dist:B Pop
|Prior Dist:A Pop
|Ability Dist:Prior Dist:B Pop
Results A Constant
Results B Constant
- Prior distribution used for linking the two forms does not have a large impact on the estimation of the A and B constants.
- Even correctly specifying the shape of the ability distribution through the weights does not help with non-normal ability distributions.
- The ability distribution shape has the most impact on accurate estimation of the A and B constants.
- Normalizing transformations of the ability distribution may be helpful to limit bias when estimating these linking constants.