snowIRT distractor analysis

[lauramar]

Hi
Would it be possible to add an additional type of analysis to the one for distractors? In my context (language assessment) I want to get the item-rest value not just for the correct option but also for the distractors. This means that the incorrect options are treated as correct and the outcome has to be a negative correlation. I use this information to make decisions about how well the distractors work for that population.
For example, an older program TiaPlus (created by CITO for the educational testing context) produces this analysis. Here is the link to the user manual: https://tiaplus.cito.nl/TiaPlus_Users_Manual.pdf. Rir and rar are explained on pages 32 and 36 and those are the coefficients that I have in mind. I'm including a screenshot of the results of the analysis for visual illustration, the results with the asterisk are the correct option for the test question.
Many thanks in advance
Marju

[seol]

Hi lauramar

Thanks for your suggestions. I think you should have some useful R codes somewhere, because with what you've presented, it's impossible for me to create an analysis because I don't fully understand it.

Best
Seol

[seol]

Hi lauramar

Distractor analysis provides the similar results for each option such as item-total correlations.

rspP The proportion of respondents with that response
pBis The point-biserial correlation between that reponse and the total score with that
item remove

Best
Seol

[lauramar]

Hi Seol,

Thank you, I think this is exactly what I was looking for.

All the best
Marju

[lauramar]

Hi Seol,

I would like to ask whether it would be possible to add an option to the distractor analysis. Currently, it is possible to specify the number of groups under the Total summary and I'm assuming the program takes into account the total score achieved on the measure and then creates the groups accordingly. Would it be possible to allow to create the groups not based on the total score but using a variable? For example, in my dataset, I have a variable called 'level' (it means language proficiency level). Every test taker has been assigned a number by me. I would like the analysis to use my pre-determined groups to calculate the values in the Total Summary.

Best wishes

Marju

[seol]

Hi Marju

Thanks for your questions.

Distractor analysis is based on R package, so it is not easy to change the codes as you request.
Let me think. . .
how about selecting Filter option in jamovi and performing analysis on preset groups?

Best Regards
Seol

[lauramar]

Hi Seol,

Thank you for your suggestion, I also thought that would be a way to solve it. I have created the 'level' filters as I'm also interested in the relevant descriptive statistics. It is just that the group-level info is so nicely presented in the table and gives a really good overview of how the distractors work

Best wishes
Marju