snowIRT distractor analysis

Everything related to development of the jamovi platform
Post Reply
lauramar
Posts: 15
Joined: Wed Oct 27, 2021 11:11 am

snowIRT distractor analysis

Post by lauramar »

Hi
Would it be possible to add an additional type of analysis to the one for distractors? In my context (language assessment) I want to get the item-rest value not just for the correct option but also for the distractors. This means that the incorrect options are treated as correct and the outcome has to be a negative correlation. I use this information to make decisions about how well the distractors work for that population.
For example, an older program TiaPlus (created by CITO for the educational testing context) produces this analysis. Here is the link to the user manual: https://tiaplus.cito.nl/TiaPlus_Users_Manual.pdf. Rir and rar are explained on pages 32 and 36 and those are the coefficients that I have in mind. I'm including a screenshot of the results of the analysis for visual illustration, the results with the asterisk are the correct option for the test question.
Many thanks in advance
Marju
Attachments
TiaPlus_rir_rar_results.JPG
TiaPlus_rir_rar_results.JPG (51.81 KiB) Viewed 2615 times
User avatar
seol
Posts: 155
Joined: Mon Dec 04, 2017 8:40 am
Location: Seoul, Korea
Contact:

Re: snowIRT distractor analysis

Post by seol »

Hi lauramar

Thanks for your suggestions. I think you should have some useful R codes somewhere, because with what you've presented, it's impossible for me to create an analysis because I don't fully understand it.

Best
Seol
User avatar
seol
Posts: 155
Joined: Mon Dec 04, 2017 8:40 am
Location: Seoul, Korea
Contact:

Re: snowIRT distractor analysis

Post by seol »

Hi lauramar

Distractor analysis provides the similar results for each option such as item-total correlations.

rspP The proportion of respondents with that response
pBis The point-biserial correlation between that reponse and the total score with that
item remove

Best
Seol
Attachments
이미지 004.png
이미지 004.png (106.56 KiB) Viewed 2431 times
lauramar
Posts: 15
Joined: Wed Oct 27, 2021 11:11 am

Re: snowIRT distractor analysis

Post by lauramar »

Hi Seol,

Thank you, I think this is exactly what I was looking for.

All the best
Marju
lauramar
Posts: 15
Joined: Wed Oct 27, 2021 11:11 am

Re: snowIRT distractor analysis

Post by lauramar »

Hi Seol,

I would like to ask whether it would be possible to add an option to the distractor analysis. Currently, it is possible to specify the number of groups under the Total summary and I'm assuming the program takes into account the total score achieved on the measure and then creates the groups accordingly. Would it be possible to allow to create the groups not based on the total score but using a variable? For example, in my dataset, I have a variable called 'level' (it means language proficiency level). Every test taker has been assigned a number by me. I would like the analysis to use my pre-determined groups to calculate the values in the Total Summary.

Best wishes

Marju
User avatar
seol
Posts: 155
Joined: Mon Dec 04, 2017 8:40 am
Location: Seoul, Korea
Contact:

Re: snowIRT distractor analysis

Post by seol »

Hi Marju

Thanks for your questions.

Distractor analysis is based on R package, so it is not easy to change the codes as you request.
Let me think. . .
how about selecting Filter option in jamovi and performing analysis on preset groups?

Best Regards
Seol
lauramar
Posts: 15
Joined: Wed Oct 27, 2021 11:11 am

Re: snowIRT distractor analysis

Post by lauramar »

Hi Seol,

Thank you for your suggestion, I also thought that would be a way to solve it. I have created the 'level' filters as I'm also interested in the relevant descriptive statistics. It is just that the group-level info is so nicely presented in the table and gives a really good overview of how the distractors work :-)

Best wishes
Marju
Post Reply