Excluding outliers with MAD

Discuss the jamovi platform, possible improvements, etc.
Vicente_inefo
Posts: 16
Joined: Wed May 29, 2019 11:39 pm

Excluding outliers with MAD

Post by Vicente_inefo »

Hi, is there a way to make a filter for the exclusion of outliers using the median absolute deviation (MAD) proposed by Leys et al., 2013 (https://www.sciencedirect.com/science/a ... 3113000668)

i.e. Median ± 2.5 * MAD.

Thanks in advance
User avatar
MAgojam
Posts: 421
Joined: Thu Jun 08, 2017 2:33 pm
Location: Parma (Italy)

Re: Excluding outliers with MAD

Post by MAgojam »

Hey @Vicente_inefo,

if you need it, my suggestion is the example with this screenshot:
Filter_MAD.png
Filter_MAD.png (59.76 KiB) Viewed 11303 times
Cheers,
Maurizio

UPDATE 2022-09-28
Sorry for the hint attached with the broken formula.
The attachment with the correct suggestion is in the post below, as a response to @anagrammarian, who drew my attention to the breakup.
Maurizio
Last edited by MAgojam on Wed Sep 28, 2022 5:27 pm, edited 1 time in total.
Vicente_inefo
Posts: 16
Joined: Wed May 29, 2019 11:39 pm

Re: Excluding outliers with MAD

Post by Vicente_inefo »

Thanks for the suggestion @MAgojam, I think it could be useful. I was thinking on calculate robust MAD using JASP and then calculate upper and lower limits for the MAD method for detecting outliers and applying it in JAMOVI
anagrammarian
Posts: 6
Joined: Tue Sep 27, 2022 4:48 pm

Re: Excluding outliers with MAD

Post by anagrammarian »

Maurizio,

I had a q re your formula. The original paper says M - 3 or 2.5* MAD and M + 3 or 2.5*MAD as the cutoffs. For MAD, they have an additional multiplier b which is 1.4826 i.e. MAD = b* MED(ABS( var - MED(var))) where b = 1.4826. Is there a reason why you omitted b in your formula? I noticed significantly more outliers obviously without that and adding 1.4826 and using 2.5 seemed to make sense.

Let me know what you think and thanks for introducing me to MAD.

-manoj
User avatar
MAgojam
Posts: 421
Joined: Thu Jun 08, 2017 2:33 pm
Location: Parma (Italy)

Re: Excluding outliers with MAD

Post by MAgojam »

anagrammarian wrote:Is there a reason why you omitted b in your formula?
The reason is that I did not double check how I should have done and forgot to add the scale constant as defined by (Huber, 1981).
I apologize to everyone who has passed by the post and trusted my suggestion.

Thank you so much manoj, for getting my attention and replying with the correct suggestion.
Correct_formula_MAD.png
Correct_formula_MAD.png (60.73 KiB) Viewed 10930 times
Cheers,
Maurizio
Vicente_inefo
Posts: 16
Joined: Wed May 29, 2019 11:39 pm

Re: Excluding outliers with MAD

Post by Vicente_inefo »

I could confirm that the result of the proposed formula matches with the MAD result reported in JASP.

Thanks to all the participants.
User avatar
jonathon
Posts: 2613
Joined: Fri Jan 27, 2017 10:04 am

Re: Excluding outliers with MAD

Post by jonathon »

should we add a MAD() function to our formula list?

jonathon
Vicente_inefo
Posts: 16
Joined: Wed May 29, 2019 11:39 pm

Re: Excluding outliers with MAD

Post by Vicente_inefo »

It sounds good to me!!
User avatar
jonathon
Posts: 2613
Joined: Fri Jan 27, 2017 10:04 am

Re: Excluding outliers with MAD

Post by jonathon »

if someone can create an issue here, describing in detail how it's calculated, i'll see about adding it.

https://github.com/jamovi/jamovi/issues

cheers

jonathon
User avatar
MAgojam
Posts: 421
Joined: Thu Jun 08, 2017 2:33 pm
Location: Parma (Italy)

Re: Excluding outliers with MAD

Post by MAgojam »

If it is of interest and can help, Rj is there:
Rj_mad.png
Rj_mad.png (76.57 KiB) Viewed 8358 times
Cheers,
Maurizio
Post Reply