Computed variables not showing?

General help and assistance with jamovi. Bug reports can be made at our issues page: https://github.com/jamovi/jamovi/issues . (If you're unsure feel free to discuss it here)

by jonathon » Fri Sep 06, 2019 11:16 am

hi,

so if we add two quantities in statistics, say A + B, and we don't know the value of A, then we don't know the value of A + B. so if the value of A is missing, then the results of A + B will also be missing.

so in the example of your data set, you have a lot of missing values, and when you sum these together, if *any* of the summed values are 'missing', then the result will also be missing.

if you want to ignore the missing values, you can add the `ignore_missing=1` argument, i.e.

SUM(INON7,TT7,TDT7, ... , ignore_missing=1)

cheers
User avatar
jonathon
 
Posts: 1341
Joined: Fri Jan 27, 2017 10:04 am

by Bellsie » Fri Sep 06, 2019 2:11 pm

THANK YOU SO MUCH!!
Bellsie
 
Posts: 10
Joined: Wed Apr 24, 2019 9:20 am

by Bellsie » Thu Mar 12, 2020 10:55 am

Help! I haven't touched Jamovi in months and am having the same problem...

I need to calculate the variable PDI TOTAL, sum of PDI1T - PDI21T but the column just goes blank. I don't have any missing data...

Please help! I've attached the file.
Last edited by Bellsie on Mon Apr 13, 2020 11:24 am, edited 1 time in total.
Bellsie
 
Posts: 10
Joined: Wed Apr 24, 2019 9:20 am

by jonathon » Thu Mar 12, 2020 11:02 am

hi,

get rid of the quotation marks surrounding your variable names :)

quotation marks makes it text. without quotation marks refers to the column/variable. if you need to refer to a variable with a space in it, then surround it with 'backticks' (you'll find the backtick key to the top left of your keyboard).

Screen Shot 2020-03-12 at 21.59.06.png
Screen Shot 2020-03-12 at 21.59.06.png (113.14 KiB) Viewed 1244 times


cheers

jonathon
User avatar
jonathon
 
Posts: 1341
Joined: Fri Jan 27, 2017 10:04 am

by Bellsie » Thu Mar 12, 2020 11:11 am

You are an absolute star! I can't thank you enough!
Bellsie
 
Posts: 10
Joined: Wed Apr 24, 2019 9:20 am

by jonathon » Thu Mar 12, 2020 11:12 am

glad i could help bellsie :)

cheers

jonathon
User avatar
jonathon
 
Posts: 1341
Joined: Fri Jan 27, 2017 10:04 am

by peter_hilpert » Tue Apr 28, 2020 11:38 am

Hi Jonathon,

I'm working on a jamovi training for our department. However, when I make up simple data sets and test the effect of different missing values, I get sometimes wrong results. Maybe we can figure out what is going wrong so that I can come up with good suggestions on how to handle missing values.

1. Reading in data

It generally works when I have a specific missing value value (e.g., -99) and use the Import, default missing option. However, when I change the default missing to any other number (e.g., 159) and import the data again, it still recognizes the -99 but not the 159. Why is that?
It also seems to work if some cells are simply empty (sysmis in SPSS).

2. Recoding
I have set up a small trial data set in order to test if recoding works with missing values.

For variable A, I chose continuous, decimal and missing value -99 and include the following values

A
1
2
-99
3
[empty cell)
5

I transform A into a recoded variable: '6 - $ source' and get the following

A A_r
1 5
2 4
-99 105
3 3
[empty cell keeps empty]
5 1

Although jamovi shows me that the missing value of -99 is recognized (it's in grey), it recodes it to 105.

3. Computing variable
The same thing happens when I compute a new variable (mean). Here is the data set I have created (continuous, decimal, again I test missing values -99 and empty cells).

A B
1 2
1
1 -99

-99 -99
-99

I compute a mean score by using = MEAN(A,B)

A B mean
1 2 1.5
1
1 -99 -49

-99 -99 -99
-99

If there is an empty cell, I get nothing. However, in a row with only -99 or actual numbers, -99 is included even though it is in grey, indicating that jamovi knows that this is a missing.

And this is what I get when including the ignore_missing command: MEAN(A,B, ignore_missing=1):

A B mean
1 2 1.5
1 1
1 -99 -49

-99 -99 -99
-99

4. z-score
And the same problem I get with z-scores.

A
1
2

3
-99

Using = Z(A) I get:
A Az
1 0.48
2 0.50

3 0.52
-99 -1.50

That is pretty wired too.

So, I'm not sure what is going on. However, what I also could see that if I create a new variable with transform without transforming it yet, for example, the -99 is not grey anymore in the new variable bit still in the old variable. So it includes the -99 in the new variable?

I guess the simplest suggestion would be to tell everyone to clean the data before, using only one defined missing value. But again, this does only seem to work for missing values (empty cells) but now if I include the -99.

Sorry for the long email. But would be great to figure this...

Thanks a lot and with warm regards,
Peter
peter_hilpert
 
Posts: 5
Joined: Tue Mar 10, 2020 11:45 am

by jonathon » Tue Apr 28, 2020 11:46 am

hi peter,

for future reference, you're better off creating a new post, rather than adding to an old one like this.

It generally works when I have a specific missing value value (e.g., -99) and use the Import, default missing option. However, when I change the default missing to any other number (e.g., 159) and import the data again, it still recognizes the -99 but not the 159. Why is that?
It also seems to work if some cells are simply empty (sysmis in SPSS).


so i'm not completely sure what you're describing here. are you opening a .csv file? opening a .sav file, or importing a .csv or .sav file into an existing data set?

perhaps if you could provide detailed steps, i could reproduce this.

I have set up a small trial data set in order to test if recoding works with missing values.


i'm embarrassed to say, that when we were working on the missing value system i never thought about this. you're quite right, the current behaviour isn't correct. the -99 should be treated as a missing value for computing/transforming values too.

i'll sit down and try and have this fixed tomorrow. thanks for bringing it to our attention.

with thanks

jonathon
User avatar
jonathon
 
Posts: 1341
Joined: Fri Jan 27, 2017 10:04 am

by peter_hilpert » Wed Apr 29, 2020 8:43 am

Hi Jonathon,

Ok, I will open a new post for the next one...

For the first problem, I have used an SPSS file. When I change the missing value value, it seems to stuck with the -99.

Yes, for the other problems, it seems to be the case the definition of the missing value is not pushed over to the newly created variables. Thanks for having a look at it!

And another question. When will it be possible to work in Rj editor and push the variables created via R syntax into the jamovi spreadsheet? This would be super useful for teachers like me. Then I can choose to teach the jamovi data handling tool to those who never will use R syntax - but I can teach data handling using R syntax directly for those who will learn more R. That would be pretty cool too...

Thanks a lot Jonathon!
peter_hilpert
 
Posts: 5
Joined: Tue Mar 10, 2020 11:45 am

by jonathon » Wed Apr 29, 2020 8:52 am

hey,

For the first problem, I have used an SPSS file. When I change the missing value value, it seems to stuck with the -99.


can you attach the .sav file, and then outline the steps you take, what happens, and what you expect?

Yes, for the other problems, it seems to be the case the definition of the missing value is not pushed over to the newly created variables. Thanks for having a look at it!


yup. i've fixed this now, and it will be in the next release. the next release is a largish one, so we're being a bit careful with it. but it should be in the next few days.

And another question. When will it be possible to work in Rj editor and push the variables created via R syntax into the jamovi spreadsheet? This would be super useful for teachers like me. Then I can choose to teach the jamovi data handling tool to those who never will use R syntax - but I can teach data handling using R syntax directly for those who will learn more R. That would be pretty cool too...


so it looks like we've received some funding to implement the analysis -> data set pipeline, so that side of things should happen in the next couple of months. rj taking advantage of that will be a bit more work. any chance you could convince your uni to sponsor it?

with thanks

jonathon
User avatar
jonathon
 
Posts: 1341
Joined: Fri Jan 27, 2017 10:04 am

Previous

Return to Help