jamovi

@Marielle that is a good point. We might give that a try.

@reason180 In generell we might never want to recreate what the students did, but we want to teach them and understand that there work has to be reproducable. It might not apply to this task but we want them to be aware of it and teach them the right scientifc culture.
So the how to reproduce does not matter that much, it is more the culture of documenting everything you do to the data that is important to us. And it should be in a way that is easy availible and readable. And yes in SPSS they have to hand in there Syntax files, they do not have to write the syntax themself but have to save it, and we strongly advise them to comment there syntax so they know what it does.
In our field it is more and more that you provide a dataset and your code with a paper that you send to a journal, so the reader or interested person can recreate and maybe look at other aspects you did not mention. And sending an jamovi project in is harder than an R-Code or SPSS Syntax. And it is not sure that your jamovi project works the same in upcoming Versions, so you would also have to provide and save the jamovi-Verison you build on. That is also true for SPSS and R but these programms are well more established and changes to core functions that we manly use are unlikely.

@DDS Thanks for the clarification!

I really think that 100% the data-modification syntax you want is already there in the jamovi file--just distributed across different locations rather than compiled into one block of text. So with jamovi, educating students about code preservation and reproducibility would entail students understanding how the jamovi method of preserving that code is different from that of other platforms. I don't think there's anything wrong with the method provided by @Marielle, of having student take screenshots. But in the end it's just duplicative screenshots and doesn't address the main sticking point, which to me is captured by the following:

[. . .] "not sure that your jamovi project works the same in upcoming Versions, so you would also have to provide and save the jamovi-verison you build on. That is also true for SPSS and R but these programs are well more established and changes to core functions that we mainly use are unlikely."

Even if, tomorrow, jamovi were to gain the capability to output its Python data-transformation-syntax to a file (instead of keeping it distributed across different locations within the GUI), jamovi would still be a young platform, causing many to still distrust that it is outputting the syntax correctly.

a few responses

> it is more the culture of documenting everything you do to the data that is important to us. And it should be in a way that is easy availible and readable.

this is exactly why jamovi is designed the way it is. every step, computation, filter, analysis is stored for future evaluation. spss solves this with 'syntax', but of course that requires learning how the syntax corresponds to the UI. i'd argue this isn't a great solution, as it requires you to specify the analysis with one language (drag and drop ui), and read or inspect the analysis with a different language (syntax).

of course, many people have spent their careers learning to read and write syntax, so it's no big deal for them ... but it's the sort of thing we set out to make unnecessary when designing jamovi... we think our approach makes everything "easily available and readable" in a way that SPSS syntax doesn't.

> And sending an jamovi project in is harder than an R-Code or SPSS Syntax.

it sounds like your work flow requires a static document (a pdf), but of course that won't be true for everyone.

having said that, i can see that there are situations where "inspecting" a jamovi file means navigating the data set to inspect, say, the computed columns to inspect their formulas ... and this would represent a burden on markers. so i definitely see the case for including this information into the results document itself.

> And it is not sure that your jamovi project works the same in upcoming Versions, so you would also have to provide and save the jamovi-Verison you build on. That is also true for SPSS and R but these programms are well more established and changes to core functions that we manly use are unlikely.

i think you'll find that you can open very old .omv files created in very old versions, and not have any problems with compatibility. i can think of a handful of situations where you might not get exactly what you want, but these are pretty marginal.

R on the other hand has breaking changes all the time. i know this, because there's a lot of R code under the hood in jamovi, and we have to deal with breaking changes pretty often. the R community (and i mean the R core project here too, not just CRAN) is more willing to break things than other programming communities. (i think there were even breaking changes to the t.test() function in the last 18 months!)

that's my take.

jonathon