Module development: can an analysis overwrite or replace existing columns?

[scro1461]

Hi,

I'm developing a jamovi module to help my students clean raw data: standardising inconsistent category labels, trimming whitespace, converting codes like -99 to missing, and tidying variable names. The aim is for them to do this in jamovi rather than Excel.

As I understand it, a module can only add new columns through `Output` options, and those stay linked to the analysis. Is there any way for a module to overwrite the original columns, or to convert an output variable into an ordinary data variable so the raw columns and the analysis can be deleted afterwards?

If neither is possible at the moment, would you consider adding a "convert to data variable" option for output columns? At present the only route I can find is copying and pasting each column by hand, which is error-prone for students with a lot of variables.

Thanks,
Joshua

[jonathon]

hi,

yeah, an analysis modifying existing columns is totally verbotten.

but in general terms, jamovi allows users to do these things already:

- standardising inconsistent category labels
- trimming whitespace
- converting codes like -99 to missing
- tidying variable names

in fact, trimming whitespace shouldn't be a thing ... jamovi *should* do that automagically, and typically does, so if you've found a gap, we'd appreciate a bug report.

but we'd be interested in hearing your use-case (user story) in more detail, and how jamovi currently falls short.

jonathon

[scro1461]

Hi Jonathon,

Thanks, and sorry, I hadn't realised Setup lets you edit several variables at once. That covers most of what I thought was missing on the fixing side.

Having thought about it more, the gap for my students is detection rather than fixing. They're undergraduates in criminology and the social sciences working with raw survey or administrative data, often exported from Qualtrics or supplied by agencies. They open a file and don't notice that a variable has "Male", "male" and "M" as separate levels, that 999 is sitting in age, or that a numeric variable came in as text, and they run analyses on it anyway.

So I'm planning a read-only audit module. Students select variables and get:
- a summary table per variable: type, inferred type, missingness and issue count
- an issues table flagging case-inconsistent levels, likely missing-value codes not yet declared, out-of-range values, numbers stored as text and similar problems
- a missingness chart showing which variables tend to be missing together
- a brief guide alongside each issue explaining what it means and how to fix it in jamovi, e.g. "set -99 as a missing value in Setup", with the multi-variable editing pointed out where it applies

Nothing gets written back to the data, so no column access is needed. The fixing stays in jamovi's own tools, and students re-run the audit to check. I'm looking at building the checks on the dataReporter R package rather than writing them from scratch.

Does that sound sensible from your side, and is there anything similar planned that I'd be duplicating? Any pointers on presenting guidance text in the results (an Html element, or notes on tables) would be welcome too.

Thanks,

Joshua

[jonathon]

Yup, this is sounding useful ... a lot of overlap with descriptives of course ... but i can see real merit in it. It's maybe something we should support out of the box! let's start with your implementation though.

For guidance text, use the Text options rather than Html

https://dev.jamovi.org/api/text

We're trying to move people away from Html as much as possible.

jonathon