The Same Analysis Approach: Detect, Avoid & Eliminate Confounds in Neuroimaging and other Data Analysis

31.05.2017 - 17:00
31.05.2017 - 19:00

Kai Görgen

PhD Student, BCCN Berlin

Classical design principles (e.g. randomization) and control analyses (e.g. on behavioural errors, reaction time, age) are routinely applied in many studies. It is typically not tested however whether these work together with new analysis methods, that involving e.g. cross-validation, classifiers, or permutation testing.I will show that – counterintuitively – such standard practices can lead to the exact opposite of what they should achieve: Classical design principles can induce confounds instead of controlling them, and standard control analyses can give false certainty that confounds have been controlled, even if they have not. This can cause systematic positive or negative biases (such as significant below-chance accuracies), potentially yielding false positive results or suppressing real effects.As a remedy, I present “the same analysis approach (SAA)” — a framework to detect, avoid, and eliminate a large class of potential confounds and other potential errors. The main idea is to perform the to-be-employed analysis on (i) design variables, (ii) control data, and (iii) artificial simulations. Although our examples come from neuroimaging, similar arguments apply to other fields such as psychology or machine learning.