Please refer to the other companion html files (S3, ART accuracy, and S4, ART rts) for more detailed description of the task and pre-processing procedures

Robustness check

With preregistered analyses we have provided evidence for the two-way interaction GVS by Reward. On the contrary, we failed to provide evidence for the three-way GVS by Validity by Reward.

In this document we ran secondary analyses and robustness checks in order to collect converging statistical evidence around the same family of tests of interest. Basically, we aim to respond to the same questions we attempted to respond to with the main inferential approach, which was based on mixed linear regression models. After this first step, we will look for data-driven and post-hoc results in a separate section.

The analyses presented in this document, with the exception of Bayesian ANOVAs, have not been pre-registered, thus they ought to be interpreted with caution.

Preliminary setup

It’s sometimes good to clean the current environment to avoid conflicts. You can do it with rm(list=ls()) (but be sure everything is properly saved for future use).

In order to run this script we need a few packages available on CRAN. You might need to install them first, e.g. by typing install.packages("BayesFactor") in the console.

#list packages
packages= c("psych", "plyr", "magrittr", "tidyverse", "BayesFactor", "lme4", "multcomp",
            "gridExtra", "ez" , "lsmeans", "retimes")

#load them
lapply(packages, require, character.only= T)

For package afex I’m using the developer version on github (you will need to install devtools).

#devtools::install_github("singmann/afex@master")
library(afex)

For reproducibility, here’s details of the system on which the script was tested and a seed for random numbers:

set.seed(1)

sessionInfo()
## R version 3.4.2 (2017-09-28)
## Platform: x86_64-w64-mingw32/x64 (64-bit)
## Running under: Windows 10 x64 (build 15063)
## 
## Matrix products: default
## 
## locale:
## [1] LC_COLLATE=English_United Kingdom.1252 
## [2] LC_CTYPE=English_United Kingdom.1252   
## [3] LC_MONETARY=English_United Kingdom.1252
## [4] LC_NUMERIC=C                           
## [5] LC_TIME=English_United Kingdom.1252    
## 
## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     
## 
## other attached packages:
##  [1] afex_0.18-0          retimes_0.1-2        lsmeans_2.27-2      
##  [4] estimability_1.2     ez_4.4-0             gridExtra_2.3       
##  [7] multcomp_1.4-7       TH.data_1.0-8        MASS_7.3-47         
## [10] survival_2.41-3      mvtnorm_1.0-6        lme4_1.1-14         
## [13] BayesFactor_0.9.12-2 Matrix_1.2-11        coda_0.19-1         
## [16] dplyr_0.7.4          purrr_0.2.3          readr_1.1.1         
## [19] tidyr_0.7.1          tibble_1.3.4         ggplot2_2.2.1       
## [22] tidyverse_1.1.1      magrittr_1.5         plyr_1.8.4          
## [25] psych_1.7.8          printr_0.1          
## 
## loaded via a namespace (and not attached):
##  [1] nlme_3.1-131        pbkrtest_0.4-7      lubridate_1.6.0    
##  [4] RColorBrewer_1.1-2  httr_1.3.1          rprojroot_1.2      
##  [7] tools_3.4.2         backports_1.1.1     R6_2.2.2           
## [10] rpart_4.1-11        Hmisc_4.0-3         lazyeval_0.2.0     
## [13] mgcv_1.8-20         colorspace_1.3-2    nnet_7.3-12        
## [16] mnormt_1.5-5        compiler_3.4.2      rvest_0.3.2        
## [19] quantreg_5.33       htmlTable_1.9       SparseM_1.77       
## [22] xml2_1.1.1          sandwich_2.4-0      checkmate_1.8.4    
## [25] scales_0.5.0        pbapply_1.3-3       stringr_1.2.0      
## [28] digest_0.6.12       foreign_0.8-69      minqa_1.2.4        
## [31] rmarkdown_1.7       base64enc_0.1-3     pkgconfig_2.0.1    
## [34] htmltools_0.3.6     htmlwidgets_0.9     rlang_0.1.2        
## [37] readxl_1.0.0        bindr_0.1           zoo_1.8-0          
## [40] jsonlite_1.5        gtools_3.5.0        acepack_1.4.1      
## [43] car_2.1-5           modeltools_0.2-21   Formula_1.2-2      
## [46] Rcpp_0.12.13        munsell_0.4.3       stringi_1.1.5      
## [49] yaml_2.1.14         grid_3.4.2          parallel_3.4.2     
## [52] forcats_0.2.0       lattice_0.20-35     haven_1.1.0        
## [55] splines_3.4.2       hms_0.3             knitr_1.17         
## [58] reshape2_1.4.2      codetools_0.2-15    stats4_3.4.2       
## [61] glue_1.1.1          evaluate_0.10.1     latticeExtra_0.6-28
## [64] data.table_1.10.4   modelr_0.1.1        nloptr_1.0.4       
## [67] MatrixModels_0.4-1  cellranger_1.1.0    gtable_0.2.0       
## [70] assertthat_0.2.0    coin_1.2-1          xtable_1.8-2       
## [73] broom_0.4.2         lmerTest_2.0-33     bindrcpp_0.2       
## [76] cluster_2.0.6

Thanks to the function retrieved here, not displayed, the following hyperlinks download the Rdata files: