lnu.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Many analysts, one dataset: Making transparent how variations in analytical choices affect results
University of Sussex Business School, UK.
INSEAD, The Business School of the World, Asia Campus, Singapore.
University of Virginia, USA.
University of Padua, Italy.
Show others and affiliations
2018 (English)In: Advances in Methods and Practices in Psychological Science, ISSN 2515-2459, Vol. 1, no 3, p. 337-356Article in journal (Refereed) Published
Abstract [en]

Twenty-nine teams involving 61 analysts used the same dataset to address the same research question: whether soccer referees are more likely to give red cards to dark skin toned players than light skin toned players. Analytic approaches varied widely across teams, and estimated effect sizes ranged from 0.89 to 2.93 in odds ratio units, with a median of 1.31. Twenty teams (69%) found a statistically significant positive effect and nine teams (31%) observed a non-significant relationship. Overall 29 differentanalyses used 21 unique combinations of covariates. We found that neither analysts' prior beliefs about the effect, nor their level of expertise, nor peer-reviewed quality of analysis readily explained variation in analysis outcomes. This suggests that significant variation in the results of analyses of complex data may be difficult to avoid, even by experts with honest intentions. Crowdsourcing data analysis, a strategy by which numerous research teams are recruited to simultaneously investigate the same research question, makes transparent how defensible, yet subjective analytic choices influence research results.

Place, publisher, year, edition, pages
Sage Publications, 2018. Vol. 1, no 3, p. 337-356
National Category
Psychology (excluding Applied Psychology)
Research subject
Social Sciences, Psychology
Identifiers
URN: urn:nbn:se:lnu:diva-70907DOI: 10.1177/2515245917747646OAI: oai:DiVA.org:lnu-70907DiVA, id: diva2:1182729
Note

Correction published in: Corrigendum: Many Analysts, One Data Set: Making Transparent How Variations in Analytic Choices Affect Results. (2018). Advances in Methods and Practices in Psychological Science. https://doi.org/10.1177/2515245918810511

Available from: 2018-02-14 Created: 2018-02-14 Last updated: 2018-12-11Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full text

Authority records BETA

Carlsson, Rickard

Search in DiVA

By author/editor
Carlsson, Rickard
By organisation
Department of Psychology
Psychology (excluding Applied Psychology)

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 59 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf