Revealed or concealed? transparency of procedures, decisions, and judgment calls in meta-analyses

Zeynep G. Aytug, Hannah R. Rothstein, Wencang Zhou, Mary C. Kern

Research output: Contribution to journalReview article

58 Scopus citations

Abstract

The authors examined the degree to which meta-analyses in the organizational sciences transparently report procedures, decisions, and judgment calls by systematically reviewing all (198) meta-analyses published between 1995 and 2008 in 11 top journals that publish meta-analyses in industrial and organizational psychology and organizational behavior. The authors extracted information on 54 features of each meta-analysis. On average, the meta-analyses in the sample provided 52.8% of the information needed to replicate the meta-analysis or to assess its validity and 67.6% of the information considered to be most important according to expert meta-analysts. More recently published meta-analyses exhibited somewhat more transparent reporting practices than older ones did. Overall transparency of reporting (but not reporting of the most important items) was associated with higher ranked journals; transparency was not significantly related to number of citations. The authors discuss the implications of inadequate reporting of meta-analyses for development of cumulative knowledge and effective practice and make suggestions for improving the current state of affairs.

Original languageEnglish
Pages (from-to)103-133
Number of pages31
JournalOrganizational Research Methods
Volume15
Issue number1
DOIs
StatePublished - 1 Jan 2012

    Fingerprint

Keywords

  • meta-analysis
  • missing data
  • quantitative research

Cite this