Thank you, Don, for your post. Collection (attempted collection) of unpublished papers has a noteworthy downside in meta-analysis. This practice really should be reconsidered. CM
-----Original Message-----
From: Business Policy and Strategy List [mailto:
BPS-NET@AOMLISTS.PACE.EDU] On Behalf Of Don Bergh
Sent: Friday, September 14, 2012 3:31 PM
To:
BPS-NET@AOMLISTS.PACE.EDU
Subject: Re: Unpublished studies of Strategic Flexibility
Rob, Dennis and Hank (and others interested in meta-analysis):
The file drawer problem in meta-analysis might be more of a myth and urban legend than a threat to your findings. Unfortunately, your own findings could be influenced by who participates in this call and who doesn't, and whether participation can be assumed to be normally distributed.
See the article below for more information. It would appear that you are on strong footing to proceed without having to rely on unpublished study findings.
Good luck!
Don Bergh
REVISITING THE FILE DRAWER PROBLEM IN
META-ANALYSIS: AN ASSESSMENT OF PUBLISHED
AND NONPUBLISHED CORRELATION MATRICES
DAN R. DALTON
Department of Management and Entrepreneurship
Kelley School of Business
Indiana University
HERMAN AGUINIS
Department of Management and Entrepreneurship
Kelley School of Business
Indiana University
CATHERINE M. DALTON
Department of Management and Entrepreneurship
Kelley School of Business
Indiana University
FRANK A. BOSCO
Department of Management, Marketing, and MIS
Lewis College of Business
Marshall University
CHARLES A. PIERCE
Department of Management
Fogelman College of Business and Economics
University of Memphis
The file drawer problem rests on the assumption that statistically nonsignificant
results are less likely to be published in primary-level studies
and less likely to be included in meta-analytic reviews, thereby resulting
in upwardly biased meta-analytically derived effect sizes. We
conducted 5 studies to assess the extent of the file drawer problem in
nonexperimental research. In Study 1, we examined 37,970 correlations
included in 403 matrices published in Academy of Management Journal
(AMJ), Journal of Applied Psychology (JAP), and Personnel Psychology
(PPsych) between 1985 and 2009 and found that 46.81% of those correlations
are not statistically significant. In Study 2, we examined 6,935
correlations used as input in 51 meta-analyses published in AMJ, JAP,
PPsych, and elsewhere between 1982 and 2009 and found that 44.31%
of those correlations are not statistically significant. In Study 3, we examined
13,943 correlations reported in 167 matrices in nonpublished
manuscripts and found that 45.45% of those correlations are not statistically
significant. In Study 4, we examined 20,860 correlations reported
in 217 matrices in doctoral dissertations and found that 50.78% of those
correlations are not statistically significant. In Study 5, we compared
the average magnitude of a sample of 1,002 correlations from Study 1
(published articles) versus 1,224 from Study 4 (dissertations) and found
that they were virtually identical (i.e., .2270 and .2279, respectively).
In sum, our 5 studies provide consistent empirical evidence that the file
drawer problem does not produce an inflation bias and
Correspondence and requests for reprints should be addressed to Herman Aguinis, Department
of Management and Entrepreneurship,Kelley School of Business, IndianaUniversity,
1309 E. 10th Street, Suite 630D,Bloomington, IN 47405-1701;
haguinis@indiana.edu.
C
2012 Wiley Periodicals, Inc.
221
222 PERSONNEL PSYCHOLOGY
does not pose a
serious threat to the validity of meta-analytically derived conclusions as
is currently believed.
________________________________
From: Business Policy and Strategy List [
BPS-NET@AOMLISTS.PACE.EDU] On Behalf Of Wilde-Mes, P.J. de [
pmes@RSM.NL]
Sent: Wednesday, September 12, 2012 1:52 AM
To:
BPS-NET@AOMLISTS.PACE.EDU
Subject: Unpublished studies of Strategic Flexibility
Strategic Flexibility Meta-Analysis & The 'File Drawer Problem'
- we need your unpublished findings, please
[cid:
image001.jpg@01CD90CC.489ED7B0]
Rob Morgan (Cardiff University), Dennis Herhausen (St. Gallen University), and Henk Volberda (Erasmus University) are attempting to further improve our understanding of Strategic Flexibility as an important and often misunderstood theoretical construct, empirical phenomenon, and managerial posture. They are in the final stages of a meta-analytic study examining Strategic Flexibility. So far, they have sourced a large number of published, publicly available, and subscription-based empirical studies but have recently been seeking to address the 'file drawer problem' or the fact that they need to identify unpublished studies of Strategic Flexibility so as to limit publication bias in their data. Therefore, in order to improve their empirical insights, ensure a census of relevant studies, and reflect the appropriate significant and non-significant (commonly unpublished) results, they kindly request your assistance in completing this meta-analysis.
If you have any unpublished data with findings on a measure(s) of Strategic Flexibility, could you please contact Rob Morgan by email (
morganre@cardiff.ac.uk<mailto:
morganre@cardiff.ac.uk>) to discuss your work. Rob, Dennis, and Henk greatly appreciate your cooperation. Please also forward this request to any colleagues whom you consider might qualify.
-------------------------------------------------------------------------------------------disclaimer-------------------------------------------------------------------------
De informatie verzonden in dit e-mail bericht inclusief de bijlage(n) is vertrouwelijk en is uitsluitend bestemd voor de geadresseerde van dit bericht. Lees verder:
http://www.eur.nl/email-disclaimer
The information in this e-mail message is confidential and may be legally privileged. Read more:
http://www.eur.nl/english/email-disclaimer