MTS 525-0
Special Topics Research Seminar
Section 20: Generalizing about Message Effects
Spring 2020
SYLLABUS: TOPIC 2
TOPIC 2: Describing primary research results: NHST and its discontents
Outline:
2.1 Null hypothesis significance testing
2.2 Effect sizes and confidence intervals
2.1 Null hypothesis significance testing
Cohen, J. (1994). The earth is round (p < .05). American Psychologist, 49, 997-1002. doi:10.1037/0003-066X.49.12.997
Nickerson, R. S. (2000). Null hypothesis significance testing: A review of an old and continuing controversy. Psychological Methods, 5, 241-301. doi:10.1037//1082-989X.5.2.241
Levine, T. R., Weber, R., Hullett, C., Park, H. S., & Lindsey, L. L. (2008). A critical assessment of null hypothesis significance testing in quantitative communication research. Human Communication Research, 34, 171-187. doi:10.1111/j.1468-2958.2008.00317.x
Nieuwenhuis, S., Forstmann, B. U., & Wagenmakers, E.-J. (2011). Erroneous analyses of interactions in neuroscience: A problem of significance. Nature Neuroscience, 14, 1105-1107. doi:10.1038/nn.2886
Amrhein, V., Greenland, S., & McShane, B. (2019). Scientists rise up against statistical significance. Nature, 567, 305-307. doi:10.1038/d41586-019-00857-9
For further reading:
Harlow, L. L., Mulaik, S. A., & Steiger, J. H. (Eds.) (1997). What if there were no significance tests? Mahwah, NJ: Lawrence Erlbaum.
Kline, R. B. (2013). Beyond significance testing: Reforming data analysis methods in behavioral research (2nd ed.) Washington, DC: American Psychological Association.
2.2 Effect sizes and confidence intervals
Levine, T. R., Weber, R., Park, H. S., & Hullett, C. (2008). A communication researchers’ guide to null hypothesis significance testing and alternatives. Human Communication Research, 34, 188-209. doi: 10.1111/j.1468-2958.2008.00318.x
Fidler, F. & Loftus, G. R. (2009). Why figures with error bars should replace p values: Some conceptual arguments and empirical demonstrations. Zeitschrift fur Psychologie, 217, 27-37. doi:10.1027/0044-3409.217.1.27
Cumming, G. (2014). The new statistics: Why and how. Psychological Science, 25(1), 7-29. doi:10.1177/0956797613504966
Austin, P. C., & Hux, J. E. (2002). A brief note on overlapping confidence intervals. Journal of Vascular Surgery, 36(1), 194-195. https://doi.org/10.1067/mva.2002.125015
O’Keefe, D. J. (2017). Misunderstandings of effect sizes in message effects research. Communication Methods and Measures, 11, 210-219. doi:10.1080/19312458.2017.1343812
For further reading:
Natrella, M. G. (1960). The relation between confidence intervals and tests of significance. American Statistician, 14(1), 20‑22, 38.
http://www.jstor.org/stable/2682129
Payton, M. E., Greenstone, M. H., & Schenker, N. (2003). Overlapping confidence intervals or standard error intervals: What do they mean in terms of statistical significance? Journal of Insect Science, 3(1), article 34. https://doi.org/10.1673/031.003.3401
Belia, S., Fidler, F., Williams, J., & Cumming, G. (2005). Researchers misunderstand confidence intervals and standard error bars. Psychological Methods, 10(4), 389-396. doi:10.1037/1082-989X.10.4.389
O’Keefe, D. J. (2011). The asymmetry of predictive and descriptive capabilities in quantitative communication research: Implications for hypothesis development and testing. Communication Methods and Measures, 5(2), 113-125. doi:10.1080/19312458.2011.568375
Cumming, G. (2012). Understanding the new
statistics: Effect sizes, confidence intervals, and meta-analysis. New
York, NY: Routledge.
Morey, R. D., Rouder, J. N., Verhagen, J., & Wagenmakers, E.-J. (2014). Why hypothesis tests are essential for psychological science: A comment on Cumming (2014). Psychological Science, 25, 1289–1290. doi:10.1177/0956797614525969
Greenland, S., Senn, S. J., Rothman, K. J., Carlin, J. B., Poole, C., Goodman, S. N., & Altman, D. G. (2016). Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations. European Journal of Epidemiology, 31, 337–350. doi:10.1007/s10654-016-0149-3
Home page for this course
Daniel J. O'Keefe home page
Daniel J. O'Keefe email