MTS 525-0
 Special Topics Research Seminar

Section 20: Generalizing about Message Effects
 Spring 2020



TOPIC 2: Describing primary research results: NHST and its discontents




2.1 Null hypothesis significance testing

2.2 Effect sizes and confidence intervals




2.1 Null hypothesis significance testing


            Cohen, J. (1994). The earth is round (p < .05). American Psychologist, 49, 997-1002. doi:10.1037/0003-066X.49.12.997

            Nickerson, R. S. (2000). Null hypothesis significance testing: A review of an old and continuing controversy. Psychological Methods, 5, 241-301. doi:10.1037//1082-989X.5.2.241

            Levine, T. R., Weber, R., Hullett, C., Park, H. S., & Lindsey, L. L. (2008). A critical assessment of null hypothesis significance testing in quantitative communication research. Human Communication Research, 34, 171-187. doi:10.1111/j.1468-2958.2008.00317.x

            Nieuwenhuis, S., Forstmann, B. U., & Wagenmakers, E.-J. (2011). Erroneous analyses of interactions in neuroscience: A problem of significance. Nature Neuroscience, 14, 1105-1107. doi:10.1038/nn.2886 

            Amrhein, V., Greenland, S., & McShane, B. (2019). Scientists rise up against statistical significance. Nature, 567, 305-307.  doi:10.1038/d41586-019-00857-9    


For further reading:

            Harlow, L. L., Mulaik, S. A., & Steiger, J. H. (Eds.) (1997). What if there were no significance tests?  Mahwah, NJ: Lawrence Erlbaum.

            Kline, R. B. (2013). Beyond significance testing: Reforming data analysis methods in behavioral research (2nd ed.) Washington, DC: American Psychological Association.





2.2 Effect sizes and confidence intervals


            Levine, T. R., Weber, R., Park, H. S., & Hullett, C. (2008). A communication researchers’ guide to null hypothesis significance testing and alternatives. Human Communication Research, 34, 188-209. doi: 10.1111/j.1468-2958.2008.00318.x

            Fidler, F. & Loftus, G. R. (2009). Why figures with error bars should replace p values: Some conceptual arguments and empirical demonstrations. Zeitschrift fur Psychologie, 217, 27-37. doi:10.1027/0044-3409.217.1.27

            Cumming, G. (2014). The new statistics: Why and how. Psychological Science, 25(1), 7-29. doi:10.1177/0956797613504966 

            Austin, P. C., & Hux, J. E. (2002). A brief note on overlapping confidence intervals. Journal of Vascular Surgery, 36(1), 194-195. 

            O’Keefe, D. J. (2017). Misunderstandings of effect sizes in message effects research. Communication Methods and Measures, 11, 210-219. doi:10.1080/19312458.2017.1343812


For further reading:

            Natrella, M. G. (1960). The relation between confidence intervals and tests of significance. American Statistician, 14(1), 20‑22, 38.

            Payton, M. E., Greenstone, M. H., & Schenker, N. (2003). Overlapping confidence intervals or standard error intervals: What do they mean in terms of statistical significance? Journal of Insect Science, 3(1), article 34.

            Belia, S., Fidler, F., Williams, J., & Cumming, G. (2005). Researchers misunderstand confidence intervals and standard error bars. Psychological Methods, 10(4), 389-396. doi:10.1037/1082-989X.10.4.389

            O’Keefe, D. J. (2011). The asymmetry of predictive and descriptive capabilities in quantitative communication research: Implications for hypothesis development and testing. Communication Methods and Measures, 5(2), 113-125. doi:10.1080/19312458.2011.568375

            Cumming, G. (2012). Understanding the new statistics: Effect sizes, confidence intervals, and meta-analysis. New York, NY: Routledge.

            Morey, R. D., Rouder, J. N., Verhagen, J., & Wagenmakers, E.-J. (2014). Why hypothesis tests are essential for psychological science: A comment on Cumming (2014). Psychological Science, 25, 1289–1290. doi:10.1177/0956797614525969

            Greenland, S., Senn, S. J., Rothman, K. J., Carlin, J. B., Poole, C., Goodman, S. N., & Altman, D. G. (2016). Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations. European Journal of Epidemiology, 31, 337–350. doi:10.1007/s10654-016-0149-3 



Home page for this course
Daniel J. O'Keefe home page
Daniel J. O'Keefe email