MTS 525-0
 Special Topics Research Seminar

Section 20: Generalizing about Message Effects
 Spring 2020



TOPIC 1: Primary research design in message effects research



1.1 The logic of experimental message effects research

1.2 Single-message designs, case-category confounds

1.3 Multiple-message designs

1.4 Message variation definition and message description

1.5 Alternative comparison conditions

1.6 Some summary discussions




1.1 The logic of experimental message effects research


            Leshner, G. (2014). The basics of experimental research in media studies. The International Encyclopedia of Media Studies (Vol. 7, pp. 236–254).  doi:10.1002/9781444361506.wbiems181




1.2 Single-message designs, case-category confounds


1.2.1 Coleman (1964), Clark (1973), Kay and Richter (1977), and sequelae in psychology


            Clark, H. H. (1973). The language-as-fixed-effect fallacy: A critique of language statistics in psychological research. Journal of Verbal Learning and Verbal Behavior, 12, 335‑359. doi:10.1016/S0022-5371(73)80014-3

            Kay, E. J., & Richter, M. L. (1977). The category-confound: A design error. Journal of Social Psychology, 103(1), 57-63. doi:10.1080/00224545.1977.9713296


For further reading:

            Coleman, E. B. (1964). Generalizing to a language population. Psychological Reports, 14(1), 219-226.

            Fontenelle, G. A., Phillips, A. P., & Lane, D. M. (1985). Generalizing across stimuli as well as subjects: A neglected aspect of external validity. Journal of Applied Psychology, 70, 101-1107. doi:10.1037/0021-9010.70.1.101

            Wells, G. L., & Windschitl, P. D. (1999). Stimulus sampling and social psychological experimentation. Personality and Social Psychology Bulletin, 25, 1115-1125. doi:10.1177/01461672992512005  

            Judd, C. M., Westfall, J., & Kenny, D. A. (2012). Treating stimuli as a random factor in social psychology: A new and comprehensive solution to a pervasive but largely ignored problem. Journal of Personality and Social Psychology, 103, 54-69. doi:10.1037/a0028347

            Westfall, J., Kenny, D. A., & Judd, C. M. (2014). Statistical power and optimal design in experiments in which samples of participants respond to samples of stimuli. Journal of Experimental Psychology: General, 143, 2020-2045. doi:10.1037/xge0000014




1.2.2. Jackson and Jacobs (1983) and sequelae in communication


            Jackson, S., & Jacobs, S. (1983). Generalizing about messages: Suggestions for design and analysis of experiments. Human Communication Research, 9, 169‑181. doi:10.1111/j.1468-2958.1983.tb00691.x 


For further reading:

            Hunter, J. E., Hamilton, M., & Allen, M. (1989). The design and analysis of language experiments in communication. Communication Monographs, 56(4), 341-363. doi:10.1080/03637758909390269 

            Jackson, S., O’Keefe, D. J., Jacobs, S., & Brashers, D. E. (1989). Messages as replications: Toward a message-centered design strategy. Communication Monographs, 56(4), 364-384. doi:10.1080/03637758909390270

            Slater, M. D. (1991). Use of message stimuli in mass communication experiments: A methodological assessment and discussion. Journalism Quarterly, 68(3), 412-421. 

            Shapiro, M. A. (2002). Generalizability in communication research. Human Communication Research, 28, 491–500. doi:10.1111/j.1468-2958.2002.tb00819.x




1.3 Multiple-message designs


For further reading:

            Jackson, S. (1991). Meta-analysis for primary and secondary data analysis: The super-experiment metaphor. Communication Monographs, 58, 449-462.  doi:10.1080/03637759109376241

            Jackson, S. (1992). Message effects research: Principles of design and analysis. New York: Guilford Press.

            Jackson, S., & Brashers, D. (1994). M > 1: Analysis of treatment x replication designs. Human Communication Research, 20(3), 356-389. doi:10.1111/j.1468-2958.1994.tb00327.x

            Jackson, S., O’Keefe, D. J., & Brashers, D. (1994). The messages replication factor: Methods tailored to messages as objects of study. Journalism Quarterly, 71(4), 984-996. doi:10.1177/107769909407100421

            Brashers, D. E., & Jackson, S. (1999). Changing conceptions of “message effects”: A 24-year overview. Human Communication Research, 25, 457-477. doi:10.1111/j.1468-2958.1999.tb00456.x




1.4 Message variation definition and message description


            O’Keefe, D. J. (2003). Message properties, mediating states, and manipulation checks: Claims, evidence, and data analysis in experimental persuasive message effects research. Communication Theory, 13(3), 251-274.


For further reading:

            Bucy, E. P., & Tao, C.-C. (2007). The mediated moderation model of interactivity. Media Psychology, 9, 647-672. doi:10.1080/15213260701283269

            Tao, C.-C., & Bucy, E. P. (2007). Conceptualizing media stimuli in experimental research: Psychological versus attribute-based definitions. Human Communication Research, 33, 397-426. doi:10.1111/j.1468-2958.2007.00305.x

            Abraham, C., & Michie, S. (2008). A taxonomy of behavior change techniques used in interventions. Health Psychology, 27, 379-387. doi:10.1037/0278-6133.27.3.379

            Shen, L. J. (2019). Features of empathy-arousing strategic messages. Health Communication, 34, 1329-1339.




1.5 Alternative comparison conditions


            Shen, F., Sheer, V. C., & Li, R. (2015). Impact of narratives on persuasion in health communication: A meta-analysis. Journal of Advertising, 44, 105-113. doi:10.1080/00913367.2015.1018467

            Braddock, K., & Dillard, J. P. (2016). Meta-analytic evidence for the persuasive effect of narratives on beliefs, attitudes, intentions, and behaviors. Communication Monographs, 83, 446-467. doi:10.1080/03637751.2015.1128555




1.6 Some summary discussions


            Thorson, E., Wicks, R., & Leshner, G. (2012). Experimental methodology in journalism and mass communication research. Journalism & Mass Communication Quarterly, 89, 112–124. doi:10.1177/1077699011430066

            Slater, M. D., Peter, J., & Valkenburg, P. M. (2015). Message variability and heterogeneity: A core challenge for communication research. Annals of the International Communication Association, 39, 3-31.  doi:10.1080/23808985.2015.11679170


For further reading:

            Reeves, B., & Geiger, S. (1994). Designing experiments that assess psychological responses to media messages. In A. Lang (Ed.), Measuring psychological responses to media messages (pp. 165–180). Hillsdale, NJ: Erlbaum.

            Grabe, M. E., & Westley, B. H. (2003). The controlled experiment. In G. H. Stempel, III, D. H. Weaver, & G. C. Wilhoit (Eds.), Mass communication research and theory (pp. 267-298). Boston, MA: Allyn and Bacon.

            Highhouse, S. (2009). Designing experiments that generalize. Organizational Research Methods, 12, 554-566. doi: 10.1177/1094428107300396

            O’Keefe, D. J. (2015). Message generalizations that support evidence-based persuasive message design: Specifying the evidentiary requirements. Health Communication, 30, 106-113. doi:10.1080/10410236.2014.974123

            Reeves, B., Yeykelis, L., & Cummings, J. J. (2016). The use of media in media psychology. Media Psychology, 19, 49–71. doi:10.1080/15213269.2015.1030083



Home page for this course

Daniel J. O'Keefe home page

Daniel J. O'Keefe email