Speakers: Dr Anna Wilson (University of Oxford), Seb Wilkes (University of Oxford), Dr Scott Hale (University of Oxford)
Abstract: The use of misinformation and disinformation campaigns has attracted much attention from academics and policy-makers. Multimodal analysis or the analysis of two or more semiotic systems (language, gestures, images, sounds, among others) in their interrelation and interaction is essential to understand mis/disinformation efforts because most human communication goes beyond just words. There is a confluence of many disciplines (Linguistics, Political Science, Communication Studies, Computer Science) that are developing methods and analytical models of multimodal communication. This review brings these research strands together, providing a map of the multi/interdisciplinary research landscape for multimodal analysis, and identifies future cross-disciplinary research directions.
This review addresses the question: What are the main challenges for the research area of multimodal analysis of mis/disinformation? To answer this question, we conducted a comprehensive review of English-language scholarly articles featuring multimodal studies of mis/disinformation communication in traditional and social media.
Our analysis suggests that there is a lack of high-quality, truly interdisciplinary studies on multimodal analysis of dis/misinformation done at scale. For instance, despite a surge of interest from computer science seldom are methods and approaches from other fields incorporated. The gap in research manifests itself at every level of our analysis: there is no unified set of terminology to refer to mis/disinformation and these differences fall along disciplinary divides), research subcommunities cite distinct parts of the literature. There is a high diversity of theoretical frameworks used by researchers. Crucially, this review notes the absence of a well-established research community focusing on multimodal analysis of dis/misinformation and offers suggestions of how to do so in the form of a future research agenda for the field.
About Dr Anna Wilson: Dr Anna Wilson is REES Head of Language Studies, Teaching Fellow and Senior Research Fellow at Oxford School of Global and Area Studies, University of Oxford. She is Director of International Multimodal Communication Centre, University of Oxford. She is Co-Principal Investigator on the project “World Futures: Multimodal Viewpoint Construction by Russian International Media” funded by AHRC and DFG. She is also leading on the project “International Multimodal Communication Collaboration” funded by the John Fell OUP Research Fund.
About Seb Wilkes: Seb is a research assistant with a background in physics. He hopes to bring a quantitative and dynamics-based perspective to multimodal communication.
About Dr Scott Hale: Dr Scott A. Hale is an Associate Professor and Senior Research Fellow at the Oxford Internet Institute and a Fellow of the Alan Turing Institute. He develops and applies techniques from computer science to research questions in the social sciences. His research seeks to see more equitable access to quality information and investigates the spread of information between speakers of different languages online, the roles of bilingual Internet users, collective action and mobilization, hate speech, and misinformation.
