Radical Multimodality: redrawing theoretical boundaries for the empirical study of contemporary communication

Professor John A. Bateman, Bremen University, Germany will speak on 30 October 2020.

Multimodality is the recognition that communicative practices commonly, perhaps always, complementarily engage multiple forms of expression. This recognition is now widespread and resurfaces in many disciplinary contexts. Almost ironically, however, many of these contexts present `multimodality' as if it were their own invention, with little interaction across boundaries. Although bridges between, and clusters of, approaches are beginning to emerge, more principled consolidation of theories, methods and results is hindered by foundational gaps concerning the nature of multimodality as a pervasive property of communication as such. Targets of investigation then appear sufficiently different that it remains unclear not only how interaction could be pursued but, indeed, even if it should. Art historians tend not to interact with conversation analysts, information design researchers with dance theory, comics research with music theory, human-computer interface designers with film theory, and so on. And yet, `multimodality' is a growing concern in each of these areas and many more. As a response to this situation, we have in Bremen over the past 15 years been concerned with developing a more robust foundation for work on multimodality, drawing particularly on perspectives from formal and functional linguistics and several forms of semiotics, and supporting an explicit orientation towards empirical research. In this talk, I briefly motivate this foundation, introducing the central theoretical categories of semiotic mode, medium and genre. Drawing on examples from both traditional and new media, I then outline some methodological guidelines for conducting multimodal research that the approach entails and show in particular how the more tightly formalised notion of semiotic mode assumed supports wide-ranging multimodal corpus-based studies that may be applied regardless of medium. Two reasons for taking a foundational perspective on the nature of multimodality will be emphasied throughout. First, contemporary communicative practices do not respect traditional disciplinary boundaries in any case and therefore remaining within those boundaries will compromise research. And second, working within a more general formal semiotic framework productively reveals principles and commonalities across diverse communicative practices and situations that would otherwise be difficult to access.