This article explores the complementary potential of intermedial and multimodal semiotic analysis. Both multimodal and intermedial research explore the multifaceted nature of communication. However, since research fields have different foci and analytical methods, they are less connected than they could be. We approach intermediality and multimodality as complementary frameworks and argue that there is much to gain in drawing on the analytical strengths of both. To this end, underlying differences in method and theoretical assumptions need to be made explicit. Drawing on John A. Bateman and Lars Elleström’s previous explorations of the common ground between the frameworks, we map an arena where multimodal and intermedial analysis can work together. We demonstrate how a combined multimodal and intermedial perspective can function by zooming in and out between the perspectives as we explore the role of “Ride of the Valkyries” in Wagner’s opera Die Walküre (1870), in a Nazi newsreel, and in Francis Ford Coppola’s movie Apocalypse Now (1979). The multimodal perspective provides us with nuanced language for transcribing and discussing how different semiotic resources work together, and the intermedial perspective allows us to discuss the chain of media transformation, where each instance increases and transforms the meaning potential of the “Ride of the Valkyries”.

In this essay, I use the song ‘Let It Go’ from Frozen as a steppingstone for addressing song and animated musical film as combined media, or rather integrations, and, based on the models put forward by Lars Elleström, propose a framework for discussing different kinds of synchronic intermediality. I propose that when analysing synchronic intermediality, we need to consider (at least) three types of combination: 1) combination of modalities (understood as formal structures framing the content), 2) combination of qualified aspects (understood as conventions of media products, herein those aspects tied to media representation), and 3) combination of semiotic content.

Animation and live-action are two closely related media, which are foremost distinguished by the ideas and conventions surrounding them. The diverging discourses around animation and live action have tended to focus on animation as something constructed to represent characters and settings and on live action as something capturing actors and sets representing characters and settings. This difference between constructing and capturing, along with the perceived indexicality of the photo, is what seems to suggest live action as the preferred medium for documenting real events. Sound effects, in the form of recorded and edited sounds of objects, actions and environments, are of particular interest here, as they can be considered to balance somewhere between these poles of construction and capture, between the non-indexical and indexical, and ultimately between representation and reproduction. In this article, I will focus on aspects of ‘truth’ (understood as corresponding to some external reality) and ‘realism’ (understood as a representation of external reality) and how something comes to be perceived as truthful or realistic in animated documentaries in relation to the role played by sound effects. By discussing the Danish film Flugt [Flee], I will show how sound effects can aid in creating representations of truth.

Media products of all sorts form a complex web of different relationships. Media products involve transformations, integrations and combinations as well as transmedial aspects. When we look at media combinations in this chapter, all these different aspects are brought into play. Media combinations of different basic media types are always, literally, intermedial combinations that involve intermedial relations between different forms of communication.

This chapter deals with different kinds of combinations of technical, basic and qualified media types in comics, films, radio drama, songs/singing and music videos. Some of them are more obvious in this aspect. The music video, for instance, integrates sound, words and (moving) images. The pages of comics display text and image, and these two basic media types communicate differently in the semiotic modality. In radio dramas or songs, the combination aspect is not as visually apparent. Still, the soundwaves of a song or a radio drama firmly integrate several auditory media types.

With specific examples, we discuss how to understand the different intermedial aspects at play whenever different media types are brought together in a particular media product. Different perspectives are possible. Should one focus on the combination of different forms of meaning-making, or instead stress how deeply integrated these different media types are? Should one approach a song as the combination of different qualified media (poetry and music), or focus on the close integration of different auditory media types (words and organized sound)? When we approach media combinations with the four modalities, we can focus on both. When words, (moving) images, and organized sounds are brought together in media products, such as comics, songs and music videos, they form an integrated whole. We can focus on how different basic media types in the material and sensorial modality are firmly integrated. We can then explore how these integrations on pages, in soundwaves, on stages or in the studio enable an intricate combination of different forms of meaning-making that support and interact with each other in the spatiotemporal and semiotic modality.

First, we will take a look at how words and images on pages convey a graphic narrative in comics. Then we will highlight the importance of sound effects in the complex combination of moving images and auditory media types in film. We then explore how different auditory basic and qualified media types together create a complex auditory narrative in radio drama, using the specific example of The Unforgiven (2018). We will also discuss how word and music combine on different levels in art and pop songs. The chapter will end with a few reflections on the audiovisual combinations of basic and qualified media types at work in music videos.

This chapter provides examples of basic analyses of macro and micro levels of intermediality in social media: YouTube entertainment, an example of a multi-layered social media practice, and GIFs, which derive from other media and migrate across different platforms on the internet. Apart from media combination and integration, even the two aspects of media transformation, transmediation and representation, are also persistent intermedial processes in social media practice as content creation. Social media entertainment stages the body, voice and personality, or persona, of a content creator outside the traditional media and acknowledges, or even addresses, the social community directly. In the authenticity discourse of social media entertainment, YouTubers pose themselves as an alternative to traditional media. The chapter looks at the Let’s Play genre and how the YouTuber PewDiePie engages with the social media entertainment dimensions and with his audiences.

When someone tells you about a film they have just seen, you would probably assume that the film in question is a film with sound that is based on recorded colour images of actors acting out a narrative, complete with synchronized dialogue, sound effects and music. The noun ‘film’ is ambiguous, however. The Merriam-Webster online dictionary provides eight different uses of the word – ranging from things such as food wrapping to the more relevant material of celluloid used for photographic imprints and film as a specific type of artistic object (Merriam-Webster 2020).

This chapter discusses the characteristics and communicative conventions that qualify Western art music, popular music and folk music. Despite major differences in the traditions of these qualified media types of music and the emphasis they put on musical notation and performance, they all share certain characteristics that relate to technical media of display and their basic media types. When we listen to sounding music, the technical media of display are sound waves that are realized via instruments and singers in a live performance or via electric speakers. The sounding music that is performed and the written sheet music both play an important part in this qualified media type of music. Where auditive text and organized sound really differ in terms of modalities is in the semiotic modality. The listener responds physically to the sounds and at the same time responds to them in the semiotic modality and transforms them into some form of subjective meaning.