Kitabı oku: «Schrift - Bild - Ton (E-Book)», sayfa 3

Teil 1: Multimodales Schreiben im Bildungskontext
Multimodality and Writing: Academic Voice across Modes

Arlene Archer

Introduction

I am the director of the Writing Centre at the University of Cape Town. This is a voluntary drop-in centre where students at all levels of study can come and consult about their academic writing. We also run workshops and seminars on writing which are more embedded within the disciplines. As social, political and economic power is closely associated with knowledge of certain discourse forms, our Writing Centre aims to promote and facilitate access to higher education, within an ethos of social justice and national redress in the South African context. Our ethos is that the Writing Centre can work effectively with students if that work is situated within a desire to understand and negotiate diversity and difference rather than the institutional need to manage it (Archer 2010).

I am currently engaged in a three-year British Academy funded project called Changing Writing. There are three themes in the project. The first theme, which this chapter feeds into, includes looking at how academic voice and argument are constructed through different modes. A mode is the culturally shaped material available for representation such as visual mode, oral mode or written mode. Secondly, the project is looking at ways of teaching and assessing writing in diverse contexts. And lastly, it is examining how writing is re-configured in different media, focusing on the materiality of the media, the mobility of the media, the imagined audiences, the spaces and places of writing. A paper-based manga comic, as compared to the same comic on screen, is a good example of the effects of the materiality of the media. The book as object does play a role in narrative and in comics, page turning typically functions as a transition device. Materiality, in this way, is incorporated into the narrative. Compare that to the online version. Many comics found online are actually produced for the page, which means that viewing them on the screen can place particular constraints on them. For example, the page on screen cannot be viewed as a full image but can be seen only when scrolled down. This affects the flow in the reading. Online comics, however, have other affordances. They allow access to other volumes of work, connection to a community and provide spaces for comment (see Huang & Archer 2014). In this example, it is clear how writing and image as modes translate differently onto the media of page and screen.

Drawing on multimodal social semiotics as a framework, this chapter aims to explore a number of theoretical tools which can «account for the whole domain of representation; which do so aptly for the specificities of each mode» (Kress 2010, 105). It aims to explore semiotic signifiers of academic voice within a range of modes and genres, and to develop a critical metalanguage or framework for academic voice, in order to assist students both in the production and critique of multimodal texts.

Voice as enabling critical access to dominant practices

At the crux of Writing Centre work is what we call the access paradox – how to provide students with access to dominant forms and practices related to academic writing and genres, whilst at the same time enabling critique of those practices (Archer 2010). By dominant practices, I mean dominant languages, language varieties, discourses, modes of representation, genres, and types of knowledge. The focus on critical access by the Writing Centre is especially important in the light of ongoing student protest action across Higher Education in South Africa over the last three years which began with the Rhodes must fall campaign, and the ongoing calls for decolonization of the curriculum. Ngugi wa Thiong’o describes how colonialism on the African continent not only included the taking of land and resources, but importantly included the «mental universe of the colonized, the control, through culture, of how people perceived themselves and their relationship to the world» (1986, 16).

Voice is a key concept in Higher Education which can enable student awareness of their own agency within the constraints of academic practices. It is not a new concept, but the investigation here is new in that it focuses on voice across modes, media and genres. Voice is an elusive concept with multiple definitions. It can be used as a proxy for an essentializing humanist view, as in finding your own voice (Elbow 2007). Or, in progressivist pedagogy, voice can be seen as taking a certain standpoint and also as a critical term for formulating an alternative pedagogy (Cope & Kalantzis 1993). Usefully, Clark and Ivanič (1997) speak about writers creating a discoursal self through the act of writing and the discourses they enter into as they write.

Why then use a mode-specific term like voice which is so closely linked to the mode of speech (and writing)? The concept of voice comes with a particular history, and with particular theoretical resonances which allow us to look at text in a way that draws on theories of writing and an academic literacies perspective. An academic literacies approach to writing takes into account issues of identity, institutional relationships of discourse and power, and the contested nature of writing practices (Lea & Street 1998, 159). The field of academic literacies has a long history of theorizing agency that a term like stance or style may not have. Stance refers to authorial point of view (Hyland 1999) whereas voice is more concerned with the relation between the writer and reader. I use the term voice to refer to the way a sign-maker establishes presence in a text through the choice and use of semiotic resources (signifiers of authorial engagement), as well as positioning in relation to other sources, and positioning in relation to the audience and site of display.

Instead of thinking about voice as aligned to coherence, which is perhaps a more liberal humanist notion of voice, I think about voice as negotiating and containing contesting voices. This is Bakhtin’s (1981) notion of dialogism which emphasizes voice as a site of struggle. However, these choices are not infinite, and voice is subject to contextual conditions which are located within larger patterns of inequality and power relations (Blommaert 2005). This notion of voice operating within constraints provides a useful way of thinking about voice and agency. There are contextual constraints which include institutional culture, as well as discursive practices, such as disciplinary genres. Besides semiotic alternatives for meaning-making, there are also semiotic constraints within different modes and media, as we saw with the manga comic on screen versus on paper.

To stay with this idea of constraints for a while, Simon Bell (2016) reports on student writing projects in the context of the Higher Education subject of design. The projects were characterized by artificially imposed constraints such as: writing within image and shape restrictions; writing within font restrictions; and writing within word restrictions. Bell argues that paradoxically more weight is given to each word by using so few. This sharpens the focus on the words, «making them work harder to produce meaning» (Bell 2016, 141). This is a lot like a PechaKucha style presentation. In this type of presentation, there are only 20 slides in the powerpoint. Each slide is to last 20 seconds and they are to be comprised of images and no writing. Here the constrictions of form can also enable a new kind of thinking and a tightening of argument, for example, finding an image to illustrate a concept could tighten or broaden your notion of that concept. Constricting time can also provide a certain energy to a presentation in the way that slides containing a list of bullet points cannot. However, as with any form, there are certain gains and losses. Definitions of concepts are difficult to do in images and are often less precise than when realized through writing. For instance, I could try to capture my definition of voice in four images – as being about presence in text, choice of semiotic resources, positioning in relation to other sources (citation), and in relation to audience (see Abbildung 1).

Abbildung 1: Definition of voice portrayed visually

Choice is represented by a fork in the road, citation as a head amongst smaller heads located in a speech bubble, and audience is held in the palm of your hand. It is clear that the images function more metaphorically here and less clearly and explicitly than a written definition would. How well this definition works is thus debatable. In addition, these images are all taken from open access websites. These kinds of generic images are problematic in some ways, as the sign-maker has to force a connection between the image and the concept. My point here is that constraints can enable creativity and different ways of thinking, but also that certain modes and genres are better suited for different functions (and definitions tend to work better in language than in image).

A framework for academic voice

In order to engage with the concept of voice, we need appropriate semiotic categories for analysis that work across modes. I would like to propose two categories for looking at academic voice. The first is authorial engagement which concerns the extent to which authors choose to engage with their audiences and subject matter (Hyland 1999) and is realized through different mode-specific conventions. The second category is citation. Citation is central to the construction of academic voice and is marked by positioning in relation to a particular discourse community. I will begin by focusing on authorial engagement.

Authorial engagement

Authorial engagement is expressed through choice, namely the type and the composition of the representation. These representational choices depend on context – aptness for purpose and audience. In writing, authorial engagement can be revealed through attitude markers, the use of pronouns, active and passive voice, and rhetorical questions. Authorial engagement can even be established through the visual design of the writing such as typographical choices, the use of punctuation (for instance, the use of commas to pause and breathe), layout, the use of white space, spelling (is it grey or is it gray?) and ways of asserting identity through cool visual lexicons (such as homophones is short text messages).

In images, authorial engagement can be demonstrated through aspects such as the choice or type of image, how the image is sourced, the use of colour and even colour saturation, composition (what is foregrounded and backgrounded), and social relations between represented participants in an image (see Archer 2013). In terms of composition, Kress and Van Leeuwen talked about the concepts of given and new in their 2006 book, Reading Images. A grammar of visual design. Perhaps they have moved away from these more prescriptive notions now, but the fact remains that in composition, position matters. For example, in the past, the figure of Johnny Walker on the whiskey bottle was represented as walking from right to left. This directionality was seen as signifying movement into the past and it was then changed to the current logo where he walks from left to right, signifying walking into the future. Positionality and directionality as such are thus important semiotic resources for meaning-making.

Now, I will look at a how authorial engagement is realized in infographics and data visualizations. It is important to be aware of the construction of these texts as the assumptions underlying the numbers are generally hidden and numerical representations are often regarded as more factual and objective than other kinds of evidence. Here we need to look at: what is measured and summarized; the type and choice of information graphic; the composition of the graphic (Prince & Archer 2014). There are design choices to be made here in terms of size, shape, colour and composition in order to represent an argument to a particular audience in the most apt way. One could, for example, choose the default design option when creating a bar chart to represent data. This would comprise blue bars on a grey background. There are a host of other design choices that could be made, however, such as using different colours for different bars for ease of comparison or making the spaces between the bars narrower or wider, including horizontal lines to facilitate ease of reading. The range of options for representation of information is endless and can be creative depending on the communicative purpose and audience (see Abbildung 2).

Abbildung 2: Authorial engagement in infographics

The representational choices made in Abbildung 2 may not be appropriate in certain academic disciplines, and may, in fact, weaken academic voice, whereas they function well in popular journalistic genres. However, in creating information graphics, we need to be aware of the full range of representational resources in order to make appropriate selections for the particular domain in which we are operating, both in terms of content and voice. Understanding the relation between agency and constraints is crucial for exploring how we construct authorial voice here. As Thesen describes it, «voice, like language, is never neutral; it is always in tension between pulls towards convention (centripetal forces) and pushes away from the centre towards more hybrid, experimental and open forms» (Thesen 2014, 6). I have shown that authorial engagement in information graphics is largely realized through the choice of type of graphic and the composition of the graphic.

I want to move on to think about something a little different, to see how the notion of authorial engagement manifests in three-dimensional texts. In certain courses and subjects, students need to produce three-dimensional texts for assessment. Although these texts do not necessarily involve writing, they are academic texts produced for assessment. Examples of such text include architectural models, jewellery designs, art, engineering models. I have argued that authorial engagement is made manifest in designed artefacts through material choices around the surface of the artefact, the substance of the artefact, and through the traces of the tools of production (Archer 2018). In terms of the surface of the artefact, colour contributes to meaning-making through associations or symbolic meanings and through combinations of colours, hues, degrees of contrast, degrees of saturation and purity. Surface texture is another aspect to consider, for instance, metals, natural materials and synthetics could have high shine, be polished or unpolished, be hammered, embossed, rough or smooth, shiny or matt. Finally, light and shadow are of interest in terms of surface choices. Light reflects off surfaces and is influenced by the surface texture and material. In Abbildung 3, the surface texture of the earrings is smooth and polished. Transparency and opacity are employed as semiotic resources, as is light and surface reflection.

Abbildung 3: Authorial engagement manifest in the surface of the artefact

Choices around the substance of the artefact also realize voice and include the stuff or material out of which the artefact is made, such as iron, wire, paper. The physical materials could be raw or processed, natural (such as wood, bone, stone, shell) or synthetic (such as plastic or rubber), durable or non-durable, upcycled (as in Abbildung 4). Finally, the tools and the traces of the tools used in the production of an artefact also realize voice. The traces can include pen inscriptions rather than printed ink or erased pencil marks, scraping or marks from hammering or chiselling, as seen in the flattened bottle top earrings in Abbildung 4.

The advantage of looking at the three-dimensional texts above in the way that I have is to show that voice manifests in a number of different kinds of texts that students need to produce, and to work with the concept of voice and see how it manifests differently across different modes, media and genres. I have argued that authorial engagement is expressed through choice; the type of representation chosen and the compositional choices made. I have suggested ways in which these choices manifest differently across modes. Now, I will move on to discuss the ways in which academic voice is realized through citation.

Abbildung 4: Earrings upcycled from cooldrink bottle tops

Citation in different modes

Citation is a resource used in the design of meanings in socially shaped and regular ways and is central to academic voice. Citation across modes involves appropriating a source into your argument and using the voices of others to negotiate your position in a particular discourse community. The rhetorical function of citation is thus to enter the academic conversation. The term citation works better across modes than a term like quoting or paraphrasing or even referencing. It is thus useful for talking about multimodal texts.

We are very familiar with the conventions around citation in writing and how this impacts on voice. For instance, the choice of where to place a reference in academic writing has implications for authorial voice. A reference placed at the beginning of a sentence, as in Bakhtin (1981) indicates that … is different to a reference in brackets following a statement, as in: (Bakhtin, 1981). Citation in images is, however, less known to us, or, at least, less theorized. Citation in images can mostly be thought of as paraphrasing, for instance, drawing a cartoon from a painting or a flow chart from a diagram, or an abstract drawing of a building. However, these terms taken from writing do not work well when looking at citation in images. We need more precise semiotic terms to describe these processes. Gunther Kress has proposed more generic terms to talk about semiotic processes, namely transformation and transduction. Transformation refers to the processes of meaning change through the re-ordering of the elements in a text, but working within the same mode, such as writing (Kress 2010, 129). Transduction names the «process of moving meaning-material from one mode to another – from speech to image; from writing to film» (Kress 2010, 129). So, if you describe an image in writing, it is not so much paraphrasing as transducting from one mode to another.

Citation in three dimensional artefacts functions slightly differently to both writing and images. In Archer (2018), I propose that citation in three dimensional artefacts comprises intertextual negotiation with authoritative conventions, such as big names, art periods, and specific genres. Citation can also be thought of as connotative provenance. Of course, the notions of intertextuality and provenance apply to all texts, including writing, but I am using them now specifically to think about voice in designed artefacts. Provenance refers to the fact that we «constantly import signs from other contexts (another era, social group, culture) into the context in which we are now making a new sign, in order to signify ideas and values which are associated with that other context» (Kress & Van Leeuwen 2001, 10). Björkvall (2018) points to how provenance is concerned with connotational meanings that require specific cultural knowledge in order to be recognized, even as these meanings may change over time. To illustrate this idea of provenance, I take an example from my university campus (see Abbildung 5).

Abbildung 5: Citation as provenance: Decolonial Struggles

This is an image of the department of architecture building taken two years ago during the student protests, which I mentioned earlier. Pieces of paper are stuck on the windows, spelling the words Decolonial Struggles. Here the relationship between writing and place is important and the temporary nature of the message is evident by the fact that some of the papers had already peeled off the windows. What is interesting was that a week or two later the same font and aesthetic (and some of the wording) was then taken into a poster for advertising a particular event, called unsettling colonialism. I would argue that this is an example of citation as provenance. The poster draws on the aesthetics, values and discourses of the protest action by replicating a version of the typography as seen on the building. Interestingly, this is very local and in-house citation, as you needed to have been there and seen the transient slogans at a particular time and place. This example highlights sharply that citation is about creating communities, with both insiders and outsiders to those communities.

See Abbildung 6 below for another example of citation as provenance. They are bangles produced from plastic water bottles. When people do upcycling they draw on different discourses and these discourses provide much of the value adding that is the key feature of upcycling. Here the material of the artefact, namely plastic, has provenance in discourses of ethical consumerism (Archer & Björkvall 2018). Something as natural as water is a scarce resource in some global contexts and has become fashionably commodified into a branded product for consumption in others. Plastic water bottles are rife and are a scourge on the environment, but here they are upcycled to make something beautiful and of value. The colour blue is foregrounded, keeping the resonances of both the plastic bottle and the water it contained. The plastic and the silver weave together like ripples in water, creating a sense of movement, transparency, and translucency. These bangles and other products upcycled from waste materials indicate alignment with and indeed cite larger discourses of ethical consumerism, given the provenance of the upcycled materials.

I have argued that citation can be thought of in terms of the semiotic terms transduction and transformation of meaning material. In addition, I have emphasized that citation is crucial to academic voice as it concerns positioning oneself in relation to others, as well as in relation to particular conventions. This can be achieved through the provenance of semiotic resources chosen and the discourses that these resources index. Tabelle 1 below summarizes the framework for academic voice across writing, image, infographics and designed artefacts. Writing, images and information graphics are often co-present in texts in Higher Education. The types of texts realized predominantly through writing, however, include academic essays and reports. Those realized predominantly through images could be powerpoint presentations, storyboards, posters. Information graphics are present in texts in economics and the social sciences and others. Designed artefacts are specific to certain disciplines. There are, of course, other aspects of academic voice besides authorial engagement and citation. For instance, modality is another aspect to consider, the perceived credibility and truth value of a text. However, for the sake of analytical clarity, this particular framework only works with two aspects.

Abbildung 6: Provenance and materiality (Plastic water bottle and sterling silver bangles. Mikhela Hawker)

Tabelle 1: A framework for looking at academic voice in multimodal texts