Hostname: page-component-669899f699-swprf Total loading time: 0 Render date: 2025-04-26T14:39:29.308Z Has data issue: false hasContentIssue false
  • English
  • Français

What counts as a multimodal metaphor and metonymy? Evolution of inter-rater reliability across rounds of annotation

Published online by Cambridge University Press:  20 July 2023

Paula Pérez Sobrino Open the ORCID record for Paula Pérez Sobrino [Opens in a new window]  and
Samantha Ford Open the ORCID record for Samantha Ford [Opens in a new window]
Paula Pérez Sobrino*
Affiliation:
Departamento de Filologías Modernas, Universidad de La Rioja, c/San José de Calasanz 33, 26004 Logroño, La Rioja, Spain
Samantha Ford
Affiliation:
Department of English Language and Linguistics, University of Birmingham, Frankland Building, Edgbaston, Birmingham B15 2TT, United Kingdom
*
Corresponding author: Paula Pérez-Sobrino; Email: paula.perezs@unirioja.es
Get access Rights & Permissions [Opens in a new window]

Abstract

An open question in research on multimodal figuration is how to mitigate the analyst’s bias in identifying and interpreting metaphor and metonymy; an issue that determines the generalizability of the findings. Little is known about the causes that motivate different annotations. Inter-rater reliability tests are useful to investigate the sources of variation in annotations by independent researchers that can help inform and refine protocols.

Inspired by existing procedures for verbal, visual, and filmic metaphor identification, we formulated instructions to identify multimodal metaphor and metonymy and tested it against a corpus of 21 generic advertisements and 21 genre-specific advertisements (mobile phones). Two independent researchers annotated the advertisements in six rounds. A joint discussion followed each round to consider conflicting annotations and refine the protocol for the ensuing round.

By examining the evolution of inter-rater reliability results, we found that (1) we reached similar levels of agreement for the identification of metaphor and metonymy, although converging on the interpretation of metonymy was more difficult; (2) some genre specificities made it easier to agree on the annotations for mobile advertisements than for the general advertisements; and (3) there was a consistent increase in the kappa scores reaching substantial agreement by the sixth round.

Keywords

advertisinginter-rater reliabilitymetaphor and metonymy identificationmultimodality
Type
Article
Information
Language and Cognition , Volume 15 , Issue 4 , December 2023 , pp. 786 - 814
Check for updates
Copyright
© The Author(s), 2023. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

References

Altman, D. G. (1990). Practical statistics for medical research: 12. Chapman & Hall/CRC.CrossRefGoogle Scholar
Armstrong, S., Davies, H., & Paulson, E. (2011). The subjectivity problem: Improving triangulation approaches in metaphor analysis studies. International Journal of Qualitative Methods, 10(2), 151–163.CrossRefGoogle Scholar
Artstein, R., & Poesio, M. (2008). Inter-coder agreement for computational linguistics. Computational Linguistics, 34(4), 555–596. https://doi.org/10.1162/coli.07-034-R2CrossRefGoogle Scholar
Bastian, H. (2016, December 5). Reproducibility crisis timeline: Milestones in tackling research reliability. Absolutely Maybe. https://absolutelymaybe.plos.org/2016/12/05/reproducibility-crisis-timeline-milestones-in-tackling-research-reliability/Google Scholar
Bobrova, L. (2015). A procedure for identifying potential multimodal metaphors in TV commercials. Multimodal Communication, 4(2), 113–131. https://doi.org/10.1515/mc-2015-0009CrossRefGoogle Scholar
Bolognesi, M. (2017). Using semantic feature norms to investigate how the visual and verbal modes afford metaphor construction and expression. Language and Cognition, 9(3), 525–552. https://doi.org/10.1017/langcog.2016.27CrossRefGoogle Scholar
Bolognesi, M., Pilgram, R., & van den Heerik, R. (2017). Reliability in content analysis: The case of semantic feature norms classification. Behavior Research Methods, 49(6), 1984–2001. https://doi.org/10.3758/s13428-016-0838-6CrossRefGoogle ScholarPubMed
Bort-Mir, L. (2019). Developing, applying, and testing FILMIP: The Filmic Metaphor Identification Procedure [Universitat Jaume I]. https://doi.org/10.13140/RG.2.2.18345.03688CrossRefGoogle Scholar
Caballero, R. (2016). Genre and metaphor: Use and variation across usage events. In Semino, E. & Demjén, Z. (Eds.), The Routledge handbook of metaphor and language (1st ed). Routledge. http://ebookcentral.proquest.com/lib/bham/detail.action?docID=4732494Google Scholar
Carletta, J. (1996). Assessing agreement on classification tasks: The kappa statistic. Computational Linguistics 22(201): 249–254Google Scholar
Cienki, A., & Müller, C. (2008). Metaphor and gesture. John Benjamins Publishing.CrossRefGoogle Scholar
Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1), 37–46. https://doi.org/10.1177/001316446002000104CrossRefGoogle Scholar
Davies, H., Watson, B., & Bakerson, M. (2015). Crowdchecking conceptual metaphors: How principles and teachers frame the principal’s role in academics through metaphor. In Wan, W. & Low, G. (Eds.), Elicited metaphor analysis in educational discourse. John Benjamins Publishing Company.Google Scholar
Davies, M. (2004). British national corpus (Version 2007) [Computer software]. Oxford University Press. https://www.english-corpora.org/bnc/Google Scholar
Davies, M. (2008). The Corpus of Contemporary American English (COCA). https://www.english-corpora.org/coca/Google Scholar
Declercq, J., & Van Poppel, L. (2023). Coding metaphors in interaction: A study protocol and reflection on validity and reliability challenges. International Journal of Qualitative Methods, 22, 1–9.CrossRefGoogle Scholar
Díez Velasco, O. I., & Ruiz de Mendoza Ibáñez, F. J. (2002). Patterns of conceptual interaction. In Dirven, R. & Pörings, R. (Eds.), Metaphor and metonymy in comparison and contrast. Walter de Gruyter.Google Scholar
Dorst, A. G. (2011). Personification in discourse: Linguistic forms, conceptual structures and communicative functions. Language and Literature: International Journal of Stylistics, 20(2), 113–135. https://doi.org/10.1177/0963947010395522CrossRefGoogle Scholar
Dunn, J. (2015). Modeling abstractness and metaphoricity. Metaphor and Symbol, 30(4), 259–289. https://doi.org/10.1080/10926488.2015.1074801CrossRefGoogle Scholar
El Refaie, E. (2003). Understanding visual metaphor: The example of newspaper cartoons. Visual Communication, 2(1), 75–95. https://doi.org/10.1177/1470357203002001755CrossRefGoogle Scholar
Forceville, C. (1996). Pictorial metaphor in advertising. Routledge.CrossRefGoogle Scholar
Forceville, C. (2009a). Metonymy in visual and audiovisual discourse. In Ventola, E. & Guijarro, A. (Eds.), The world told and the world shown: Multisemiotic issues. Palgrave Macmillan.Google Scholar
Forceville, C. (2009b). Non-verbal and multimodal metaphor in a cognitivist framework: Agendas for research. In Forceville, C. J. & Urios-Aparisi, E. (Eds.), Multimodal metaphor. Walter de Gruyter.CrossRefGoogle Scholar
Forceville, C., & Urios-Aparisi, E. (Eds.) (2009). Multimodal metaphor. Mouton de Gruyter. https://doi.org/10.1515/9783110215366CrossRefGoogle Scholar
Ford, S. (2017). Multimodal metaphor and metonymy and the figurative complexity of mobile phone advertising: A comparison between mobile phone manufacturer and network provider advertisements [Bachelor of Arts, University of Birmingham]. https://www.researchgate.net/publication/361909308_Multimodal_metaphor_and_metonymy_and_the_figurative_complexity_of_mobile_phone_advertising_A_comparison_between_mobile_phone_manufacturer_and_network_provider_advertisementsGoogle Scholar
Gkiouzepas, L., & Hogg, M. K. (2011). Articulating a new framework for visual metaphors in advertising. Journal of Advertising, 40(1), 103–120. https://doi.org/10.2753/JOA0091-3367400107CrossRefGoogle Scholar
Goossens, L. (1990). Metaphtonymy: The interaction of metaphor and metonymy in expressions for linguistic action. In Dirven, R. & Pörings, R. (Eds.), Metaphtonymy: The interaction of metaphor and metonymy in expressions for linguistic action (pp. 349–378). De Gruyter Mouton. https://doi.org/10.1515/9783110219197.3.349Google Scholar
Grady, J. (1997). Foundations of meaning: Primary metaphor and primary scenes. A dissertation submitted in partial satisfaction of the requirements for the degree of Doctor of Philosophy in Linguistics in the graduate division of the University of California, Berkeley. https://escholarship.org/content/qt3g9427m2/qt3g9427m2.pdfGoogle Scholar
Hanks, P. (2006). Metaphoricity is gradable. In Stefanowitsch, A. & Gries, S. T. (Eds.), Corpus-based approaches to metaphor and metonymy (pp. 17–35). Mouton de Gruyter. https://doi.org/10.1515/9783110199895.1CrossRefGoogle Scholar
Hidalgo-Downing, L., & Mujic, B. K. (2020). Performing metaphoric creativity across modes and contexts. John Benjamins Publishing Company. https://doi.org/10.48550/arXiv.cmp-lg/9602004CrossRefGoogle Scholar
Hyndman, S. (2016). Why fonts matter. Gingko Press.Google Scholar
Jiang, J., Lupoiu, R., Wang, E. W., Sell, D., Paul Hugonin, J., Lalanne, P., & Fan, J. A. (2020). MetaNet: A new paradigm for data sharing in photonics research. Optics Express, 28(9), 13670. https://doi.org/10.1364/OE.388378CrossRefGoogle ScholarPubMed
Jonauskaite, D., Parraga, C. A., Quiblier, M., & Mohr, C. (2020). Feeling blue or seeing red? Similar patterns of emotion associations with colour patches and colour terms. i-Perception, 11(1), 2041669520902484. https://doi.org/10.1177/2041669520902484CrossRefGoogle ScholarPubMed
Krippendorff, K. (2013). Content analysis: An introduction to its methodology. SAGE.Google Scholar
Lakoff, G. & Johnson, M. (2003). Metaphors we live by (2nd edn). University of Chicago Press.CrossRefGoogle Scholar
Levinson, S. C., & Majid, A. (2014). Differential ineffability and the senses. Mind & Language, 29(4), 407–427. https://doi.org/10.1111/mila.12057CrossRefGoogle Scholar
Lievers, F. S. (2017). Figures and the senses: Towards a definition of synaesthesia. Review of Cognitive Linguistics. Published under the Auspices of the Spanish Cognitive Linguistics Association, 15(1), 83–101. https://doi.org/10.1075/rcl.15.1.04strCrossRefGoogle Scholar
Littlemore, J. (2015). Metonymy: Hidden shortcuts in language, thought and communication (First paperback edition). Cambridge University Press.Google Scholar
Littlemore, J., & Tagg, C. (2018). Metonymy and text messaging: A framework for understanding creative uses of metonymy. Applied Linguistics, 39(4), 481–507. https://doi.org/10.1093/applin/amw018Google Scholar
Maslen, R. (2016). Finding systematic metaphors. In Semino, E. & Demjén, Z. (Eds.), The Routledge handbook of metaphor and language. (pp. 106–119). Routledge.Google Scholar
Müller, C. (2008). Metaphors dead and alive, sleeping and waking: A dynamic view. University of Chicago Press.CrossRefGoogle Scholar
Musolff, A. (2006). Metaphor Scenarios in Public Discourse. Metaphor and Symbol, 21(1), 23–38. https://doi.org/10.1207/s15327868ms2101_2Google Scholar
Pan, M. X., & Tay, D. (2020). Identifying creative metaphor in video ads. In LIN, K. L., Mwinlaaru, I. N., & Tay, D. (Eds.), Approaches to specialized genres: In memory of Stephen Evans (pp. 216–240). Routledge.CrossRefGoogle Scholar
Pérez-Sobrino, P. (2017). Multimodal metaphor and metonymy in advertising. John Benjamins Publishing Company. https://ebookcentral.proquest.com/lib/bham/detail.action?docID=5155788CrossRefGoogle Scholar
Pérez-Sobrino, P., Littlemore, J., & Ford, S. (2021). Unpacking creativity: The power of figurative communication in advertising. Cambridge University Press.Google Scholar
Pérez-Sobrino, P., Littlemore, J., & Houghton, D. (2019). The role of figurative complexity in the comprehension and appreciation of advertisements. Applied Linguistics, 40(6), 957–991. https://doi.org/10.1093/applin/amy039CrossRefGoogle Scholar
Pragglejaz Group. (2007). MIP: A Method for Identifying Metaphorically Used Words in Discourse. Metaphor and Symbol, 22(1), 1–39.CrossRefGoogle Scholar
Ruiz de Mendoza, F. (2000). The role of mappings and domains in understanding metonymy. In Barcelona, A. (Ed.), Metaphor and metonymy at the crossroads (pp. 109–132). Mouton de Gruyter.Google Scholar
Šorm, E., & Steen, G. J. (2018). VISMIP: Towards a method for visual metaphor identification. In Steen, G. J. (Ed.), Visual metaphor: Structure and process (p. 41). John Benjamins Publishing Company.Google Scholar
Spence, C. (2012). Managing sensory expectations concerning products and brands: Capitalizing on the potential of sound and shape symbolism. Journal of Consumer Psychology, 22(1), 37–54. https://doi.org/10.1016/j.jcps.2011.09.004Google Scholar
Stampoulidis, G., & Bolognesi, M. (2019). Bringing metaphors back to the streets: A corpus-based study for the identification and interpretation of rhetorical figures in street art. Visual Communication, 22, 1470357219877538. https://doi.org/10.1177/1470357219877538Google Scholar
Stampoulidis, G., Bolognesi, M., & Zlatev, J. (2019). A cognitive semiotic exploration of metaphors in Greek street art. Cognitive Semiotics, 12(1), 20192008. https://doi.org/10.1515/cogsem-2019-2008CrossRefGoogle Scholar
Steen, G. J., Dorst, A. G., Herrmann, J. B., Kaal, A., Krennmayr, T., & Pasma, T. (2010). A method for linguistic metaphor identification: From MIP to MIPVU (Vol. 14). John Benjamins Publishing.CrossRefGoogle Scholar
Velasco-Sacristan, M. O., & Fuertes-Olivera, P. (2006). Olfactory and olfactory-mixed metaphors in print ads of perfume. Annual Review of Cognitive Linguistics, 4, 217–252.CrossRefGoogle Scholar
Yu, N., Yu, L., & Lee, Y. C. (2017). Primary metaphors: Importance as size and weight in a comparative perspective. Metaphor and Symbol, 32(4), 231–249. https://doi.org/10.1080/10926488.2017.1384276CrossRefGoogle Scholar
Zbikowski, L. (2009). Music, language, and multimodal metaphor (pp. 359–381). Multimodal Metaphor.CrossRefGoogle Scholar

Secondary references

Figure 1. Advertisement for Cat mobile phone (Text: “Rugged. Resilient. Reliable. One seriously hard worker.”). Provided courtesy of Caterpillar. ©Caterpillar Inc. All rights reserved. Retrieved on 9th February 2017, original source (last accessed: 10th May 2023): https://www.conormasterson.com/portfolio/category/advertising/caterpillar-mobile-phone-welder.Google Scholar
Figure 2. Advertisement for giffgaff mobile phone network (Text: “*fist bump*@LayolaLotus. We don’t like contracts. But we do like you.”). Illustrator: Serge Seidlitz. Retrieved on 9th February 2017, original source: www.newsworks.org.uk.Google Scholar
Figure 3. Advertisement for buyresponsibly.org (Text: “What’s behind the things we buy? Buyresponsibly.org”). Source (last accessed: 10th May 2023): https://www.adsoftheworld.com/campaigns/shopping-cart-4a93aea6-6fb6-47e6-a42e-6771e2baeb11.Google Scholar
Figure 4. Advertisement for Fender guitar. The Fender logo and headstock are registered trademarks of FMIC. Source (last accessed: 10th May 2023): https://www.adsoftheworld.com/campaigns/exit-b0d175d4-bc81-458d-ba7b-fbe42fff0639.Google Scholar
Figure 5. Advertisement for Peugeot. Source (last accessed: 10th May 2023): https://www.adsoftheworld.com/campaigns/drives-any-business.Google Scholar
Figure 6. Advertisement for MAX shoes (Text: “You are what you wear”). Source (last accessed: 10th May 2023): https://www.adsoftheworld.com/campaigns/face-c32976c7-fda4-440e-b0ec-d4fb1ceeea23.Google Scholar
Figure 7. Advertisement for Lumia phone (Text: “Everyone loves a comeback”). Retrieved on 9th February 2017, original source: www.newsworks.org.uk.Google Scholar
Figure 12: Advertisement for EE mobile phone network (Text: “More beats per minute. Download albums superfast with mobile #4GEE. Only on EE.”). Retrieved on 9th February 2017, original source: www.newsworks.org.uk.Google Scholar
Figure 13: Advertisement for Blackberry phone (Text: “Work narrow. Work wide. See the bigger picture.”). ©2014 BlackBerry Limited. BlackBerry Passport® smartphone advertisement is used with permission from BlackBerryLimited. All Rights Reserved. Retrieved on 9th February 2017, original source: news.phonesltd.co.uk, source (last accessed: 10th May 2023): https://blogs.blackberry.com/en/2014/09/meet-blackberry-passport.Google Scholar
Figure 15: Advertisement for O2 network (Text: “Surprises of all sized every time you top up”). Retrieved on 9th February 2017, original source: www.theinspirationroom.com, source (last accessed: 10th May 2023): https://www.pinterest.com.mx/pin/270849365064206178/.Google Scholar