Assessing the impact of reading in online book reviews
Peter Boot | Huygens Institute for the History of the Netherlands
Marijn Koolen | Humanities Cluster, Royal Netherlands Academy of Arts and Sciences
What is the impact of reading fiction? We analyze online Dutch book reviews to detect overall affective impact, narrative
feelings, response to style and reflection. We create a set of rules that analyze the reviews and detect the impact aspects. We evaluate the
detection by asking raters about the presence of these aspects in reviews and comparing these ratings to our detection. Interrater
agreements are weak to moderate; however, there is a significant correlation between the model’s predictions for all impact aspects except
reflection. The detected impact correlates with book genres in the way one would expect: Narrative feelings are highest for thrillers, and
stylistic response is highest for literary books. We can thus estimate some aspects of the response books evoke in readers. Initial results
suggest that the appreciation of style is linked to reflection in the reader. However, the concepts underlying the impact categories need
further exploration.
Albrechtslund, A. -M. B. (2019). Amazon, Kindle, and Goodreads: implications for literary consumption in the digital age. Consumption Markets & Culture, 1–16.
Appel, M., Koch, E., Schreier, M., & Groeben, N. (2002). Aspekte des Leseerlebens: Skalenentwicklung. Zeitschrift für Medienpsychologie, 141, 149–154.
Bachmann-Stein, A. (2015). Zur Praxis des Bewertens in Laienrezensionen. In H. Kaulen & C. Gansel (Eds.), Literaturkritik heute. Tendenzen–Traditionen–Vermittlung (pp. 77–91). Göttingen: V&R Unipress.
Berthoud, E., & Elderkin, S. (2013). The Novel Cure: An A to Z of Literary Remedies. Edinburgh, London: Canongate Books.
Boot, P. (2011). Towards a Genre Analysis of Online Book Discussion: socializing, participation and publication in the Dutch booksphere. Selected Papers of Internet Research, IR 12.0. [URL]
Boot, P. (2012). Contextual factors in literary quality judgments: A quantitative analysis of an online writing community. Paper presented at Digital Humanities 2012. [URL]
Boot, P. (2017). A Database of Online Book Response and the Nature of the Literary Thriller. Paper presented at Digital Humanities 2017. [URL]
boyd, d., & Ellison, N. B. (2008). Social network sites: Definition, history, and scholarship. Journal of computer-mediated Communication, 13(1), 210–230.
Busselle, R., & Bilandzic, H. (2009). Measuring narrative engagement. Media Psychology, 12(4), 321–347.
Chevalier, J. A., & Mayzlin, D. (2006). The effect of word of mouth on sales: Online book reviews. Journal of marketing research, 43(3), 345–354.
Dal Cin, S., Zanna, M. P., & Fong, G. T. (2004). Narrative persuasion and overcoming resistance. In E. S. Knowles & J. A. Linn (Eds.), Resistance and persuasion (pp. 175–191). Mahwah (NJ) & London: Lawrence Erlbaum Associates.
Driscoll, B., & Rehberg Sedo, D. (2019). Faraway, So Close: Seeing the Intimacy in Goodreads Reviews. Qualitative Inquiry, 1077800418801375.
Finn, E. F. (2011). The Social Lives of Books: Literary Networks in Contemporary American Fiction. (PhD), Stanford University. Retrieved from [URL]
Gerrig, R. J., & Rapp, D. N. (2004). Psychological processes underlying literary impact. Poetics Today, 25(2), 265–281.
Goffman, E. (1956). The presentation of self in everyday life. Edinburgh: University of Edinburgh. Social Sciences Research Centre.
Gruzd, A., & Rehberg Sedo, D. N. (2012). #1b1t: Investigating Reading Practices at the Turn of the Twenty-first Century. Mémoires du livre, 3(2). [URL]
Gutjahr, P. C. (2002). No Longer Left Behind: Amazon.com, Reader-Response, and the Changing Fortunes of the Christian Novel in America. Book History, 51, 209–236.
Hajibayova, L. (2019). Investigation of Goodreads’ reviews: Kakutanied, deceived or simply honest?Journal of Documentation, 75(3), 612–626.
Hosoya, G., Schindler, I., Beermann, U., Wagner, V., Menninghaus, W., Eid, M., et al.. (2017). Mapping the conceptual domain of aesthetic emotion terms: A pile-sort study. Psychology of Aesthetics, Creativity, and the Arts, 11(4), 457.
Jaakkola, M. (2019). From re-viewers to me-viewers: The #Bookstagram review sphere on Instagram and the uses of the perceived platform and genre affordances. Interactions: Studies in Communication & Culture, 10(1–2), 91–110.
Jamieson, S. (2004). Likert scales: How to (ab) use them. Medical education, 38(12), 1217–1218.
Keen, S. (2007). Empathy and the Novel. Oxford: Oxford University Press.
Kidd, D. C., & Castano, E. (2013). Reading literary fiction improves theory of mind. Science, 342(6156), 377–380.
Knobloch-Westerwick, S., & Keplinger, C. (2006). Mystery appeal: Effects of uncertainty and resolution on the enjoyment of mystery. Media Psychology, 8(3), 193–212.
Knoop, C. A., Wagner, V., Jacobsen, T., & Menninghaus, W. (2016). Mapping the aesthetic space of literature “from below”. Poetics, 561, 35–49.
Koopman, E. (2016). Effects of “Literariness” on Emotions and on Empathy and Reflection After Reading. Psychology of Aesthetics, Creativity, and the Arts, 10(1), 82–98.
Koopman, E. M. E., & Hakemulder, F. (2015). Effects of literature on empathy and self-reflection: A theoretical-empirical framework. Journal of Literary Theory, 9(1), 79–111.
Kuijpers, M. M. (2014). Absorbing stories. The effects of textual devices on absorption and evaluative responses. (Ph. D), Utrecht, Utrecht.
LeBreton, J. M., & Senter, J. L. (2008). Answers to 20 questions about interrater reliability and interrater agreement. Organizational research methods, 11(4), 815–852.
Lindell, M. K., and Brandt, C. J. (1997). Measuring interrater agreement for ratings of a single target. Appl. Psychol. Meas. 211, 271–278.
Martens, M. (2016). Publishers, Readers, and Digital Engagement. London: Palgrave McMillan.
McDonald, R. (2007). The death of the critic. London, New York: Continuum International Publishing Group.
Mehling, G., Kellermann, A., Kellermann, H., & Rehfeldt, M. (2018). Leserrezensionen auf amazon.de: Eine teilautomatisierte inhaltsanalytische Studie. Bamberg: University of Bamberg Press.
Miall, D. S., & Kuiken, D. (2002). A feeling for fiction: Becoming what we behold. Poetics, 30(4), 221–241.
Murray, S. (2015). Charting the Digital Literary Sphere. Contemporary Literature, 56(2), 311–339.
Murray, S. (2018). Reading Online: Updating the State of the Discipline. Book History, 21(1), 370–396.
Naik, Y. (2012). Finding good reads on Goodreads. Reference & User Services Quarterly, 51(4), 319–323.
Naper, C. (2016). Experiencing the Social Melodrama in the Twenty-first Century. In P. M. Rothbauer, K. I. Skjerdingstad, L. E. McKechnie, & K. Oterholm (Eds.), Plotting the Reading Experience: Theory/Practice/Politics. Waterloo (Ontario): Wilfrid Laurier University Press.
Norman, G. (2010). Likert scales, levels of measurement and the “laws” of statistics. Advances in health sciences education, 15(5), 625–632.
Oatley, K. (1994). A taxonomy of the emotions of literary response and a theory of identification in fictional narrative. Poetics, 23(1–2), 53–74.
O’Neill, T. A. (2017). An overview of interrater agreement on Likert scales for researchers and practitioners. Frontiers in psychology, 81, 777.
Ott, M., Cardie, C., & Hancock, J. (2012). Estimating the prevalence of deception in online review communities. Paper presented at the Proceedings of the 21st international conference on World Wide Web, Lyon, France. [URL].
Papacharissi, Z. (2011). Conclusion: A networked self. In Z. Papacharissi (Ed.), A networked self: Identity, community, and culture on social network sites (pp. 304–318). New York: Routledge.
Pianzola, F., Rebora, S., & Lauer, G. (2020). Wattpad as a resource for literary studies. Quantitative and qualitative examples of the importance of digital social reading and readers’ comments in the margins. PloS one, 15(1), e0226708.
Rebora, S., Lendvai, P., & Kuijpers, M. (2018). Reader experience labeling automatized: Text similarity classification of user-generated book reviews. Paper presented at the EADH 2019, Galway. [URL]
Rehfeldt, M. (2017a). „Ganz große, poetische Literatur–Lesebefehl!“ Unterschiede und Gemeinsamkeiten von Amazon-Rezensionen zu U-und E-Literatur Lesen X. 0. Rezeptionsprozesse in der digitalen Gegenwart. Göttingen: V&R unipress.
Rehfeldt, M. (2017b). Leserrezensionen als Rezeptionsdokumente. Zum Nutzen nicht-professioneller Literaturkritiken für die Literaturwissenschaft. In A. Bartl & M. Behmer (Eds.), Die Rezension. Aktuelle Tendenzen der Literaturkritik (pp. 275–289). Würzburg: Köningshausen und Neumann.
Ridenour, L., & Jeong, W. (2016). Leveraging the power of social reading and big data: an analysis of co-read clusters of books on Goodreads. IConference 2016 Proceedings. [URL].
Ross, C. S. (1999). Finding without seeking: the information encounter in the context of reading for pleasure. Information Processing & Management, 35(6), 783–799.
Sabine, G., & Sabine, P. (1983). Books That Made the Difference: What People Told Us. The Shoe String Press, Inc.
Sairio, A. (2014). ‘No other reviews, no purchase, no wish list’: Book reviews and community norms on Amazon.com. Studies in Variation, Contacts and Change in English, 151. Retrieved from [URL]
Saricks, J. G. (2005). Articulating a Book’s Appeal. Readers’ advisory service in the public library. Third Edition (pp. 40–73). Chicago: American Library Association.
Schindler, I., Hosoya, G., Menninghaus, W., Beermann, U., Wagner, V., Eid, M., et al.. (2017). Measuring aesthetic emotions: A review of the literature and a new assessment tool. PloS one, 12(6), e0178899.
Smith, D. (2004). Amazon reviewers brought to book. Guardian. Retrieved from [URL]
Spiteri, L. F., & Pecoskie, J. (2016). Affective taxonomies of the reading experience: Using user-generated reviews for readers’ advisory. Proceedings of the Association for Information Science and Technology, 53(1), 1–9.
Stein, S. (2015). Laienliteraturkritik–Charakteristika und Funktionen von Laienrezensionen im Literaturbetrieb. In H. Kaulen & C. Gansel (Eds.), Literaturkritik Heute (pp. 59–76). Göttingen: V&R Unipress.
Steiner, A. (2010). Personal Readings and Public Texts: Book Blogs and Online Writing about Literature. Culture unbound, 2(28), 471–494.
Streitfeld, D. (2012, 2012–08–25). The Best Book Reviews Money Can Buy. New York Times. Retrieved from [URL]
Taboada, M. (2011). Stages in an online review genre. Text & Talk-An Interdisciplinary Journal of Language, Discourse & Communication Studies, 31(2), 247–269.
Thelwall, M., & Kousha, K. (2016). Goodreads: A social network site for book readers. Journal of the Association for Information Science and Technology, 68(4), 972–983.
Thelwall, M. (2019). Reader and author gender and genre in Goodreads. Journal of Librarianship and Information Science, 51(2), 403–430.
Tóth, M., & Audunson, R. (2012). Websites for booklovers as meeting places. Library Hi Tech, 30(4), 655–672.
Van Noord, G. (2006). At last parsing is now operational. Paper presented at the TALN06. Verbum Ex Machina. Actes de la 13e conference sur le traitement automatique des langues naturelles. [URL]
Van Putten-Brons, S., & Boot, P. (2017). June is Dutch Literature Month!: Online Book Reviewers and Their Role in the Transmission of Dutch Literature to the English-Speaking World. In E. Brems, O. Réthelyi & T. van Kalmthout (Eds.), Doing Double Dutch (pp. 313–327). Leuven: Leuven University Press.
Vásquez, C. (2014). ‘Usually not one to complain but…’: Constructing identities in user-generated online reviews. The language of social media (pp. 65–90). London: Palgrave Macmillan.
Wallace, L. K. (2016). “My History, Finally Invented”: Nightwood and Its Publics. QED: A Journal in GLBTQ Worldmaking, 3(3), 71–94.
Worrall, A. (2019). “Connections above and beyond”: Information, translation, and community boundaries in Library Thing and Goodreads. Journal of the Association for Information Science and Technology, 70(7), 742–753.