Interactive Digital Displays

Abstract

Drawing from the self-presence theory and engagement, we specifically investigate the gesture interactivity and self-mirroring effect of large digital displays, also known as digital signage. By extending the interactive effects impact of gesture and self-mirroring from the immersive environment to a large interactive display, this study found that the implicit mediating roles of social presence and engagement for advertising effectiveness through a 2 (gesture; non-gesture vs. gesture interactivity) x 2 (self-representation; non-self-mirroring vs. self-mirroring interactivity) between-subject design (N = 81) experiment. The results indicated that gesture interactivity enhances a feeling of self-presence, not the self-mirroring, and it positively fosters psychological engagement with advertising content. In turn, the heightened engagement results in better advertising effectiveness e.g., brand attitude, purchase intention, and recall. 

Keywords: Interactive Digital Screens; Gesture Interactivity; Self-Mirrored; Self-Presence; Engagement; Advertising.

Introduction

Interactive digital displays in the public space, also known as digital signage for advertisement, have evolved along with technology advancement (Davies, Clinch, and Alt 2014) consisting of various forms (i.e., poster, windows, mirror, overlay) with interactive features (i.e., interaction modalities; gestures, body position, posture, gaze, speech, keys, and touch) (Müller et al., 2010). Initially, the large public displays have a similar premise to commercial billboards and posters, which containcontaining little to no content other than advertising. As a result, one of the challenges in developing the content of the public digital displays is not only to appeal to viewers but also to cater to the demand of other stakeholders that could be a persuasive message deliverance (Alt, Müller, et al., 2012). Catering to the demands of the viewers and stakeholders, the implicit persuasive processes in digital interactive displays need to be investigated so that the designers and developers can create more persuasive content.

Despite a plethora of research on the advertising effectiveness of digital public display with interactive features (Alt et al., 2012; Davies et al., 2014; Fischer & Hornecker, 2012; Michelis & Müller, 2011; Sahibzada et al., 2017), few studies have focused on more implicit persuasive processes with interactive features in public digital display for advertisements. It is known that the role of self-presence in gesture interactivity and mirrored-self-image is a mediator for psychological engagement in general human-computer interaction research (Adachi et al., 2020; Borum et al., 2013; Jin & Park, 2009). Also, the psychological engagement, which is induced by immersive digital media, mediates the user satisfaction and message effect in the context of the virtual environment (Calder & Malthouse, 2008; Chen & Wang, 2019; Han & Cho, 2019; Kim et al., 2017; Singh & Pandey, 2014).

For these reasons, drawing from the self-presence theory and psychological engagement for advertising effectiveness, we investigated whether gesture and self-mirroring interactivity for advertising in the context of large display contribute to enhancing users’ sense of self-presence. Moreover,and whether this influences users’ psychological engagement with advertising content, in turn, it results in users’ satisfaction, purchase intention, and memory. 

According to Müller et al.’s digital display taxonomy (2010), the gesture interaction modality (e.g., Pendle, Magical Mirrors, Diaper Selector, Traveling TicTac-Toe, Interactive Ambient Public Display), and the mirror as a display type (e.g., Palimpsest, Video place, Vision Kiosk, and eMir) are relatively applied features for large displays. These applied features would be the reason for having mid-air gestures and self-mirroring of large displays as the main features for the current study. Moreover, Michelis and Müller (2011) observed how the mirroring and gesture interaction on the large displays were used by examining Magical Mirrors. By reviewing ad-hoc cases, the Magical Mirrors have consisted of the mirrored image of the environment. The environment, including users and gesture-based interaction as the main variables to observe, for example, “... glancing at a first display while passing it, moving the arms to cause some effects then directly approaching one of the following displays and positioning oneself in the center of a display (p. 562).”  To sum up, the dominantly applied features in public displays for advertising are gesture (mid-air gesture) interactivity and self-mirrored features, which will be the primary technological domains of the current study. Also, while several interactive techniques have been suggested and built in an attempt to enhance immersion and memory effect (Heeter 2000; Lombard and Snyder-Duch 2001; Rodgers and Thorson 2000)., tThe impact of adding these interactivities for the digital interactive large displays and the persuasive communication purpose is relatively little known.

The purpose of this study is to investigate whether similar advertising effectiveness can be observed by extending the interactive effect of an immersive environment to a large interactive display. To find out whether gesture interactivity and self-mirroring enhance or hinder advertising effectiveness, which includes a memory of product, brand attitude, and purchasing intention., Tthis study aims to examine the underlying theoretical mechanisms of the effects of interactivity in relation toconcerning the sense of self-presence and engagement as the key mediator of its advertising effect in the context of a large interactive display. 

Interactive Advertisement: Self-presence and Engagement

Interactivity refers to users manipulating the content or form of the mediated experience (Heeter 2000). Heeter (2000) understands interactivity as “an episode or series of episodes of physical actions and reactions of an embodied human with the world, including the environment to create affordances for a human participant” (p. 7). The features of interactivity are kno