1. Introduction
The music listening experience has undergone a remarkable transformation due to technological advancements, transitioning from traditional formats like CDs and MP3 players to interactive and immersive environments. With the rise of smartphones and advanced wireless technologies, music access has evolved through online streaming services, redefining how we enjoy music. However, this shift brings forth various user experience challenges. While online streaming offers numerous conveniences, users often face common frustrations—one of the most significant being the complexity of user interfaces. Many streaming platforms feature complicated or cluttered interfaces that hinder users’ ability to navigate effectively, discover new music, or quickly locate their favorite tracks. Inadequate search functionality and poor categorization further contribute to user dissatisfaction.
This paper investigates the intersection of auditory perception and tangible interaction within music player experiences, focusing on how physical engagement with music players can enhance the listening experience. The integration of tangible interfaces—such as music control devices, touch-sensitive surfaces, and gesture-based controls—enables users to engage with music in ways that extend beyond passive listening. These interactions can profoundly affect emotional responses, cognitive engagement, and overall user satisfaction.
Existing research emphasizes that physical manipulation can deepen listeners’ connection to music, fostering a more personalized experience. By exploring the relationship between tangible interactions and music listening, this paper aims to identify essential design principles for music players that optimize user engagement. Through an analysis of current technologies, case studies, and empirical findings, we seek to provide insights into how tangible interfaces can transform music listening into an interactive journey, ultimately enriching the emotional and social dimensions of music consumption in the digital age.
1.1. Tangible interfaces in controlling music playback
The evolution of music playback has been significantly influenced by technological advancements, leading to the development of innovative interfaces that enhance user interaction. Tangible interfaces, which allow users to manipulate physical objects to control digital music, have gained attention for their potential to provide engaging and intuitive experiences. This literature review explores the various designs, applications, and implications of tangible interfaces in music playback, highlighting their impact on user engagement, emotional connection, and social collaboration.
Tangible interfaces in music playback encompass a variety of interaction methods, including knobs, sliders, and physical objects that alter audio parameters. Johnson et al. (Reference Johnson2011) describe how these interfaces leverage physical manipulation, allowing users to engage in a more hands-on manner compared to traditional touchscreens. Studies suggest that such interactions enhance engagement, as they provide immediate tactile feedback and a sense of agency (Hornecker & Buur, 2006).
The literature indicates that tangible interfaces can significantly enhance user engagement with music. For instance, Leite et al. (2016) found that users reported higher levels of enjoyment and satisfaction when interacting with tangible controls. This engagement is attributed to users’ ability to conduct playful and exploratory movements while controlling playback, fostering a deeper connection to the music itself (Reference Blythe, Monk and OverbeekeBlythe et al., 2008). Researchers, such as Holmquist et al. (Reference Holmquist2005), emphasize that these interactive elements allow for a more immersive listening experience, bridging the gap between physical and digital realms.
Tangible interfaces also play an essential role in facilitating emotional connections to music. Studies by Tijs et al. (2015) demonstrate that users experience heightened emotional responses when controlling playback through tangible means, as the physical interaction creates a more sensory-rich experience. The embodiment of control enables users to resonate with the music on a personal level, contributing to a more profound emotional engagement. Tangible interfaces are particularly effective in promoting social interaction and collaborative music experiences. Brands et al. (Reference Brands and Farnell2017) describe how shared tangible interfaces encourage group engagement, allowing multiple users to manipulate the same controls simultaneously. This collaborative aspect not only enhances the social experience but also fosters communication and interaction among users, making music playback a communal activity (Reference Farnell, Hornecker and LeiteFarnell et al., 2016). Such environments have been identified as beneficial for educational contexts, where group learning and creativity can flourish (Reference KatzKatz et al., 2018).
The effectiveness of tangible interfaces in controlling music playback is also contingent upon thoughtful design. Key principles identified in the literature include usability, aesthetic appeal, and the provision of clear feedback (Wagner & Hornecker, 2016). However, challenges exist, such as ensuring intuitive interaction and addressing the potential for over-complication in interface design (Reference KjeldskovKjeldskov et al., 2015). Designers must balance the physical characteristics of tangible controls with usability to create meaningful user experiences. Recent advancements in technology have further enriched the possibilities of tangible interfaces in music playback. The integration of sensors, mobile devices, and wireless connectivity allows for dynamic interactions, enabling users to engage with music in new and innovative ways (Ranjan et al., 2020). These technological advancements have expanded the applicability of tangible interfaces, making them suitable for a wide range of environments and user contexts.
In summary, the literature demonstrates that tangible interfaces significantly enhance the experience of controlling music playback by promoting user engagement, fostering emotional connections, and facilitating social collaboration. While design principles and technological integration present opportunities for innovation, ongoing challenges must be addressed to optimize user experience. Future research should explore long-term user interactions, preferences, and the impact of emerging technologies on tangible interfaces in music playback, solidifying their role in the evolving landscape of music consumption.
1.2. Aesthetic interaction as music player experience
To measure the influence of differing music player experience we turn towards the literature on aesthetic interaction. The aesthetic experience accompanies both cognitive and emotional processes; the experience consists of aesthetic judgment and emotions (Kant, 1790). Aesthetic experience is not just about pleasure from visual perception of an object. Desmet (2007) defined aesthetic experience that indicates a product’s features to satisfy our sensory modalities. Looking into definitions of aesthetic experience in more detail, the object of beauty is not seen as a tool for satisfaction of bodily needs, but rather pleasures of the mind as a provocation of higher-level pleasures (Marković, 2012). Therefore, aesthetic experience is considered beyond “looks good” interaction between user and artifact to the extent “feels good” (Hashim, Noor, & Adnan, 2009). Nonetheless, the focus of the design process is mostly on the aesthetics of appearance and on the creation of artifacts that are attractive and pleasurable from that perspective (Locher, Overbeeke, & Wensveen, 2010).
As a result, and as an unusual event, the music player experience has the potential for creating unique aesthetic experiences in the sense that it does not frequently happen. Based on Rampino’s (2011) definition of Aesthetic - an added value to an artifact, this study adopts the concept of ‘aesthetic’ for application in packaging design to add value in unboxing experiences through aesthetic interaction.
However, the use of aesthetic interaction as means to explore the music player experience is complicated by the vagueness of the term. ‘Aesthetic’ is often used to describe a product’s visual appearance. As an attempt to distinguish visual aspects from aesthetic interaction. Locher (2010) defined aesthetic interaction as the aesthetics of interactive systems, which implies that aesthetics is tightly connected to context, use and instrumentality. In addition, the term “interaction aesthetics” refers to “the qualities of a design that lead to the feelings, emotions, and the behaviors that result from these more bodily types of interactions” (Eden, 2010). In this sense, the aesthetics of interaction do not have tangible properties because the definition emphasizes that the aesthetics is not intrinsic to the artifact itself, but to the way people experience it (Reference Lim, Stolterman, Jung and DonaldsonLim et al., 2007). In this sense, the nature of the aesthetic interaction emerges from a relationship between the user (i.e. their past experiences, expectations and cultural frame of reference) and the product (its characteristics, forms and materials for example). In the current study, we focus upon differences in the unboxing experience and implications for the nature of the aesthetic interaction. At the same time limiting the influence of the user through a repeated measures design.
Moreover, the focus on human-product interaction has shifted from user’s behavior and cognition to user’s affective experience (Desmet & Hekkert, 2007). Emotional functions of the product have emerged as an important factor for designers to consider beyond fulfilling functional requirement. In addition, although design for positive emotion is the main goal of designing these days, it is not sufficient for designers only to rely on their intuition and personal sensitivities. Instead, they need to understand what emotions could be experienced, knowing that these may be different from their own (Desmet & Schifferstein, 2012). Likewise, there have been many attempts to examine aesthetic interaction and its attributes. For example, it is defined as the dynamic, ongoing interaction between the artifact itself and the user’s cognitive structures (Locher et al., 2010).
From the interaction design domain, four aspects were defined in a case study on user interaction in website usage - the perceived usefulness, ease of use, hedonic quality and visual attractiveness (Mahlke, 2002). Apart from visual interaction between people and products, Desmet (2008) also suggested five elements of interaction dimension: Force, Sound, Motion, Texture, and Performance. Djajadiningrat et al. (Reference Djajadiningrat, Wensveen, Frens and Overbeeke2004) sees product appearance in the context of interaction design as the way in which objects appeal to our senses and motor skills, introducing three factors that play a role in aesthetics of interaction (Table 1). Freedom of interaction implies that a user can express herself in the interaction rather than following a fixed order or single path. On the other hand, interaction pattern draws out the timing, flow and rhythm that correspond between user action and product reaction, considerably influencing the feel of aesthetic interactions. For Richness of motor actions, operating and manipulating an object is a required interaction between user and product, implying that the way a product encourages human motor skills leads to aesthetic interaction.
Table 1. Three factors that play a role in aesthetics of interaction (Reference Djajadiningrat, Wensveen, Frens and OverbeekeDjajadiningrat et al., 2004)
2. Method
2.1. Experimental stimuli
A research-through-design approach was adopted applying the features of aesthetic interaction to the design of product packaging stimuli. The stimuli were then used to control other packaging factors and compare how different interaction styles evoked emotional responses. Three operational definitions (Reference Djajadiningrat, Wensveen, Frens and OverbeekeDjajadiningrat et al., 2004) of aesthetic interaction were used as guidelines and applied to the design of the experiment stimuli:
Based upon the three operational definitions (Table 1), an idea generation session for three experiment stimuli was conducted and three were chosen considering their fit to the three types of aesthetic interaction. Each stimulus was determined through unanimous consent between four researchers. For freedom of interaction (Type A), When a user interacts with a music player, do not restrict them in order or in rules. For interaction pattern (Type B), it should apply a pattern of behavior, in which interaction between the user and the product can lead to functionality. Richness of motor actions (Type C) increased the number of tasks required; it should give the user the opportunity to use a lot of athletic performance.
While the authors acknowledge the limitations of splitting the operational definitions into three distinct interaction types, we attempted to isolate the three to understand their influence within the context of the music playback control experience. While this was not the original domain of the Djajadiningrat et al (Reference Djajadiningrat, Wensveen, Frens and Overbeeke2004) constructs, they were useful to theoretically underpin how different interactions implicated emotional response in the current study of the music playback control experience.
2.1.1. Design workshop session
The goal of the first session, concept of aesthetic interaction, was to provide participants with an understanding of the concepts and components of aesthetic interaction and to provide requirements for the next session. In the second session, idea generation, participants were asked to develop an idea of various interaction methods for playing music according to the guidelines provided. In the third session, discussions, we were free to give feedback and give feedback on how the ideas developed in the previous sessions came up. Participants were 7 students who majored in industrial design at UNIST. 4 students were master’s students, and 3 students were doctoral students. Their age ranged from 25 to 28 years, with three men and four women. This design workshop consists of three sessions: concept of aesthetic interaction, idea generation, and discussion. Participants fully understood the concept of what an aesthetic interaction was, the three elements of it, and then received a requirement in the first session, concept of aesthetic interaction, for each element to be clearly applied to the music player. Since then, they have had enough time to answer and answer questions.
In the idea generation session, participants were asked to develop design ideas for various interaction methods that could play music by three aesthetic interaction elements. This was to see design implications with aesthetic interaction. This second session totaled 3 parts; It consists of freedom of interaction, interaction pattern and richness of motor action. Each part lasted for 15 minutes, for a total of 45 minutes, and was continuously delivered through the screen to remind us of the requirements provided in the previous session.
At the end of the design session, participants spent 30 minutes rotating their presentation of how and why the requirements were reflected in their ideas, and everyone was free to give and receive feedback. This session allowed me to refine the idea of a prototype design concept to act as a stimulus. This process was recorded video.
Figure 1. A picture of the idea generation session (left) and an example of generated ideas (right)
2.1.2. Three experimental prototypes for aesthetic interaction
-
Freedom of interaction
The user picks up the stick on the play icon and throws it on the silver plate to start running the product. Then, to execute any other desired function, simply pick the bar on the icon and just throw it. However, you need to pick a different stick after putting the stick back in place.
Figure 2. Freedom of interaction in the experimental stimulus
-
Interaction pattern
The user picks up the stick on the play icon, plugs it into a hole in the white rotating part, and rotates it once. Then, to execute other desired functions, plug the used stick into place, select the stick in the same way, insert it into the hole and rotate it one turn to execute the function.
Figure 3. Interaction pattern in the experimental stimulus
-
Richness of motor-action
The user needs to create an icon shape that we know as play, pause, previous, and next, with four bars plugged in to run the desired function.
Figure 4. Richness of motor-action in the experimental stimulus
To facilitate the experiment of the three types of aesthetic interaction, a remote controller was needed to control all functions related to the product. So, we used Android phone and BT chat application. This allows the functions to be executed, stopped, and changed in variable values necessary for determining the status. Interaction experiment number was assigned. When an experiment is conducted on one interaction, the function for the other two interactions is disabled. This completely blocks malfunctions and variables that occur during the experiment. Number 1 is Aesthetic interaction 1: Freedom of interaction, number 2 is Interaction 2: Interaction pattern, number 3 is interaction 3: richness of motor-actions. For example, if you enter 1 in the input box of the phone, only the functions related to interaction1 are activated, and the functions for the remaining interactions 2 and 3 are deactivated.
2.2. Measuring Emotional Responses
The main goal was to get information about what emotions dominated by aesthetic interaction. The emotions given for users to choose were six positive emotions (desire, satisfaction, pride, hope, joy, fascination) and six negative emotions (disgust, dissatisfaction, fear, shame, boredom) used in PreEmo. Table 2 is a detailed description and image of the emotions that users have received. This information was delivered via ppt to the MacBook, and the user was asked to be fully aware before using the product. Occasionally, if a user wants to be reminded in writing a self-emotion report after using the product, the image and explanation can be shown again, or a question can be asked. The user then wrote a five-point measure of emotion, written in a Google form, after sufficient product use. (Measured from 1 to 5, meaning that the closer the emotion is to 1, the smaller the emotion is the closer to 5, the stronger the emotion.)
Table 2. 12 emotions presented to participants in the experiment
2.3. Participants
The experiment lasted for 10 days and included 48 students who liked and enjoyed music at UNIST. (22 women, 26 men, their ages were 19-29 years old.) Prior to the experiment, they were asked for six favorite songs these days. The received playlist is inserted into the mp3 module of the product. This was to give the user an emotional familiarity as if it were their personal product.
2.4. Procedure
Participants are not told at all about the concept of aesthetic interaction. They are told that they will experience three ways to control music. Participants are then provided with information on 12 emotions. The participants were then asked to use each interaction method in less than five minutes. They were then asked to create a 5-point scale for feelings immediately after use. After going through this process three times because it was three methods, the participant had a short semi structure interview of five minutes. The interview was recorded for qualitative analysis.
3. Results
The Friedman test was performed using SPSS to identify the emotional differences between the three different interaction methods. Because the population does not follow a normal distribution, we chose the Friedman test, a nonparametric test of ANOVA. The independent variable is a stimuli with three different aesthetic interaction methods, and the dependent variable is 12 different emotions (positive emotion: 6, negative emotion: 6). The population is a random sample.
Figure 5 shows the mean value of the 48 subjects’ emotions for each of the three stimuli. The three Stimuli have the fact that each triggers a different intensity of emotion. In the case of positive emotions, all three stimuli showed statistically significant differences. (Positive emotion: Asymp. Sig. = 0.000, p <0.01). Among them, Stimuli 3 had significantly higher levels of positive emotions of all kinds compared to the other two stimuli. In particular, the values for Joy and fascination are the highest. (Joy: M = 4.50, SD = 0.652, fascination: M = 4.04, SD = 0.944). Stimuli1 has the same curvature as Stimuli3, but the numbers are slightly lower. (Joy: M = 4.27, SD = 0.893, fascination: M = 3.65, SD = 0.978). On the other hand, Stimuli2 showed all positive emotions as low as 3 or less on average, and Joy only showed 3 or higher. (Joy: M = 3.46, SD = 1.184). Overall, the three stimuli showed high Joy and Fascination, and the lowest pride and hope. (Stimuli1; pride: M = 3.06, SD = 1.156, hope: M = 3.27, SD = 0.984, Stimuli2; pride: M = 2.50, SD = 1.011, hope: M = 2.54, SD = 0.967, Stimuli3; pride: M = 3.54, SD = 1.091, hope: M = 3.67, SD = 0.930).
For negative emotions, the three stimuli showed statistically significant differences except fear (Asymp. Sig = 0.664, p <0.05) and sadness (Asymp. Sig = 0.062, p <0.05). (All three stimuli had very low levels of fear and sadness with M ≤ 1.5). In the case of Stimuli2, Dissatisfaction (M = 3.25, SD = 1.313), Shame (M = 2.56, SD = 1.382 and Boredom (M = 2.33, SD = 1.136) were relatively higher than those of the other two stimuli. On the other hand, dissatisfaction (M = 1.85, SD = 1.072), which was the highest in stimuli3, was about 2 times lower than that of Stimuli2, i.e. the least negative of the three stimuli.
Figure 5. Emotional responses to the three of aesthetic interaction
This study measures the semantic differences in terms of Usability and Interaction when an auditory experience is made through Stimuli, which has three different aesthetic interactions. ‘Dangerous – Safe’ (Asymp. Sig = 0.06, p <0.05), ‘Delicate-Robust’ (Asymp. Sig = 0.083, p <0.05), and ‘Difficult to Clean-Easy to Clean’ (Asymp. Sig = 0.363, p <0.05), and there were no statistically significant differences in the three items. Participants felt that all three stimuli were close to safe and robust, and neither was easy nor difficult for Clean. On the other hand, stimuli2 and stimuli3 were the salient features of the remaining five semantic items with significant differences. In the case of stimuli2, uncomfortable (M = 3.83, SD = 1.642) was most dominant, followed by impractical (M = 3.31, SD = 1.401). Stimuli3 felt clear without any confusion when compared to the rest of the stimuli (M = 5.44, SD = 1.236) and felt trustworthy. (M = 5.21, SD = 1.148)
Next, measure the difference in terms of product quality. There was no statistical difference in feeling that all 3stimuli were Geometric (Asymp = 0.915, p <0.05), but statistically significant difference for the other five semantic items. Most notable features include Stimuli3 Innovative (M = 5.48, SD = 1.111), Ornate (M = 4.77, SD = 1.341), Elegant (M = 2.54, SD = 1.246) and Stimuli2 Large (M = 3.29). , SD = 1.501) and Inelegant (M = 3.73, SD = 1.554). Stimuli1 compared with stimuli2 and stimuli3, the semantic value was the median between the two values, but for the ‘asymmetrical-symmetrical’ (M = 4.79, SD = 1.429) category, it was the most dominant of the three and felt close to symmetrical.
Finally, we measure the differences of stimuli for the product characteristics with 10 opposing adjectives. There was no statistically significant difference in Submissive-Aggressive (Asymp = 0.279, p <0.05), Immature-Mature (Asymp = 0.129, p <0.05), and Masculine-Feminine (Asymp = 0.393, p <0.05). All three Stimuli felt close to 4 on all three, slightly aggressive, close to immature, and almost perfectly neutral. Looking at the remaining seven items with statistically significant differences, Stimuli3 is attractive (M = 5.65, SD = 0.934), exciting (M = 5.04, SD = 1,271), friendly (M = 3.77, SD = 1.325), interesting. The values for (M = 5.79, SD = 0, .944) were dominant over the other two stimuli. For the remaining stimuli, stimuli1 felt noisy (M = 3.15, SD = 1.473) strongly, but not stimuli2, but stimuli2 felt nostalgic (M = 3.46, SD = 1.398).
4. Discussion
In this study, three core elements of aesthetic interaction were examined to understand their impact on the emotional and image evaluation of products offering auditory experiences. Design guidelines for each type of aesthetic interaction were established. The first element, ‘Freedom of Interaction,’ denotes open-ended or flexible forms of interaction, allowing for varied operational approaches. The second element, ‘Interaction Pattern,’ reflects the synchrony between user actions and the resulting feedback, ensuring that user behavior and product response are seamlessly connected in both timing and flow. The final element, ‘Richness of Motor Action,’ involves interactions composed of multiple sequential tasks that engage the user’s cognitive skills. Adopting a research-through-design approach, we devised a prototype incorporating these three operational types to assess emotional responses and image evaluations.
The self-reported emotion measurements revealed significant differences in all emotions except fear and sadness, with positive emotions prevailing. Notably, all three aesthetic interaction types predominantly elicited feelings of joy. For ‘Freedom of Interaction,’ participants frequently experienced joy and fascination, describing the interaction as fun, interesting, and remarkable, largely due to its simplicity. However, concerns about potential device misoperations were noted, highlighting the challenge of balancing freedom and reliability in interactions.
Regarding the ‘Interaction Pattern,’ joy was again prevalent, with users enjoying the interactive experience. However, the interaction evoked an old-fashioned image for some, likely due to cultural associations with actions resembling familiar rituals, like turning a “medal.” This underscores how cultural and social contexts influence product image perceptions.
In the ‘Richness of Motor Action’ interactions, joy and fascination emerged as dominant emotions. Qualitatively, participants noted the ease of engaging in the interactions without the burden of complexity. This suggests that designing multifunctional interactions in a straightforward manner can enhance user engagement and result in positive evaluations, indicating potential for application in broader design contexts.
5. Conclusions
This study contributes to the expanding field of aesthetic interaction research by building on the foundational concepts established by earlier scholars. By exploring three distinct methods of aesthetic interaction, each embodying unique elements, we developed a prototype that integrates these concepts into a physical object that offers auditory enjoyment. Through both qualitative and quantitative analysis, we identified the potential for these interaction methods to be adapted to other design contexts. We anticipate that future research will validate the significance of aesthetic interaction by empirically demonstrating its applicability across a broader range of products and services.
Open access
