Emotional Communication between Remote Couples: Exploring the Design of Wearable Ambient Displays

We explore the potential of limited vocabulary wearable ambient displays and their preferred characteristics in communicating intimacy between couples at a distance. The research comprises two studies: a co-design workshop and a field study of the design concepts generated from the co-design workshops using low-fidelity prototypes. Our findings reveal that wearable ambient displays could potentially help to increase awareness and emotional connections between couples. A limited vocabulary display is considered as a complementary channel to a smartphone, supporting the communication of subtle and lightweight messages that the sender does not necessarily expect a response to. From the user studies, we identify the preferred characteristics, as well as the potential and challenges of low vocabulary wearable ambient displays.


INTRODUCTION
Technology assisted means for communication have greatly increased during recent decades. However, mainstream communication means, e.g., voice, text messaging and video chat [14], mainly focus on functionality supporting the exchange of information, but show deficits in mediating affection and emotions [10,14]. Interestingly, subtle and minimal communication has been found to promote intimacy between couples [11]. In this paper, we further explore this minimal communication in the form of wearable ambient displays to support emotional communication, i.e., communicating emotions, feelings and awareness, between couples. Wearable displays are an interesting communication channel for such purposes, as they provide means for unobtrusive, easily glanceable messages, but, meanwhile, are publicly visible. Nonetheless, they are less investigated in the context of mediating emotional communication for couples. We aimed to reveal social, public and private aspects related to wearable ambient displays for intimate and unobtrusive communication between remote couples.

RELATED WORK
Researchers in the field of human-computer interaction (HCI) have shown that ambient displays can be used to communicate affection, emotions, and one's affective state, from the early works presenting the LumiTouch picture frame [4], followed by solutions such as the BioCrystal display [21] and EEG hearth pendant [1]. In line with Pousman and Stasko, we believe that the emotion communicated is "information that is important but not critical" [18] to the partner and can thus be displayed through a wearable ambient display. Harrison et al. [9] provide guidelines for the placement of such ambient displays on the body, advising e.g., to take into account the obscuring objects in everyday life, such as furniture. Wearable displays offer new design opportunities for functionalities, interaction design, and aesthetics. Dierk et al. [6] demonstrated different application domains for wearable displays, from notification to personal expression and social engagement. Information can be presented in a concrete way using texts, numbers, or icons, e.g., [22], as well as through symbolic and abstract visualisation. e.g., [8]. Pearson et al. [17] highlighted that a wearable display, even intentionally designed for individual use, such as a smartwatch, is noticeable by others in the surroundings. Hence, it is important to note that wearable displays are not entirely private [7]. In this paper, we further investigate how couples would use publicly visible wearable displays as an intimate communication channel between themselves, as well as exploring how they perceive such contrast.
The nature of wearable form factors provides a potential communication channel for intimate and affective communication. They can be worn directly on the body, may employ visual or multimodal designs beyond conventional mobile communication means, and can take the form of jewellery integrating the gift-giving traditions between couples. A growing body of research has emerged to explore a variety of wearable form factors for connecting remote couples. One line of research focuses on facilitating haptics via wearable form factors to mediate physical intimacy. For instance, Flex-N-Feel [23] is a glove which can imitate a feeling of touch by capturing the flex actions of fingers and transmit them to the other partner as vibro-tactile sensations. Another line of research explores the use of displays via wearable form factors to mediate emotional communication and enhance connectedness. For example, Ring U [19] is a system that consists of a wearable ring-shaped device and a smartphone. When a user squeezes the ring, their partner's ring will produce vibrotactile stimulation and display an accompanying colour lighting. More recently, the emergence of virtual reality has opened up a vibrant design space for creating a sense of remote presence. My eyes [16] provides a first person view, enabling remote partners to exchange each other's view on their displays.
Although the potential of wearable form factors has been recognised, a systematic literature review on emotional communication systems for remote couples [12], has reported that research around the area of wearable form factors focusing on the target group of remote couples is scarce. Moreover, the work [12] revealed that user participation was lacking in most of the reviewed systems. On the other hand, there are some exceptions. Vetere et al. [25] employed ethnographic techniques on couples to understand how technologies can be designed to support intimate acts. Based on the empirical work, they produced a series of related ideas that were generated within the research team, e.g., Secret Touch [24], as well as engaging HCI experts and users in two workshops to create design sketches [25]. Although Vetere et al. [25] thoroughly investigated the users to understand how technologies are used within intimate relationships with an ethnographic study, the users were only involved in the early phase of the design process. To bridge this gap, we take a co-design approach by engaging potential users in designing wearable ambient displays to support emotional communication between remote couples. We engaged participants throughout the design process as co-designers, not only in the ideation phase, but also in evaluating the concepts in-the-wild.

RESEARCH PROCEDURE
This research employed a research through design approach [28], which consisted of two parts: a co-design workshop with lowresolution prototyping, and a concept evaluation in-the-wild. In the co-design workshop, we explored how wearable ambient displays could be used for remote couples to communicate intimacy and co-designed four concepts with the participants. After this, we created polished low-fidelity prototypes of the concepts, and evaluated them in real-world contexts. All participants gave informed consent for participation and were compensated with one movie ticket, worth about ten euros. Data were collected through open-ended questionnaires, online surveys, photographs, videos and audio recordings during the study. Personal data collected in the research were encoded and strictly processed for scientific research purposes. As this study was both qualitative and exploratory, we adopted the thematic analysis method [3] to carry out the data analysis which was conducted by two of the authors to ensure accurate interpretation of the data. We started with data cleaning, where the video and audio recordings were transcribed in verbatim. Following this, multiple coding collections were created and then grouped based on perceived commonality, in order to identify a number of themes and patterns, which are presented in section 4.

The Co-design Workshop
The co-design workshop (N=16, mean age 27) consisted of two identical, two-hour sessions which were arranged based on the participants availability. Each session had eight participants (4 F, 4 M) who were equally divided into two groups. Each group was tasked to create a design concept using basic craft supplies. There was at least one participant who had a design background in each group. After creating the low-fidelity prototypes, all groups were engaged in a role-playing activity, where they demonstrated the use cases of the concepts. Altogether, the workshop resulted in four concepts. All participants were currently involved in a romantic relationship, with eleven participants being in a long-distance relationship. They all had experience in using wearable devices, e.g., smart watch.

Concept Evaluation In-the-Wild
To evaluate the four concepts in the wild, we had an industrial designer create and 3D printed polished low-fidelity prototypes based on the concepts the participants came up with. As the prototypes were non-functional, it made sense to have the same participants who had created the concepts to (imaginarily) use their own design, as they had the best knowledge of how the concepts should work. Hence, we asked eight participants from the earlier workshop, i.e., one male and one female from each group who participated in designing the concepts, to use and interact with the prototypes as if they were functional for two days. The experience sampling method (ESM) [5] was used to capture the participants' experiences, by sending ESM triggers to ask them to provide systematic self-reports on their thoughts, feelings and behaviours when (imaginarily) using the prototypes at random occasions during the study. This resulted in a dataset of text descriptions and 96 photos from the participants (for an example, see Figure 1-3). At the end of each day, the participants were asked to complete an online survey to share their overall experience with the prototype. The questions were designed in order to investigate three main aspects: 1) positive and negative experiences with the prototypes; 2) contexts when using the prototypes; and 3) improvements for the prototypes. Finally, we conducted 15-20 minutes one-to-one semi-structured exit interviews with the participants after the study, in order to seek clarification on the answers they had given in the online surveys.

FINDINGS 4.1 The Concepts
The four concepts created in the co-design workshop were: a bracelet, a ring, a multi-wear pin and a necklace. The bracelet shows four different visual symbols that convey intimate and personal sentiments accompanied with haptic feedback to simulate the touch sensation from their partner (Figure 1-1a). The ring has an ambient display which shows a symbol visualising their partner's emotion. The emotion status can be shared by pressing the ring. This triggers colour changes on the partner's ring display (Figure 1-1b). The multi-wear pin consists of a display which shows abstract patterns as a representation of the partner's emotional state, a flexible cord which supports the user to wear the display at different body locations according to their preference, and a dial for sending messages. The user needs to find a deliberately quiet moment to be able to send a message, in order to create a thoughtful form of communication, cherishing the effort needed (Figure 1-1c). The necklace has a pendant with a display for the couple to convey intimate and personal sentiments, where the changing colour and pattern create a subtle vibration (Figure 1-1d).
The two main reasons that influenced on the participants' design choices for their wearable devices were the practicality and the symbolic meaning of certain form factors. Visibility was also discussed. While the participants would like their design to be always visible to them, they preferred the devices to be discreet and not to be too easily noticeable by others.

Overall Experience with the Concept Prototypes
Overall, the participants appreciated the abstract form of communication that the prototypes would provide, albeit at low-resolution. Apart from the two participants who had the rings which were found less discreet due to their size, the rest of the participants were comfortable wearing the prototypes in different contexts, from being at home, to being at a public place, e.g., at the gym or on a bus. This was mainly because their prototypes looked like a discreet item of jewellery which would not be obviously perceivable by others in the vicinity. The prototypes were appreciated for allowing them to receive messages from their partners, particularly in the contexts when using mainstream communication tools, such as calling and texting, might not be possible or appropriate. For instance, Participant 5 (P5) used the prototype when she was attending a church service. Additionally, the advantage of multi-wear form factors was noted for being able to fit multiple contexts, not just for style and aesthetic purposes, but also for practicality. For example, P10 wore the multi-wear pin on his wrist during exercise, so as to make it easier to notice the incoming message (see Figure  1-3c). The prototypes were believed to be useful as an alternative communication channel to communicate intimacy. P6 would use the prototype to send a good night message to his partner, because he found using the phone distracting before going to sleep, "I'm going to sleep now, I'm in bed and ready to tuck in. I could just send her a good night by pressing the heart symbol. I don't want to use the phone because the light from the screen could distract me from sleep. I'd be bothered by notifications popping up." (P6, Bracelet, see Figure 1-1c). Messages that the participants envisioned sending to their partner varied from thinking about their loved one, reporting a daily routine, to expressing their feelings. The participants reported that they would assign different meanings to the visual content shown on the display, so as to convey non-verbal messages that are symbolic and private. For example, P14 would customise different patterns and colours to express his daily routine, e.g., "I'm playing with Tuli [the dog's name]" (P14, Necklace, see Figure 1-4c).

Simplicity and Devotion -Connected but Not Distracted
The participants highlighted the potential of their prototypes to be simple and having only one function, that is, to communicate with their partner. In contrast to smartphones that are used to communicate with everyone and are equipped with numerous functions, the prototypes made the communication and connection feel special as it was dedicated only to their partner. Furthermore, the limited vocabulary of the prototypes would restrict the communication to be mostly non-verbal. The participants reported that this could make their communication feel special and different from the mainstream communication channels, "I just imagined, okay now I could send just a little emotion thing like 'thinking about you'. It wasn't with words but just with colours and patterns, and that made it feel a bit different [...] So if you can get this special little thing, you don't want to make it too complicated, because then it's just another mini smartphone on the chest. ", (P14, Necklace). The participants envisioned that with their prototypes, communication with their partner would be easy and convenient, "It felt great being able to do normal daily things and still be able to communicate feeling and moment with a loved one [...] " (P16, Necklace). The participants considered the prototypes would offer a communication channel that would enable a feeling of always connected between the two, but meanwhile would be less distracting, " I could reach my partner at any time without disturbing her or without needing an answer back from her. It's a way of communicating without being intrusive." (P6, Bracelet). Furthermore, the participants considered their prototypes would offer a lightweight communication.
That is, getting messages through the prototypes would reduce the obligation to react instantly, thus would be less stressful, unlike communications through instant messaging applications, " I didn't feel under pressure to respond immediately which I would have felt in instant messaging." (P2, Ring).

Challenges and Suggestions
As a challenge, it was commented that sending a message by touching the ring might be not be ideal, as a message might be sent unintentionally while they were fiddling with the ring. Given the visual change on the display would be subtle, adding additional output modalities to notify an arriving message was also suggested. With the bracelet, it was suggested that the device could offer replaceable displays, which would allow users to choose different visual content and could also better enrich the intimate communication. Intriguingly, the participants showed a point of divergence on effortless and effortful communication. P10 found using the multi-wear pin to send a message in the context of hiking could be challenging, as he would have to find a quiet spot and turn the dial whilst on the move. Although being effortful, the participants who designed the multi-wear pin still appreciated the act of putting effort into communicating intimacy with a loved one, which was viewed as a means of showing deliberateness and a more expressive way of communication, compared to simply pushing a button. Nonetheless, touch was still favoured in the design concepts as a relatively effortless input modality. One of the reasons might be that touch is widely used a common interaction modality in current communication devices. The input modality of the necklace was firstly designed as turning a switch in the workshop, however, during the in-the-wild study, P16 started to question whether it would be easier to activate the device by pressing buttons, e.g., when wearing gloves.

DISCUSSION AND CONCLUSION
We acknowledge that our work is limited by the fact that the duration of the evaluation of the concepts in the wild was relatively short, and we did not have functional prototypes in the study. Our field study focused on investigating how the concepts created in the workshop concepts would actually work in different contexts in real life. Given using the prototypes would be imaginary, we asked the same participants who had created the concepts to evaluate their own design, as they had the best knowledge of how the concepts should work, although there might be a chance of being (positively) biased. The methodology sought to take the advantages of in-the-wild studies [20], while avoiding the common weaknesses, e.g., unreliability of prototype functionality and bulkiness of the implementation. We also note the use of imaginary device functionality as a method in prior field study based research [2,26]. Our method provided early feedback and first-hand speculative experience of using such unconventional concepts in complex, real-world contexts.
Our findings show that the messages wished to be conveyed through wearable ambient displays were implicit, subtle and with a high level of personal meaning. Positive feedback was given to its simplicity and focus. The limited vocabulary of the communication channel could lead to abstract and poetic kind of communication between a couple, which our participants considered to be special. Wearable ambient displays have the potential to serve as a poetic communication channel for couples to communicate intimacy in a subtle way. Even though the display itself is publicly visible, the personal meanings "embedded" and "encoded" in it would colour the experience when using the device. This can be reflected against the technology experience argued by McCarthy and Wright [13]. We suggest that wearable ambient displays should be designed to be as discreet as possible in order to serve the purpose of everyday wear. The form factors our participants chosen were jewelries that can be commonly seen in life. Customisation was frequently highlighted by the participants. We suggest that users should be given the freedom to customise some small details, e.g., symbols, colours and patterns. Similar to how couples have reportedly repurposed the "intended" use of emoji in highly personalised and purposefully secretive ways [27], by providing users with the opportunity to customise the visual contents into "secret codes", it would then be socially comfortable to wear the ambient display as a decorative piece in different contexts since the meaning would not be understandable to bystanders. To avoid being an extra burden of technology for the world, the design of wearable ambient displays should be ambitiously different from a conventional communication device which is often used for information exchange and explicit communication, in order to emphasise the use on the sharing of emotions and affection. Compared to a smartphone, glancing at a wearable ambient display does not initiate the reported phone checking behaviour, which easily leads to the habit of extensive phone use and interrupts the social context [15].

ACKNOWLEDGMENTS
This research has been supported by the Academy of Finland as part of the 'TechFashion -Design of Future Wearable Computing' project, the European Union's Horizon 2020 DecoChrom project (Grant Agreement no. 760973), as well as the China Scholarship Council fellowship (201606150085).