Uncategorized

1992-93 Aynur Erdogan 150 150 John

1992-93 Aynur Erdogan

Date of MSc1992-93

 

Project Title: Controls and Stereotypes

  

Pre-MSc Background: BSc in Industrial Engineering at Yildiz University, Turkey

 

Pre-MSc View of HCI/Cognitive Ergonomics:

I had some ideas about different principles – physiology, bio-mechanics, and anthropometry and their applications in the physical design of the work environment, but I was totally ignorant about any relation of Psychology to Ergonomics.

 

Post-MSc View of HCI/Cognitive Ergonomics: 

I was surprised to see people with a Psychology background on the course and wondered why Psychology was relevant to the study of Ergonomics. However, it was very engaging and fascinating to be exposed to new concepts.

 

Subsequent-to-MSc View of HCI/Cognitive Ergonomics:

I recognised the relevance of Psychology for the study of Ergonomics.

 

Additional Reflections

My interest in Ergonomics started, while I was studying for my BSc. It was part of my Industrial Engineering degree and I wanted to take it a stage further. In those days, industries in Turkey were just starting to realise the benefits of Ergonomics and what  it might bring  to product design and to the work environment, more generally. I wanted to be part of it. After my degree, I spent time studying English in London and travelling. I spent quite a lot of time researching different universities before applying. The Ergonomics Unit at UCL looked very strong and well-connected and I love London, so I guess UCL was the obvious place to apply. I was very proud to be accepted.

John and Rachel were clearly extremely respected and highly regarded in the Ergonomics Society. I vividly remember being very anxious before my interview with John Long; but John made the interview very informal and relaxing.

I always thought John had a degree in Philosophy, just because his lectures were always very deep and cerebral and all about the principles of Ergonomics. On the other hand, Rachel’s lectures were more about practical applications.

I thoroughly enjoyed the course. The subjects were very detailed, the topics were very stimulating, and the course was very well structured. Studying in another language can be very daunting and studying such technical topics was  challenging in my second language.

We were given the opportunity to experience different applications of Ergonomics in a wide range of industries through the visits, that were organised for us. Two of my favourite ones were the visits to a coal mine and to an airport control room. During these visits, you could see that the course was highly valued and very well connected with different industries. We also were presented with a wide range of case studies and had many experienced guest tutors to give of their specialist knowledge.

 

I never used ergonomics in my career. I try applying it in my own work and home environment in an informal way. My friends and family must be fed up with me complaining about products that are badly designed and not user-friendly. Being students, my children spend lots of time at a desk and on their computers and of course, the work environment constantly needs to be changed, because they are still growing up. I ensure that they have the best designed student work environment.

 

After my studies, I started to work for a charity and decided to pursue a different career path from my education. People often ask me if I regret about not following my education path. I guess the answer is no, I do not regret it. Rarely I do wonder what would have happened, if I had. Although, people on the course were wonderful and very friendly, I have only managed to keep in contact with one person from the course. She did very well in her Ergonomics career.

 
1993/1994 Ismail Ismail 150 150 John

1993/1994 Ismail Ismail

1993/1994 Ismail ISMAIL

 

Date of MSc: 1993/1994

 

Project Title: Reasoning about Fidelity Requirements in User Interface Simulation: The Case of Human Simulation

 

Pre-MSc Background: Graduated Computing Science, University of Greenwich in 1993. Took the HCI option in my final year.

 

Pre-MSc View of HCI/Cognitive Ergonomics:

HCI was more to the user interface design and widgets than anything else,  I learnt about research methods, tools and techniques to better understand the human but didn’t really understand how this methodological insight would support better design.

 

Post-MSc View of HCI/Cognitive Ergonomics:

The MSc Ergonomics was by far the most significant academic milestone in my life (more so than studying for a doctorate). The course blew my mind and my assumptions and introduced me to proper ‘Science’ and all it entails. The highly multidisciplinary nature of the course was particularly important as it introduced me to different branches of psychology and other human sciences (e.g. biomechanics was simply brilliant) and how this insight could be used practically to create better products, tools and techniques.

In reality the MSc was really only scratching the surface of the discipline but my goodness it was an awesome scratch!

 

Additional Reflections:

John Long’s Foundations lectures were legendary. They elegantly brought together the different facets of the discipline in a framework that helped unify my understanding of it.

Finally, the MSc Ergonomics alumni are all over the world and it gives me great pleasure when I meet someone else that did it.

 

 

 

 

1990/91 Ted Snowdon 150 150 John

1990/91 Ted Snowdon

 

This CV is a temporary place-holder for Ted’s MSc Reflections

 

 

Screen shot 2014-10-03 at 19.47.53

 

 

Summary

As of 2014 I have over 20 years experience of working in the field of statistics/analytics and the software development of analytical applications.

Experience

Workforce Analytics, Data Scientist

IBM

2013 – Present (1 year)

unknown.png

User Experience Professional

IBM Hursley Laboratories

 

February 2000 – 2013 (13 years)

Statistician

Office for National Statistics

February 1992 – January 2000 (8 years)London, United Kingdom

Final position: Statistician responsible for leading the team compiling, seasonally adjusting and publishing the headline unemployment figures for the UK, based on the Labour Force Survey.

User Interface designer

Rank Xerox ,

1991 – 1992 (1 year)Welwyn Garden City, Herts

 

 

Screen shot 2014-10-03 at 19.47.53

 

 

1997/98 Florian Egger 150 150 John

1997/98 Florian Egger

This brief CV acts as a temporary place-holder until Florian’s reflections are received.

Owner & Principal

Telono

August 2005 – Present (9 years 2 months)Geneva Area, Switzerland

Owner, Managing Director and Principal Consultant of Telono, a Swiss-based User Experience (UX) research and design agency specialized in multi-lingual and international user research projects.

Partner & Swiss Representative

UXalliance

March 2009 – Present (5 years 7 months)Geneva Area, Switzerland

Swiss representative of the User Experience Alliance (UXalliance), an international network of local user experience experts providing global coverage.

Freelance Usability & User Experience Consultant

ecommUSE

June 1999 – August 2005 (6 years 3 months)

Freelance consulting in HCI, usability engineering and user experience, first based in The Netherlands, later in Switzerland.

Research Assistant in Human-Computer Interaction (HCI)

Eindhoven University of Technology

January 1999 – May 2003 (4 years 5 months)

The title of my PhD thesis was:
”From Interactions To transactions: Designing the Trust Experience for Business-to-Consumer Electronic Commerce”

Education

Eindhoven University of Technology

PhD, Human-Computer Interaction

1999 – 2003

UCL

MSc, Human-Computer Interaction

1997 – 1998

City University London

BSc, Psychology & Philosophy

1994 – 1997

CESSRIVE

Swiss Federal Maturity & Baccalaureate, Latin-English

1991 – 1994

Invention 150 150 John

Invention

An invention goes beyond the known. It may take many forms – object; product; machine; device etc. It may also be realised as a method; composition; process etc. The key feature of an invention is its novelty, characterised generally in terms such as – new; unique; original; never been designed or made before etc. An invention may be patentable, that is, protected in law. A patent requires an invention to be non-obvious and workable.

An invention is recognised as the product of some unique intuition or creativity, as distinguished from more ordinary skill or craftsmanship. Going beyond the known, its knowledge cannot be codified to make its practice replicable, although the latter is supported by the inventor’s experience, their reflections, communication with other inventors and familiarity with relevant patents etc.

Inventors may be engineers; architects; designers; scientists; artists etc. Famous inventions and their inventors (in brackets) include the: steam engine (James Watt); computer {Charles Babbage); light bulb (Thomas Eddison); World-wide Web (Tim Berners-Lee) etc.

 

The process of invention typically starts from a novel idea or concept. It is then developed by trial and error or generate and test practice, involving drawing; writing; making models etc. Inventing is often a collaborative practice, as well as a creative one. The idea of an invention needs ultimately to be realised as a working device; prototype; simulation etc. Inventing can be contrasted to innovating, which implies replication to meet some specified social need.

HCI Frameworks

Discipline

Problem

Research

Know;edge

Practice

Paper – Obrist et al. Temporal, Affective, and Embodied Characteristics of Taste Experiences: A Framework for Design 150 150 John

Paper – Obrist et al. Temporal, Affective, and Embodied Characteristics of Taste Experiences: A Framework for Design

Temporal, Affective, and Embodied Characteristics of Taste Experiences: A Framework for Design Marianna Obrist1,2, Rob Comber1, Sriram Subramanian3, Betina Piqueras-Fiszman4, Carlos Velasco4, Charles Spence4 m.obrist@sussex.ac.uk | rob.comber@ncl.ac.uk | sriram@cs.bris.ac.uk | betina.piqueras-fiszman@psy.ox.ac.uk | carlos.velasco@psy.ox.ac.uk | charles.spence@psy.ox.ac.uk 1Culture Lab, School of Computing Science Newcastle University, UK 2School of Engineering and Informatics University of Sussex, UK 3Deptartment of Computer Science, University of Bristol, UK 4Department of Experimental Psychology, University of Oxford, UK ABSTRACT We present rich descriptions of taste experience through an analysis of the diachronic and synchronic experiences of each of the five basic taste qualities: sweet, sour, salt, bitter, and umami. Our findings, based on a combination of user experience evaluation techniques highlight three main themes: temporality, affective reactions, and embodiment. We present the taste characteristics as a framework for design and discuss each taste in order to elucidate the design qualities of individual taste experiences. These findings add a semantic understanding of taste experiences, their temporality enhanced through descriptions of the affective reactions and embodiment that the five basic tastes elicit. These findings are discussed on the basis of established psychological and behavioral phenomena, highlighting the potential for taste-enhanced design. Author Keywords Taste; user experience; taste experiences; sensory research; explicitation interview technique; sensual evaluation tool. ACM Classification Keywords H.5.2 Information interfaces and presentation (e.g., HCI): Miscellaneous. INTRODUCTION Experts in taste perception agree on at least five basic tastes [40]. Beyond this, however, we lack insights into the rich experience of these tastes. This lack of experiential understanding extends beyond HCI, as sensory researchers have also acknowledged that: What is not well researched is the link between the food that goes into our mouth and what we think of it [12]. There is a growing interest in taste within the HCI community [e.g., 16,17,18,22,27,28], particularly relating to technical challenges in designing for taste stimulation and one-off designs to enhance user experiences through the manipulation of taste. There is a need for a more systematic study of people’s taste experiences and their specific characteristics in order to make a fuller use of this sense in future taste-enhanced technologies. This paper stands as a first step in addressing this need. Drawing on neuroscience and sensory research in combination with user experience evaluation techniques, we investigated how all five basic tastes are experienced at a given time (synchronic) and how they evolve over time (diachronic). We used pure tastants (i.e., that have no smell or visual qualities) with an explicitation interview technique [41] designed to encourage the participants to verbalize their experiences. Additionally, we used physical objects from the Sensual Evaluation Instrument [13] to elicit affective responses, and create a flexible, non-verbal channel of communication between the user and designers. This paper makes a number of contributions: First, we provide a rich description of subjective taste experiences along both the diachronic and synchronic characteristics of the five basic tastes. Second, these taste characteristics establish a framework for taste experiences and elucidate the potential design qualities of individual tastes. We demonstrate how each quality can be described along three main themes: temporality, affective reactions, and embodiment. Third, our findings extend human-computer interaction research on taste through a user experience perspective. Overall, our findings provide interaction designers and user experience researchers with a richer understanding of taste experiences and their specific power to influence human behavior and decision-making. The framework presented here enables the HCI community to think and talk about taste in the design of interactive systems in a fine-grained manner. RELATED WORK This section provides an overview of the human sense of taste and its relevance for HCI based on ongoing research. The sense of taste Sensory researchers and neuroscientists agree on five basic tastes (sweet, sour, salty, bitter and umami), and a ‘gustotopic map’ linking these classes of receptors with particular brain areas is currently being developed [40]. However, despite breakthroughs in understanding the sense Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from Permissions@acm.org. CHI 2014, April 26 – May 01 2014, Toronto, ON, Canada Copyright 2014 ACM 978-1-4503-2473-1/14/04…$15.00. http://dx.doi.org/10.1145/2556288.2557007 of taste, scientists have still not approached the phenomenology of taste nor developed a semantic understanding of how taste is experienced [30]. Although a wide body of sensory research has studied the temporal evolution of taste perception using labeled intensity scales [e.g., 1,8] and more specific time-intensity sensory evaluation scales [26], insights are limited to the quantification of temporal responses to perceived taste intensities. Such scale-based evaluations leave us uninformed as to the subjective qualities that lie behind the ratings of the perceived taste experience over time. Recently, neuroscientists have studied taste-specific temporal profiles by comparing sensory evaluation scales with functional MRI (fMRI) data [19]. Their results suggested that salty tastes change more rapidly than sweet tastes in the cerebral cortex, and confirm the same patterns that have been observed using time–intensity sensory evaluation [19]. While such results are intriguing, they cannot explain the differences in experienced tastes. To account for subjective differences, the ‘taster status’ measure has been introduced [2,6]. By means of such tests, it is possible to identify participants’ subjective sensitivity to bitter tastes and to distinguish between supertasters (25% of population), medium tasters (50%), and non-tasters (25%) [3]. Taster status has been considered to partially explain why some consumers like certain foods more than others and how they describe the way they experience them. Food-interaction design The last few years have seen increasing interest in designing human-food interaction in HCI [e.g., 4,9,11,33]. Such research looks to position human-food interaction within the wider spectrum of social, environmental, and physiological influences on our food practices. In this area, there is a growing realization of the potential for new technologies to support pleasurable experiences around food [20,35], and the potential for designers to draw on the extensive research on multisensory experiences (i.e., auditory, tactile, visual, olfactory, and gustatory). Despite this increased interest in food experience, we know little about the richness of people’s taste experiences. The majority of the studies on food experience combine taste with other modalities, where taste is but one component [e.g., 18,27,28]. For instance, Schifferstein et al. [31] elicited emotional experiences across the different stages of food product usage, from choosing a product in the supermarket through to cooking and eating [31]. Taste experience is interwoven with vision, touch, and olfaction, which, in combination create multisensory food experiences. Desmet and Schifferstein [5] also explored the emotions elicited through eating and tasting food. They describe variables related to food-evoked emotions, such as sensory features, product type, food-related activities, context, and the agent (who consumes, prepares, or produces). Due to the wide range of influencing variables, it is not clear how well these findings translate beyond the specific context of their studies. Taste-enhanced technology Technological advances in creating taste stimulations [27,28] and one-off applications exploiting taste in games [16] and other scenarios [18,22] demonstrate a growing interest in the use of taste in interactive applications. For instance, Ranasinghe et al. [27,28] developed a tongue interface that creates taste through the combination of electrical and thermal stimulation. They use electrical pulses applied to the tongue. Verbal descriptors provided by participants were, for instance, a ‘refreshing taste’ or ‘minty taste’ in relation to the change in temperature. The authors call for future work to understand the particularities of such taste (flavor) experiences. They focused on the introduction of taste in digital communication to enhance long-distance family relations and create remote co-presence and coliving experiences (e.g., remote dining) [28]. Murer et al. [16] designed a gustatory game device, LOLLio, which consists of an interactive lollipop that serves as a haptic input device that dynamically changes its taste between sweet and sour. Remote triggering of taste while motion sensing with accelerometers allows LOLLio to be used as an input modality. The authors identify various ways in which taste could be used in an interaction, such as to provide reward or punishment or else to provide hidden information through taste stimuli. LOLLio was evaluated in a game context with children [17]. Sweetness was constantly used in the game session and sour stimuli were used in combination with game mechanics to provide ‘negative reinforcement’. Their findings suggest an enhanced playing experience through taste stimulation motivating further explorations of such taste-enhanced interaction experiences. STUDY METHOD AND PROCEDURE Sensory research provides important information regarding the objective measures of taste perception, temporality, and subjective sensitivity levels. Yet, an understanding of the subjective understanding of taste experiences is missing. This study explores the diachronic and synchronic structure (explained below) of each of the five basic tastes. Methodology For our study, we combine two verbal and non-verbal user experience and elicitation methods, the explicitation interview technique (verbal method) and the ‘Sensual Evaluation Instrument’ (non-verbal method). The explicitation interview technique [41] is used to elicit verbalizations of subjective experiences. This technique helps to explore the unfolding of an experience over time, the ‘diachronic’ dimension, and examines the specific facets of the experience at a particular moment, the ‘synchronic’ structure (see also [24,39]). The value of this interview technique lies in helping participants to express their experiences at a specific moment. Participants are encouraged to talk about the experiential (cognitive, perceptive, sensory, and affective) aspects of the moment without building on rational comments and explanations [24]. Questions related to the diachronic structure help to understand how the description of an experience unfolds over time (e.g. “What happened after you opened the door?” and “What did you perceive next?”). With respect to the synchronic structure of an experience, the participant is questioned about a particular moment (e.g. “At the moment when you pushed the handle down, how did it feel?” or “What else came in your mind?”). In comparison to open questioning approaches, this technique is non-inducive but directive [24] in the sense that it keeps the participant talking about the experience without inducing any content; it focuses on the structure of the experience, and directive, as it keeps the participant focused on the singular experience being explored. Although it is typically used retrospectively to support the reconstruction of an experience, it has also been used in-situ (e.g., [15,23]). The Sensual Evaluation Instrument (SEI) is a non-verbal tool that can be used to elicit users’ affective reactions [13]. SEI is composed of sculpted objects that can be held in the hand, used by a person to indicate how they are feeling as they interact with a system. The SEI includes eight objects with different shapes, which represent various levels of arousal and valence (positive and negative). Isbister et al. [13] describe SEI objects as evoking and expressing a range of emotions; they do not claim a direct mapping between the objects and the mentioned emotions, but emphasize the benefit of the objects for stimulating expressiveness. The value of the SEI is to elicit real-time, affective responses, and to create a flexible, non-verbal channel of communication between user and designers. The latter defines a key advantage compared to other methods that are often limited to verbalizations or visualizations that lack physicality. Taste stimuli The stimuli used for each taste are specified in Table 1. Each stimulus was prepared as an odorless and colorless water solution using a stock solution as specified in ISO 3972. We prepared the solutions according to the specifications detailed by Hoehl et al. [10] and used deionized water for the tastants. These compounds standardised stimulus features and controlled for sensory differences, such as texture, vision, etc. All of the solutions were prepared the day before each study day. The participants received 20 ml of each stimulus in a disposable 40 ml cup. A Latin square design was used to avoid order bias [42]. Taste Stimuli used Solution (g/L) Sweet Sucrose 24.00 mg Sour Citric acid 1.20 mg Salty Sodium chloride 4.00 mg Bitter Caffeine 0.54 mg Umami Monosodium glutamate 2.00 mg Table 1. Stimuli used for the five main tastes, including the stock solution (indicating the threshold specified in ISO 3972). Participants The study was conducted with 20 participants (nine female) aged between 21-38 years (M=29.4, SD=5). Participants were recruited based on the following criteria: not having any food allergies, being non-smokers, not being pregnant, and not having any sensory dysfunction (e.g., dysguesia, a taste disorder), by self-report. The participants were recruited through the staff list within the lead university. 16 participants were native English speakers, and the remaining four were fluent in English. All participants gave informed consent prior to the study. Study set up and procedure The participants were instructed and reminded 2 days prior to the study not to eat spicy food 24 hours before the study and not to drink or eat 1 hour before attending the study. The study had 2 parts (see Figure 1): In the first part, we applied the explicitation interview technique for all five tastes; in the second part we introduced the SEI objects to enhance the verbalizations for each taste. Figure 1. Study set up: Left shows the five taste stimuli (40ml cups with odorless and colorless water solutions for each stimulus). Right shows the SEI objects placed inside a box. In the first part, participants were given 5 minutes per stimulus. They could take as many sips as they wanted of the stimulus and were prompted with specific questions about their taste experience (e.g., Could you describe what you perceive? How does it feel in your mouth?). The aim was to receive insights regarding the diachronic and synchronic structure of the taste experience. We used this technique in-situ in order to account for the rapidly decaying sensory memory trace related to the human sense of taste [21]. Before continuing with the next stimulus, the participants were asked to have a sip of the deionized water in order to cleanse their mouth. The same procedure was repeated for all stimuli. In the second part of the study, the participants were instructed to match each taste experience to one or more of the eight shapes inside the box. The participants could only feel, and not see, the objects, to exclude any visual influences and to focus on the mapping between ‘taste and shape’ via the sense of touch. The participants were instructed to select one or more or none of the shapes (they could also reuse shapes for different tastes). Before going through each taste stimulus again, the participants were given the chance to put their hands into the box and familiarize themselves with the 8 shapes. Next they were asked to take a sip of water and start with the first taste stimulus. They were asked to express the thoughts they had in mind and to describe their choices or lack thereof (if none of the shapes was selected). Finally, the participants were asked to rate the pleasantness/ unpleasantness of the shapes on a four-point Likert scale from ‘very pleasant’ to ‘very unpleasant’. They were also asked about their personal favorites amongst the 5 taste stimuli and their personal food preferences to support the interpretation of the data. In a final step, we tested the participants for their taster status, which classified participants into supertaster, normal tasters, and non-tasters. Overall, the study lasted one hour and was audio/video recorded with the consent of the participants. No incentives were paid to the participants. Data analysis All 20 tasting sessions were transcribed and a qualitative analysis based on the transcripts was conducted. Two researchers independently performed an open thematic coding based on 5 cases (25%). The resulting themes were discussed and an initial coding scheme was established. Two more cases (10%) were coded independently leading to a final coding scheme consisting of three main themes (described in the next section), which were then applied to the remaining 13 cases by both researchers. We also performed a qualitative analysis of the mapping between the SEI objects (see Figure 4) and the taste experiences, captured through the transcripts and the visual material from the recorded hand movements in the second part of the study. Based on participants’ ratings of the shapes (their physical pleasantness/unpleasantness) we could confirm previous ratings of Isbister et al. [13] – the more spiky shapes were rated as ‘unpleasant to slightly unpleasant’ (shapes 8,7,2), the more rounded shapes were rated ‘very pleasant to pleasant’ (shapes 3,4,5,6), and only one shape was perceived as ‘neutral’ (shape 1). Finally, the supertaster test provided us with insights on the different taste sensibility of participants and ensures a good distribution of taster statuses in our study. Overall, we identified 5 nontasters, 11 normal taster (4 tending towards the upper edge of bitterness sensitivity), and 4 supertasters. These results are consistent with the known distribution amongst the general population [3]. STUDY FINDINGS The description of taste experiences is based on both parts of the study. We describe the characteristics of taste experiences across all five tastes along three identified themes: (1) temporality, (2) affective reactions, and (3) embodiment (see overview in the supplementary material). We also discuss the particularities of each individual taste in order to elucidate the potential design qualities of single tastes. Each identified theme is represented in a pictorial visualization of its key characteristics based on the identified patterns across participants’ verbalizations. Temporality While taste experiences have expected elements of changing intensity (e.g., strong taste, weak taste), the tastes were also perceived as being mobile (e.g., moving within the mouth, moving intensities), and occasionally exerted a physical presence (e.g., building up, eroding, lingering). These temporal characteristics are intertwined in the unfolding of the experiences from its initial stimulation (diachronic structure) and set the stage for the different taste journeys (synchronic structure). Below, we describe the different time-intensity profiles of taste experiences. Taste intensities are generally experienced as being dynamic and participants’ verbalizations offer a lexicon of growth and decline. The diachronic nature of taste experience is also revealed in the immediacy or longevity of dynamic intensities. For instance, all participants agree on the immediacy of the sour taste. Such immediacy is expounded in similes such as ‘a firework in the mouth’, ‘a punch’, and ‘a flash that hits you’. Yet, despite the immediacy of this experience, it is short-lasting and decays rapidly. “When you drink it, you get that bit of a rush. Yes, it’s basically gone now [P15, sour]. In contrast, other tastes were described as slowly building up or maintaining consistent intensities (e.g., high for umami, and low for salty). Such intensities could be seen to be ‘lingering’, rather than ‘explosive’, as one participant described it: “You’ve got this “Whoa” sensation, feels quite strong to start with. Then it has gone super quick” [P19, sour]. While the dynamics of intensity imply variation (intensity increasing and decreasing), the vocabulary of movement animates these changes. Describing the bitter taste, one participant stated: “I guess it’s not sticky like the first one [umami]. It’s a bit lively… I feel like it’s moving around” [P15, bitter]. While certain movements can be attributed to mouth-feel (e.g., moving left to right across the tongue), others were externalized (e.g., “I feel it almost into my sinuses and into the rest of my face” [P14, bitter]). These expressions were not confined to the temporal characteristics of taste experiences, but already shed light on the bodily reactions that can be elicited by tastes. Movement was also invoked to describe stasis (e.g., ‘stays’) and repetitive movement (e.g., ‘waves’). “So it is kind of strong and it also stays. It doesn’t have a peak; it doesn’t go up and down; it just stays” [P2, umami]. Other tastes fluctuate rapidly: “Yes, ups and downs, but quite quick. They’re quite sudden crests and falls…” [P3, sour]. Participants often appealed to similes of physicality in order to explain their taste experiences (e.g., ‘round’, ‘soft’, ‘heavy’). Such physical experiences are tied to a synchronic perception of taste. In contrast, the diachronic physicality of taste experiences is given in the implied and experienced characteristics of taste as a residual presence (e.g., ‘lingering’, ‘stays there’): “It just stays in your mouth, so it kind of keeps developing” [P10, umami] or “it just leaves its mark in your mouth and doesn’t go” [P7, umami]. Such experiences are, much like the increasing intensities, those that ‘build up’, or ‘get a bit stronger”. Such presence is understood to ‘erode’. Moreover, the implied residual physicality is associated with experiences of absence. When tasting sourness, many participants described the immediate, almost physically imposing intensity followed by a marked absence. This absence is seen to draw the taster back into the taste, leaving them wanting more: “it creates an expectation of sweet flavour, like if you were biting into a slice of orange or something. … It’s gone now and actually I’d quite happily have another sip, to be honest” [P18, sour]. This residual physicality can also be seen to afford agency to taste experiences, where tastes ‘grab you’, and ‘hit you in the face’. As such, taste experiences can become reified in exerting influence over the taster. This can be achieved in the residual physicality or in absence, for instance, where the marked absence in sourness is seen as “a forward feeling… It has the feeling of tartness, your mouth moves forwards” [P14, sour]. Sweetness in contrast is associated with the feeling of filling the mouth, and when the taste is gone it leaves one with a kind of stickiness on the teeth. Figure 2 shows a pictorial representation of the different types of temporality identified based on the above descriptions across all five tastes. The intensity is represented through the thickness of the lines in the bars, while movement is captured through the frequency of the lines. Finally, residual physicality as temporal characteristic is shown through the length of the whole bar. Overall, sour is the taste delivering the highest intensity, followed by umami and bitter. Umami presents a high intensity, and is also characterized by lingering without losing much of its intensity. Such an extensive residual presence can also be seen for bitter, however with a lower intensity. Sweet and salty are also of low intensity and can be characterized by particular movements. While sweet starts slowly, builds up and then dies out, salty does not peak at all and is constant in its perception and moderate in unfolding over time. Sour, by contrast, is short-lived with a rapid end. Specific to sour is the sharp beginning followed by the absence of a taste and the return of it through a forward pulling feeling, which disappears quickly. Affective reactions Affective reactions refer to both the sense of pleasure or displeasure gained from the taste experience, but also feelings most often regarding familiarity, such as comfort, or, by contrast, unfamiliarity, such as surprise and suspicion. These affective characteristics, to be captured as pleasant-unpleasant and familiar-unfamiliar, operate not only as a static attitudinal response to taste experiences (synchronic structure), but also as evolving characteristics of the taste experience (diachronic structure). When sampling the taste stimuli many participants related their own uncertainty (e.g., I don’t know what to expect). After one sample, this uncertainty is replaced for familiar tastes. For unfamiliar tastes, particularly bitter and umami, the sense of unease pervades and persists. Thus familiarity produces responses at singular points (e.g., I am/am not familiar with this), while also producing responses across time (e.g., I know/do not know what to expect). A recurring phrase throughout the taste study was “I know what it is, but I don’t”. While we can at times attribute this to the nature of the stimuli as water solutions (i.e., those not regularly experienced by participants), the sentiment expressed also refers to the lived and felt experiences of the tastes. That is, while participants on the one hand had the taste ‘on the tip of their tongue’, those tastes also brought to mind a variety of known experiences, or, in the absence of known experiences, feelings of uncertainty or unease. Such feelings must presumably be associated with evolutionary causes (considering many bitter foods are poisonous) or in form of personal memories (e.g., salt, salty water, and the seaside) and cross-modal experiences (e.g., with color, or sounds). “If I drink or eat something that leaves that kind of trace, I always imagine a colour. Glowing…. It’s weird. I have no idea what this is, but there’s a bitterness that stays” [P2, bitter]. Participants identified as supertasters expressed their affective reaction more clearly: “Definitely bitterness… I don’t like it” [P8, bitter], or “It’s immediately bitter.… It’s like swallowing medicine” [P18, bitter]. There were few predictable or consistent affective reactions among participants, and those experienced as pleasurable by some, were experienced as disgusting or unsettling by others. The affective response of participants could often be tied to the participant’s familiarity with the taste. This was particularly noticeable with umami. Participants who were familiar with this taste indicated familiarity with savory Asian cuisine, and could therefore interpret the perceived taste and experienced it as pleasant. Those who did not eat Asian cuisine were less familiar with the taste, particularly in this intensity, and described unease and uncertainty when tasting it. Such responses also evolved over time, notably with sweet and sour tastes. While, as mentioned, sour produced an immediately unpleasant experience, followed by a refreshingly pleasant experience (e.g., “yes it probably gets more pleasant as the intensity of the taste dissipates” [P17, sour]), the taste of sweet was often initially pleasant, followed by a distinct unpleasantness. This unpleasantness could be so strongly felt as to produce nausea for some participants (e.g., “although it’s dying off over time. It’s quite sickly actually” [P20, sweet]). Such experiences were tied to the physicality of the taste residing in the mouth, and were perceived in two extremes for umami, influenced through the participants’ familiarity/unfamiliarity with this taste. Participants familiar with this taste perceived the mouth filling and lingering experiences as comforting (satisfaction after a full meal), while other participants who were unfamiliar with it perceived it as disgusting, obtrusive, and annoying referring to the fact that the taste takes over control, without the chance to get rid of it quickly. Figure 2. Temporal characteristics of taste experiences showing the intensity (thickness of the lines), the movement (frequency of the lines) and the residual physicality (length). Figure 3. Affective characteristics of taste experiences (green = pleasant, red = unpleasant, orange = neutral, white = absence of taste). Umami shows two experiences: pleasant and unpleasant. As with temporality, we created a representation of the different types affective reactions on the five tastes (see Figure 3). The pleasant-unpleasant characteristics of the taste experience are represented through the ‘green’ and ‘red’ colors and in cases of a neutral experience colored as ‘orange’, and finally ‘white’ in case of absence of the taste. The familiar-unfamiliar characteristics only find an explicit representation for the umami. The familiarity of the taste lead to its pleasant perception (upper bar for umami), while unfamiliarity with the taste was expressed through unpleasantness (lower bar for umami). Overall, some tastes are characterized by the change from unpleasant to pleasant (sour) or the other way around from pleasant to unpleasant (sweet), while the bitter taste was clearly unpleasant and salty was described as neutral. For umami, we identified two separate experiences (participants either love or hate it) grounded in the familiarity and unfamiliarity of the taste. Embodiment Although we would expect food experiences to involve embodied, textural, responses (such as ‘crunchy’, ‘slimy’), here each taste stimuli is experienced in the same form (i.e., as a colorless and odorless solution), and yet produce varied embodied responses. Embodiment in relation to the described diachronic and synchronic taste experiences refers to the mouth-feel of tastes (how something is felt in your mouth). Some participants additionally describe whole body reactions (reactions described beyond the mouth) and others refer to imagined and disembodied responses (resulting from the taste stimulation and its associations). Mouth-feel, referring to the experienced chemical and physical sensations in the mouth, is frequently used to describe different characteristics of foods, including coffee, wine, and textured foods. Such descriptions are offered by our participants for qualities of texture and viscosity. “It’s just like a softness, but I guess a little bit more viscosity even though I’m quite sure it doesn’t have any viscosity. It’s just sort of the feeling of viscosity, the sweetness and this cloud is just a bit more mouth feel” [P14, sweet]. The mouth-feel also relates to a sense of movement, where tastes evolve in space. Most often these are lateral movements within the mouth, or commonly tastes are felt to move backwards. Such experiences can be a feature of the physical movement of the taste stimuli during the swallow reflex and also associated with the location of taste receptors on the tongue. However, in other cases, taste experiences defied the location of taste receptors and tastes could be experienced on the teeth, gums, and lips. One participant goes as far as to describe the absence of mouthfeel: “I don’t know really. It leaves this numbness in my mouth like the lemon, but without the initial burst” [P9, sour]. In addition to the sensations described in mouth, some participants described bodily reactions that were opposed to the mouth-feel or isolated taste experiences. “I think the first part of it, the sour part, is a bit of a shock to the system. I don’t think you’re expecting it to be like that” [P16, sour]. Another participant said “I kind of see it from the moment it enters my mouth and goes down all the way to my stomach. It’s like I can see where it’s going” [P2, bitter]. In this sense, participants described tastes as producing expansive responses, including pleasure, nausea, and, others including reactions associated with allergy such as increased body heat (e.g., “If you eat it, it’s like your body – the heat just changes” [P2, umami]). Feelings of pleasure were often described as filling, particularly filling the face or the whole body. A participant describes it as such: “I feel that my whole face feels pleased with it” [P14, umami]. Such feelings were not always positive and for some participants, overwhelming feelings of nausea accompanied tastes of salt, umami, and sweet. Nausea could also be experienced in undulating taste experiences – those taste which were experienced as prone to fluctuations in intensity, almost mimicking travel or sea sickness. Participants also described disembodied reactions, which refer to something experienced that lingers between the mouth and the body. Rather than experiencing direct bodily reactions, participants describe an imagined reaction. “It’s like it’s there but it’s not there” [P2, salty]. Disembodied reactions could also be seen in expected or caricatured responses, such as the imagined ‘pucker face’ of the sour taste. Although few participants actually exhibit such a reaction, it is an ingrained image of biting into a lemon. “It feels a little bit, not uncomfortable, but it feels like it makes you kind of screw you face up a bit” [P11, sour]. Shapes assigned to the overall taste experiences The usage of the SEI objects (see Figure 4) as a physical engagement with shapes enriched the description of the taste experiences. The shapes particularly contributed descriptors related to the combined temporal, affective, and embodied experiences of each individual taste. Below, we summarize the key characteristics and the mapping of the eight objects for each taste across all participants. The sweet taste, generally experienced as ‘smooth’ and ‘rounded’, was most reflected in shapes that present elements of change such as “phases” (shapes 4 and 5) or have protruding elements (like 7 and 1, or even the halfspiky shape 2). While typically a pleasant taste, there is a dynamic modulation of intensity and pleasure in the shapes. The sour taste produces a ‘sharp’ response and for many is best characterized by shapes such as 8 or 2. There are, however, also elements of temporality, a shifting/phasing associated with shape 4, starting with the big part as an explosion and then rapidly decaying. The salty taste has a broad aspect (mapped towards 3 and 6) and a finer Figure 4. SEI – Sensual Evaluation Instrument consisting of 8 objects with different shapes by Isbister et al. [13]. granulated and dynamic experience expressed through the shape 1. Similar to sweetness and sourness there is a repeating wave assigned to this taste experience, verbalized around shapes 4 and 5 though this time associated with an unpleasant feeling/sickliness as aftertaste. More than other tastes, salty was associated with a subtlety of the temporal characteristics, an experience of something moving, not doing much, but still being there. This made participants want a shape that they could manipulate (“These [objects] are kind of too permanent; you’re not able to manipulate them” [P6]) or something more neutral, such as a flat shape, or a shape, which can be changed. Despite the fact that the bitter taste was experienced as unpleasant, the mapping to the shapes created two distinct experiences. For some participants, bitter is a spiky but lingering experience associated with a dull unpleasantness (1, 2, and 7 shapes selected). For others it is a rounded and smooth taste (these participants chose shapes 5 and 6), associating it with medicine (form of pills), which dissolves in the mouth, and you cannot get rid of it. Similarly to bitter, the mapping for umami resulted in two distinct experiences. If umami was experienced as unpleasant, participants tended to describe the taste as disgusting and chose the shape 8 or 2. In those cases where umami was perceived as pleasant, participants described it as a more rounded taste with depth and chose combinations of the rounded shapes (such as 3 and 1 shape were used most, and combined with either the 5, 4, or/and 7 shape). This mapping confirms the descriptions of umami as a full, mouth-filling experience with lots of things to it. Overall, sweet and sour seem to be the two tastes where participants show high agreement with respect to mapping the shapes to taste experiences. Bitter and umami seem to share some associations and create two different mappings between shapes and taste experiences, while salty shows a tendency towards smooth and round shapes, but with the lack of the ability to change and manipulate the shapes. Combined representation of the taste experiences Figure 5 shows the final pictorial representation of all three characteristics combined for each of the five tastes. The length of the forms represents the temporal aspects, while the width captures the mouth-feel. Whole body and imagined embodiment could not be captured as such, but are described in detail above. The expression ‘lingering’ was used particularly for sweet, bitter, and umami. When used for sweet and umami ‘lingering’ is experienced in combination with a ‘mouth filling’ element (it is filling the whole mouth), while for bitter there is no filling experience but it is described as a thin (straight through your mouth to the back) experience, next to being unpleasant. In the bitter case, ‘lingering’ thus refers to the residual physicality of this taste (in the back of your mouth). Sour has an initial unpleasant taste, dies down quickly, but comes back after a short absence and leaves one with the feeling of wanting more. Salty at last is similar to bitter, however with a shorter life and perceived as less unpleasant. Salty is perceived as a neutral taste with little consequence. DISCUSSION AND FUTURE RESEARCH While sensory researchers and neuroscientists study the perception of taste and its temporality, their focus is on quantifying the intensity and perceived changes of intensity via a wide range of evaluation scales [26] or, in some recent attempts, by means of time-intensity profiles of fMRI data [19]. Our findings add a semantic level of understanding underlying the taste experiences, their temporality enhanced through descriptions of the affective reactions and embodiment that the five basic tastes provoke. This understanding may be useful when designing for taste experiences as it provides designers and developers a vocabulary to talk about taste and the design potentials related to the different characteristics. First, we discuss the particularities of each taste quality, and then discuss them with respect to established psychological and behavioral phenomenon highlighting their design potential for HCI. How is taste experienced? Here we discuss the specific experiences each of the five basic taste qualities create and can inspire design in HCI. Sweet: Pleasant but with a bittersweet ending The sweet taste was consistently described as pleasant, which turned into something unpleasant. Participants struggled between the instinctive taste likeability and the learned taste values and rules (sweet is bad for the teeth), which can be seen in light of learned associations, discussed by Schifferstein and Hekkert [32] with respect to taste experiences of products. Of particular interest with respect to our findings on crossmodal interactions for sweet stimulations are the embodied reactions (e.g., “It’s just sort of the feeling of viscosity, the sweetness and this cloud is just a bit more mouth feel” [P14]). Such reactions can be explained through learned associations with sweetened food and beverages. It is a combination of learned as well as innate, genetic, and cognitive factors [32]. Sweet sensations can be used to stimulate and enhance positive experiences, however, on a limited timescale, as the sweetness is quickly disappearing leaving one unsatisfied. It’s a pleasant taste but one that is tinged with a bittersweet ending. Figure 5. All taste characteristics combined: the temporality shown through its length; affective reactions through the color (green pleasant, red unpleasant, orange neutral experience); and the embodiment through its form (mouth feeling). Sour: Unpleasant at first, but with the need for more In contrast to the sweet taste, the sour taste is described as short-lived and it often comes as a surprise due to its explosive and punchy character. This taste overwhelms one with its rapid appearance and quick decay. It leaves one with the feeling that there is something missing. Based on childhood memories, such as for instance of sweet-sour drops, participants were expecting sweetness, but were left disappointed, leaving them with the feeling of wanting more. This phenomena was also observed in the evaluation of a gustatory gaming interface with children, where sour was used for negative reinforcement linked to the game dynamics [17]. Children intentionally failed in the game in order to get another sour stimulation. Salty: Not doing much The salty taste experience was not linked to an extreme reaction unlike sour, bitter, and umami. This taste is often described as ‘bland’, ‘discrete’, and ‘just being there and not doing much’. It is minutely moving around, giving the feeling of cleansing the mouth, but not being mouth filling as sweet or umami, and certainty not as unpleasant as bitter, however lingering almost as long as the bitter taste. The modesty of saltiness in contrast to all of the other tastes opens up some interesting questions when looking at the neuroscience findings. Nakamura’s [19] findings based on time-intensity fMRI profiles suggest that salty tastes change more rapidly than do sweet tastes. This is not quite consistent with how our participants described their experiences and needs further studies. Bitter: Unpleasant, not to be experienced again The perceived intensity of the bitter taste was not the same for everyone, as confirmed by the supertaster test. While supertasters felt the experience with more immediacy, others had to allow the taste to travel to the back of their mouth to recognize it. After this initial difference, the bitter experience becomes consistent with respect to its ‘lingering’ features, of ‘staying’ either on the tongue or at the back of the mouth. Bitter was also described as ‘thin’. The character of bitter was further revealed through learned associations referring to ‘biting into a flower’, or ‘medicine’, things you had to take as a child, but after which you would rather avoid this experience of bitterness. Bitterness can indicate the presence of toxins [32] and is found in evolutionary development of humans (e.g., feeling of suspicion regarding bitter food as poisonous) [7]. It may be useful for design to make people avoid certain behaviors. Umami: Like/dislike, but still confusing as a taste The familiar-unfamiliar characteristics of umami caused much confusion in our study and participants could not rely on their intuition. While the ‘like’ or ‘dislike’ of the taste was decided instantly, the unpicking of the still ‘confusing’ elements of the umami taste was more challenging. Different word pairs depending on the like/dislike of the taste were expressed: ‘pleasant–unpleasant’, ‘comforting– uncomforting’, and ‘liking–disgusting’. We could also see participants using additional bodily descriptors, in particular when describing umami as a pleasing experience (‘face feels pleased’ or ‘body heat changes’). In cases of dislike, the focus of attention in the verbalizations was the lingering characteristic of the taste founded in the inability to get rid of it. In these cases, the residual physicality can be seen to afford agency. The taste experience becomes reified in the influence it exerts over the taster. Depending on personal familiarity/unfamiliarity (which may be defined by cultural factors) and personal preferences, this taste experience is quite interesting for design. Umami grabs one’s attention and initiates a conscious process of reflection. While judgment on the taste is defined quickly, the reflective thinking brings to the fore the richness and variety of the taste. Even when perceived as unpleasant, the richness is recognized, and linked to the motivation to remove the taste from the mouth. How can we design with taste experiences? Taste experiences can be discussed with respect to their relevance for design, building on existing psychological and behavioral phenomenon: rational and intuitive thinking, anchoring effects, and behavior change. The dual process theory [14,37], for instance, accounts for two styles of processing: the intuition based System 1 with associative reasoning that is fast and automatic with strong emotional bonds, and reasoning based on System 2 which is slower and more volatile, being influenced by conscious judgments and attitudes. Based on our findings, we can see that sweet is intuitively perceived as pleasant, and bitter as unpleasant, while sour, salty, and umami cause a reflective process, confused, for instance, by the surprise appearance and rapid disappearance of the sour taste. Our findings also give insights into how to time the presentation of the taste qualities so that the user can transition from System 1 thinking to System 2 thinking. Figures 2, 3, and 5 can be used to create the appropriate transitions and time them. For example, the rapidity of the sour taste experience does not leave enough time for System 1 to engage with it and triggers System 2 to reflect on what just happened. Such reactions when carefully timed can prime users to be more reason based in their thinking during a productivity task (e.g., to awaken someone who may be stuck in a loop). Moreover, an appropriately presented taste can create a synchronic experience that can lead to stronger cognitive ease (to make intuitive decisions) or reduce the cognitive ease to encourage rational thinking. For example, a pleasant taste can be used to provide achievements across the workflow, however with the slight hint that there are still more tasks to do before you are finished (e.g., the slight unpleasant aftertaste of sweetness). Below, we outline potential design directions for using taste experiences in work-related activities and for personal behavior management. Doing so, we draw on the potential of different taste qualities and their power to stimulate intuitive and rational thinking described above. Managing anchoring effects through taste A common aspect of everyday activity is interruption. We are often interrupted by emails, telephone calls, or other unanticipated events. These interruptions can either be short (e.g., a quick glance at an email pop-up) or slightly longer requiring us to change our activity (e.g., a line-manager walking into your office to ask for something). All these activities have anchoring effects. In other words, the initial activity affects our judgments and decision making in the latter activities. It has also been shown that users often find it hard to avoid these biases in their judgments [38]. Our study of taste experiences suggests that taste interfaces can be carefully designed to manage interruptions in such a way that anchoring effects can be either minimized or maintained. For example, we know that the salty taste has a long temporal component with a feeling of “not doing much but being there”. This taste could be very useful in those situations where the interruption is small and the user is expected to return to the initial activity soon. As an example, when the user notices a pop-up in the bottom left corner of their desktop (for email or other social media interruptions) a small salty taste in their mouth which starts just before the user switches their activity can be useful. This will prolong their initial experience and remind them of the initial activity when still checking the social media page. This could enable smoother transitions back to the initial activity. Alternatively, however, if the interruption is a longer activity then it is useful for the user to drop any priming effect that might transfer to the new activity. In this case, a sour taste in the mouth would leave the user a quick sharp taste engaging their rational System 2 but rapidly decaying helping the user return to a more neutral state by the time they switch to the new activity. Such management of anchoring effects is not only useful for productivity activities but also in other activities, such as gaming. For example, LOLLio – the taste-based game device described above [16], currently uses sweet and sour for positive and negative stimulation during the game play. We suggest that such a game could be improved based on our framework by providing fine-grained insights regarding the specific characteristics of taste experiences, which can be integrated into the game play. When a person moves between related levels of a game a continuing taste like bitter or salty is useful. Whereas when a user is moving to distinct levels or is performing a side challenge an explosive taste like sour, sweet, or umami might be useful. The choice of specific tastes in each category can be tuned by the designer to create different affective reactions and a sense of agency. Priming positive behavior through taste Taste and taste preferences play an important role in our food choices [24] and food plays a significant role in our health and wellbeing. The stimulation and manipulation of taste experiences therefore offers potential to improve a variety of food behaviors. Using taste stimulation technology to alter the taste of unpleasant but healthy food is one obvious route. Expanding the design space for healthy taste technology, our framework suggests alternative routes. Taste experiences might be heightened through appeal to related experiences and sensations. Morphing physical objects, such as recently suggested shape-changing devices [29], might also be used to replicate the embodied expansiveness of the umami taste to stimulate an increased taste experience for patients receiving chemotherapy who may suffer from hypogeusia, a decrease in taste sensitivity. Taste stimulation might also facilitate sustainable food practices, for instance, linking food waste to taste experiences. Taste stimuli might thus supplement other post-actional cues in the effective disturbance of food waste habits and promote critical reflection. When disposing overripe bananas, a user might get a sour stimulation for the waste of food but the immediate reward for waste separation. Taste stimulation might also reflect various characteristics of food waste, such as its lengthy impact on environmental sustainability through the bitter taste. In this way, the framework for design points to the potential for taste experiences to be incorporated into timely and rewarding persuasive messages for positive food behaviour. CONCLUSIONS In this paper we presented the results of a user study exploring the experiential characteristics for each of the five basic taste qualities. Our analysis of participants’ verbalizations, collected by means of verbal and non-verbal methods, resulted in three key themes. We provide rich descriptions on the temporality, affective reactions, and embodiment of taste experiences. We discuss these themes for each individual taste elucidating the design potentials with respect to the specific structure and qualities of sweet, sour, salty, bitter, and umami tastes. Our findings help to establish a framework for the design of taste experiences in HCI, enhancing existing technology driven research around taste, and food interaction design research. Although we do not provide guidance for the design of a specific interactive system in this paper, we are convinced that our framework provides a starting point for designers and developers to think about design/development potentials for taste in HCI. ACKNOWLEDGMENTS This work is supported by the EU Marie Curie Action (FP7- PEOPLE-2010-IEF) and RCUK SiDE (EP/G066019/1). We wish to thank our study participants and Annika Haas for the audio-visual support, as well as Katerhine Isbister and Kristina Höök for providing us with a set of the SEI objects. REFERENCES 1. Bartoshuk, L.M., Duffy, V.B., Fast, K., Green, B.G, Prutkin, J., Snyder, D.J. Labeled scales (e.g., category, Likert, VAS) and invalid across-group comparisons: What we have learned from genetic variation in taste, Food Qual Pref, 14(2), (2003), 125-138. 2. Bartoshuk, L.M. Comparing sensory experiences across individuals: Recent psychophysical advances illuminate genetic variation in taste perception. Chem. Senses, 25(4), (2000) 447-460. 3. Chen, J., Engelen, L. (Eds.). Food oral processing: Fundamentals of eating and sensory perception. Wiley- Blackwell: Oxford, UK (2012). 4. Comber, R., Ganglbauer, E., Choi, et al. Food and interaction design: designing for food in everyday life. CHI EA (2012), 2767-2770. 5. Desmet, P.M.A., Schifferstein, H.N.J. Sources of positive and negative emotions in food experience. Appetite, 50(2-3), (2008), 290-301. 6. Drewnowski, A., Kristal, A., Cohen, J. Genetic taste responses to 6-n-propylthiouracil among adults: A screening tool for epidemiological studies. Chem. Senses 26(5), (2001), 483-489. 7. Glendinning J. Is the bitter rejection response always adaptive? Physiol. Beh., 56, (1994), 1217-1227. 8. Green, B.G., et al. Evaluating the ‘Labeled Magnitude Scale’ for measuring sensations of taste and smell. Chem. Senses, 21(3), (1996), 323-334. 9. Grimes, A., Harper, R. Celebratory technology: new directions for food research in HCI. CHI (2008), 467- 476. 10. Hoehl, K., Schoenberger, G.U., Busch-Stockfisch, M. Water quality and taste sensitivity for basic tastes and metallic sensation. Food Qual Pref 21(2), (2010), 243- 249. 11. Hupfeld, A., Rodden, T. Laying the table for HCI: uncovering ecologies of domestic food consumption. CHI (2012), 119-128. 12. Humphries, C. Delicious science. Chefs are teaming up with researchers to create avantgarde dishes. Is ‘molecular gastronomy’ more than a fad? Nature, 486 (2012), 10-11. 13. Isbister, K., Höök, K., Sharp, M., Laaksolahti, J. The sensual evaluation instrument: Developing an affective evaluation tool. CHI (2006), 1163-1172. 14. Kahneman, D. A perspective on judgement and choice. American Psychologist, 58(9), (2003), 697-720. 15. Light, A. Adding method to meaning: A Technique for exploring peoples’ experiences with technology. Beh. and Information Technology, 25(6) (2006), 175-187. 16. Murer, M., Aslan, I., Tscheligi, M. LOLLio: Exploring taste as playful modality. In Proc. TEI 2013, 299-302. 17. Moser, C., Tscheligi, M. Playful taste interaction. IDC (2013), 340-343. 18. Narumi, T., Nishizaka, S., Kajinami, T., Tanikawa, T., Hirose, M. Augmented reality flavors: gustatory display based on edible marker and cross-modal interaction. CHI (2011), 93-102. 19. Nakamura, Y., Goto, T.K., Tokumori, K., et al. The temporal change in the cortical activations due to salty and sweet tastes in humans: fMRI and time-intensity sensory evaluation. Neurorep, 23(6), (2012), 400-404. 20. Ngo, M.K., Velasco, C., Salgado, A., et al. Assessing crossmodal corresponddences in exotic fruit juices: The case of shape and sound symbolism. Food Qual Pref 28, (2013), 361-369. 21. Núnez-Jaramillo, L., Ramirez-Lugo, L., Herrera- Morales, W., Miranda, M.I. Taste memory formation: Latest advances and challenges. Behav Brain Res, 207(2), (2010), 232-248. 22. Maynes-Aminzade, D. Edible bits: Seamless interfaces between people, data and food. CHI EA (2005), 2207- 2210. 23. Obrist, M., Seah, S. A., Subramanian, S. Talking about tactile experiences. CHI (2013), 1659-1668. 24. Palmer, S. E., Schloss, K. B. An ecological valence theory of human color preference. Proc. of the National Academy of Sciences, 107(19) (2010), 8877-8882. 25. Petitmengin, C. Describing one’s subjective experience in the second person. An interview method for the science of consciousness. Phen. Cog. Sci., 5(3-4), (2006), 229-269. 26. Pineau, N., Schlich, P., Cordelle, S., et al. Temporal dominance of sensations: Construction of the TDS curves and comparison with time-intensity. Food Qual Pref, 20(6), (2009), 450-455. 27. Ranasinghe, N., Cheok, A.D., Nakatsu, R. Taste/IP: The sensation of taste for digital communication. ICMI (2012), 409-416. 28. Ranasinghe, N., Karunanayaka, K., Cheok, A.D., Fernando, O.N.N., Nii, H., Gopalakrishnakone, P. Digital taste and smell communication. BodyNets (2011), 78-84. 29. Roudaut, A., Karnik, A., Löchtefeld, M. et al. Morphees: toward high “shape resolution” in selfactuated flexible mobile devices. CHI (2013), 593-602. 30. Savage, N. Technology: The taste of things to come. Nature 486(21), (2012), 18–19. 31. Schifferstein, H.N.J., Fenko, A., Desmet, et al. Influence of package design on the dynamics of multisensory and emotional food exp. Food Qual Pref, 27 (2013), 18-25. 32. Schifferstein, H.N.J. Hekkert, P. Product experience. USA: Elsevier (2007). 33. Smets, G.J.F., Overbeeke, C.J. Expressing tastes in packages. Design Studies, 16(3), (1995), 349-365. 34. Spence, C. Crossmodal correspondences: A tutorial review. Atten Percept Psychophys 73, (2011), 971-995. 35. Spence, C., Piqueras-Fiszman, B. Technology at the dining table. Flavour, (2013), 2-16. 36. Spence, C., Smith, B., Auvray, M. Confusing tastes and flavours. In M. Matthen and D. Stokes (Eds.). The senses. Oxford: University Press. (in press, 2014). 37. Stanovich, K.E., West, R.F. Individual difference in reasoning: implications for the rationality debate?. Behav. Brain Sciences 23(5), (2000), 645–665. 38. Strack, F., Mussweiler, T. Explaining the enigmatic anchoring effect: Mechanisms of selective accessibility. Pers. and Social Psy. 73(3), (1997), 437-446. 39. Tosey, P., Mathison, J. Exploring inner landscapes through psychophenomenology. Qual. Research in Organiz. And Management: Inter. J., 5(1) (2010), 63-82. 40. Trivedi, B.P. Hardwired for taste. Research into human taste receptors extends beyond the tongue to some unexpected places. Nature, 486, (2012), 7-9. 41. Vermersch P. L’entretien d’explicitation [translated as Explicitation interview], ESF (1994). 42. Wakeling, I.N., MacFie, H.J. Designing consumer trials balanced for first and higher orders of carry-over effect when only a subset of k samples from t may be tested. Food Qual Pref, 6(4), (1995), 299-308.

Innovation Framework Illustration – Obrist et al. Temporal, Affective, and Embodied Characteristics of Taste Experiences: A Framework for Design 150 150 John

Innovation Framework Illustration – Obrist et al. Temporal, Affective, and Embodied Characteristics of Taste Experiences: A Framework for Design

Temporal, Affective, and Embodied Characteristics of

Taste Experiences: A Framework for Design

Marianna Obrist1,2, Rob Comber1, Sriram Subramanian3, Betina Piqueras-Fiszman4, Carlos Velasco4, Charles Spence4

m.obrist@sussex.ac.uk | rob.comber@ncl.ac.uk | sriram@cs.bris.ac.uk | betina.piqueras-fiszman@psy.ox.ac.uk | carlos.velasco@psy.ox.ac.uk | charles.spence@psy.ox.ac.uk

1Culture Lab, School of Computing Science Newcastle University, UK; 2 School of Engineering and Informatics University of Sussex, UK; 3 Department of Computer Science, University of Bristol, UK; 4 Department of Experimental Psychology, University of Oxford, UK

ABSTRACT

We present rich descriptions of taste experience through an analysis of the diachronic and synchronic experiences of each of the five basic taste qualities: sweet, sour, salt, bitter, and umami. Our findings, based on a combination of user experience evaluation techniques highlight three main themes: temporality, affective reactions, and embodiment.

Comment 1

The research reports data (that is, newly acquired knowledge), identifying three main themes, characterising user taste experience –  temporality, affective reactions, and embodiment.

We present the taste characteristics as a framework for design and discuss each taste in order to elucidate the design qualities of individual taste experiences.

Comment 2

The research proposes the three themes of temporality, affective reactions and embodiment as a framework for design.

These findings add a semantic understanding of taste experiences, their temporality enhanced through descriptions of the affective reactions and embodiment that the five basic tastes elicit.

Comment 3

The research enhances our understanding of taste experiences.

These findings are discussed on the basis of established psychological and behavioral phenomena, highlighting the potential for taste-enhanced design.

Comment 4

The research claims potential for taste-enhanced design.

Author Keywords

Taste; user experience; taste experiences; sensory research; explicitation interview technique; sensual evaluation tool.

ACM Classification Keywords

H.5.2 Information interfaces and presentation (e.g., HCI):

Miscellaneous.

INTRODUCTION

Experts in taste perception agree on at least five basic tastes [40]. Beyond this, however, we lack insights into the rich experience of these tastes. This lack of experiential understanding extends beyond HCI, as sensory researchers have also acknowledged that: What is not well researched is the link between the food that goes into our mouth and what we think of it [12]. There is a growing interest in taste within the HCI community [e.g., 16,17,18,22,27,28], particularly relating to technical challenges in designing for taste stimulation and one-off designs to enhance user experiences through the manipulation of taste.

There is a need for a more systematic study of people’s taste experiences and their specific characteristics in order to make a fuller use of this sense in future taste-enhanced technologies. This paper stands as a first step in addressing this need. Drawing on neuroscience and sensory research in combination with user experience evaluation techniques, we investigated how all five basic tastes are experienced at a given time (synchronic) and how they evolve over time (diachronic). We used pure tastants (i.e., that have no smell or visual qualities) with an explicitation interview technique [41] designed to encourage the participants to verbalize their experiences. Additionally, we used physical objects from the Sensual Evaluation Instrument [13] to elicit affective responses, and create a flexible, non-verbal channel of communication between the user and designers.

This paper makes a number of contributions: First, we provide a rich description of subjective taste experiences along both the diachronic and synchronic characteristics of the five basic tastes. Second, these taste characteristics establish a framework for taste experiences and elucidate the potential design qualities of individual tastes. We demonstrate how each quality can be described along three main themes: temporality, affective reactions, and embodiment. Third, our findings extend human-computer interaction research on taste through a user experience perspective. Overall, our findings provide interaction designers and user experience researchers with a richer understanding of taste experiences and their specific power to influence human behavior and decision-making. The framework presented here enables the HCI community to think and talk about taste in the design of interactive systems in a fine-grained manner.

RELATED WORK

This section provides an overview of the human sense of taste and its relevance for HCI based on ongoing research.

The Sense of Taste

Sensory researchers and neuroscientists agree on five basic tastes (sweet, sour, salty, bitter and umami), and a ‘gustotopic map’ linking these classes of receptors with particular brain areas is currently being developed [40]. However, despite breakthroughs in understanding the sense of taste, scientists have still not approached the phenomenology of taste nor developed a semantic understanding of how taste is experienced [30]. Although a wide body of sensory research has studied the temporal evolution of taste perception using labeled intensity scales [e.g., 1,8] and more specific time-intensity sensory evaluation scales [26], insights are limited to the quantification of temporal responses to perceived taste intensities. Such scale-based evaluations leave us uninformed as to the subjective qualities that lie behind the ratings of the perceived taste experience over time.

Recently, neuroscientists have studied taste-specific temporal profiles by comparing sensory evaluation scales with functional MRI (fMRI) data [19]. Their results suggested that salty tastes change more rapidly than sweet tastes in the cerebral cortex, and confirm the same patterns that have been observed using time–intensity sensory evaluation [19]. While such results are intriguing, they cannot explain the differences in experienced tastes. To account for subjective differences, the ‘taster status’ measure has been introduced [2,6]. By means of such tests, it is possible to identify participants’ subjective sensitivity to bitter tastes and to distinguish between supertasters (25% of population), medium tasters (50%), and non-tasters (25%) [3]. Taster status has been considered to partially explain why some consumers like certain foods more than others and how they describe the way they experience them.

Food-interaction Design

The last few years have seen increasing interest in designing human-food interaction in HCI [e.g., 4,9,11,33]. Such research looks to position human-food interaction within the wider spectrum of social, environmental, and physiological influences on our food practices. In this area, there is a growing realization of the potential for new technologies to support pleasurable experiences around food [20,35], and the potential for designers to draw on the extensive research on multisensory experiences (i.e., auditory, tactile, visual, olfactory, and gustatory). Despite this increased interest in food experience, we know little about the richness of people’s taste experiences. The majority of the studies on food experience combine taste with other modalities, where taste is but one component [e.g., 18,27,28]. For instance, Schifferstein et al. [31] elicited emotional experiences across the different stages of food product usage, from choosing a product in the supermarket through to cooking and eating [31]. Taste experience is interwoven with vision, touch, and olfaction, which, in combination create multisensory food experiences. Desmet and Schifferstein [5] also explored the emotions elicited through eating and tasting food. They describe variables related to food-evoked emotions, such as sensory features, product type, food-related activities, context, and the agent (who consumes, prepares, or produces). Due to the wide range of influencing variables, it is not clear how well these findings translate beyond the specific context of their studies.

Taste-enhanced Technology

Technological advances in creating taste stimulations [27,28] and one-off applications exploiting taste in games [16] and other scenarios [18,22] demonstrate a growing interest in the use of taste in interactive applications. For instance, Ranasinghe et al. [27,28] developed a tongue interface that creates taste through the combination of electrical and thermal stimulation. They use electrical pulses applied to the tongue. Verbal descriptors provided by participants were, for instance, a ‘refreshing taste’ or ‘minty taste’ in relation to the change in temperature. The authors call for future work to understand the particularities of such taste (flavor) experiences. They focused on the introduction of taste in digital communication to enhance long-distance family relations and create remote co-presence and coliving experiences (e.g., remote dining) [28].

Murer et al. [16] designed a gustatory game device, LOLLio, which consists of an interactive lollipop that serves as a haptic input device that dynamically changes its taste between sweet and sour. Remote triggering of taste while motion sensing with accelerometers allows LOLLio to be used as an input modality. The authors identify various ways in which taste could be used in an interaction, such as to provide reward or punishment or else to provide hidden information through taste stimuli. LOLLio was evaluated in a game context with children [17]. Sweetness was constantly used in the game session and sour stimuli were used in combination with game mechanics to provide ‘negative reinforcement’. Their findings suggest an enhanced playing experience through taste stimulation motivating further explorations of such taste-enhanced interaction experiences.

STUDY METHOD AND PROCEDURE

Sensory research provides important information regarding the objective measures of taste perception, temporality, and subjective sensitivity levels. Yet, an understanding of the subjective understanding of taste experiences is missing. This study explores the diachronic and synchronic structure (explained below) of each of the five basic tastes.

Methodology

For our study, we combine two verbal and non-verbal user experience and elicitation methods, the explicitation interview technique (verbal method) and the ‘Sensual Evaluation Instrument’ (non-verbal method). The explicitation interview technique [41] is used to elicit verbalizations of subjective experiences. This technique helps to explore the unfolding of an experience over time, the ‘diachronic’ dimension, and examines the specific facets of the experience at a particular moment, the ‘synchronic’ structure (see also [24,39]). The value of this interview technique lies in helping participants to express their experiences at a specific moment. Participants are encouraged to talk about the experiential (cognitive, perceptive, sensory, and affective) aspects of the moment without building on rational comments and explanations [24].

Questions related to the diachronic structure help to understand how the description of an experience unfolds over time (e.g. “What happened after you opened the door?” and “What did you perceive next?”). With respect to the synchronic structure of an experience, the participant is questioned about a particular moment (e.g. “At the moment when you pushed the handle down, how did it feel?” or “What else came in your mind?”). In comparison to open questioning approaches, this technique is non-inducive but directive [24] in the sense that it keeps the participant talking about the experience without inducing any content; it focuses on the structure of the experience, and directive, as it keeps the participant focused on the singular experience being explored. Although it is typically used retrospectively to support the reconstruction of an experience, it has also been used in-situ (e.g., [15,23]).

The Sensual Evaluation Instrument (SEI) is a non-verbal tool that can be used to elicit users’ affective reactions [13]. SEI is composed of sculpted objects that can be held in the hand, used by a person to indicate how they are feeling as they interact with a system. The SEI includes eight objects with different shapes, which represent various levels of arousal and valence (positive and negative). Isbister et al. [13] describe SEI objects as evoking and expressing a range of emotions; they do not claim a direct mapping between the objects and the mentioned emotions, but emphasize the benefit of the objects for stimulating expressiveness. The value of the SEI is to elicit real-time, affective responses, and to create a flexible, non-verbal channel of communication between user and designers. The latter defines a key advantage compared to other methods that are often limited to verbalizations or visualizations that lack physicality.

Taste stimuli

The stimuli used for each taste are specified in Table 1. Each stimulus was prepared as an odorless and colorless water solution using a stock solution as specified in ISO 3972. We prepared the solutions according to the specifications detailed by Hoehl et al. [10] and used deionized water for the tastants. These compounds standardised stimulus features and controlled for sensory differences, such as texture, vision, etc. All of the solutions were prepared the day before each study day. The participants received 20 ml of each stimulus in a disposable 40 ml cup. A Latin square design was used to avoid order bias [42].

Table 1. Stimuli used for the five main tastes, including the stock solution (indicating the threshold specified in ISO 3972).

Participants

The study was conducted with 20 participants (nine female) aged between 21-38 years (M=29.4, SD=5). Participants were recruited based on the following criteria: not having any food allergies, being non-smokers, not being pregnant, and not having any sensory dysfunction (e.g., dysguesia, a taste disorder), by self-report. The participants were recruited through the staff list within the lead university. 16 participants were native English speakers, and the remaining four were fluent in English. All participants gave informed consent prior to the study.

Study set up and procedure

The participants were instructed and reminded 2 days prior

to the study not to eat spicy food 24 hours before the study

and not to drink or eat 1 hour before attending the study.

The study had 2 parts (see Figure 1): In the first part, we

applied the explicitation interview technique for all five

tastes; in the second part we introduced the SEI objects to

enhance the verbalizations for each taste.

In the first part, participants were given 5 minutes per

stimulus. They could take as many sips as they wanted of

the stimulus and were prompted with specific questions

about their taste experience (e.g., Could you describe what

you perceive? How does it feel in your mouth?). The aim

was to receive insights regarding the diachronic and

synchronic structure of the taste experience. We used this

technique in-situ in order to account for the rapidly

decaying sensory memory trace related to the human sense

of taste [21]. Before continuing with the next stimulus, the

participants were asked to have a sip of the deionized water

in order to cleanse their mouth. The same procedure was

repeated for all stimuli.

In the second part of the study, the participants were

instructed to match each taste experience to one or more of

the eight shapes inside the box. The participants could only

feel, and not see, the objects, to exclude any visual

influences and to focus on the mapping between ‘taste and

shape’ via the sense of touch. The participants were

instructed to select one or more or none of the shapes (they

could also reuse shapes for different tastes). Before going

through each taste stimulus again, the participants were

given the chance to put their hands into the box and

familiarize themselves with the 8 shapes.

Next they were asked to take a sip of water and start with

the first taste stimulus. They were asked to express the

thoughts they had in mind and to describe their choices or

lack thereof (if none of the shapes was selected). Finally,

the participants were asked to rate the pleasantness/

unpleasantness of the shapes on a four-point Likert scale

from ‘very pleasant’ to ‘very unpleasant’. They were also

asked about their personal favorites amongst the 5 taste

stimuli and their personal food preferences to support the

interpretation of the data.

In a final step, we tested the participants for their taster

status, which classified participants into supertaster, normal

tasters, and non-tasters. Overall, the study lasted one hour

and was audio/video recorded with the consent of the

participants. No incentives were paid to the participants.

Data analysis

All 20 tasting sessions were transcribed and a qualitative

analysis based on the transcripts was conducted. Two

researchers independently performed an open thematic

coding based on 5 cases (25%). The resulting themes were

discussed and an initial coding scheme was established.

Two more cases (10%) were coded independently leading

to a final coding scheme consisting of three main themes

(described in the next section), which were then applied to

the remaining 13 cases by both researchers. We also

performed a qualitative analysis of the mapping between

the SEI objects (see Figure 4) and the taste experiences,

captured through the transcripts and the visual material

from the recorded hand movements in the second part of the

study. Based on participants’ ratings of the shapes (their

physical pleasantness/unpleasantness) we could confirm

previous ratings of Isbister et al. [13] – the more spiky

shapes were rated as ‘unpleasant to slightly unpleasant’

(shapes 8,7,2), the more rounded shapes were rated ‘very

pleasant to pleasant’ (shapes 3,4,5,6), and only one shape

was perceived as ‘neutral’ (shape 1). Finally, the supertaster

test provided us with insights on the different taste

sensibility of participants and ensures a good distribution of

taster statuses in our study. Overall, we identified 5 nontasters,

11 normal taster (4 tending towards the upper edge

of bitterness sensitivity), and 4 supertasters. These results

are consistent with the known distribution amongst the

general population [3].

STUDY FINDINGS

The description of taste experiences is based on both parts

of the study. We describe the characteristics of taste

experiences across all five tastes along three identified

themes: (1) temporality, (2) affective reactions, and (3)

embodiment (see overview in the supplementary material).

We also discuss the particularities of each individual taste

in order to elucidate the potential design qualities of single

tastes. Each identified theme is represented in a pictorial

visualization of its key characteristics based on the

identified patterns across participants’ verbalizations.

Temporality

While taste experiences have expected elements of

changing intensity (e.g., strong taste, weak taste), the tastes

were also perceived as being mobile (e.g., moving within

the mouth, moving intensities), and occasionally exerted a

physical presence (e.g., building up, eroding, lingering).

These temporal characteristics are intertwined in the

unfolding of the experiences from its initial stimulation

(diachronic structure) and set the stage for the different

taste journeys (synchronic structure). Below, we describe

the different time-intensity profiles of taste experiences.

Taste intensities are generally experienced as being

dynamic and participants’ verbalizations offer a lexicon of

growth and decline. The diachronic nature of taste

experience is also revealed in the immediacy or longevity

of dynamic intensities. For instance, all participants agree

on the immediacy of the sour taste. Such immediacy is

expounded in similes such as ‘a firework in the mouth’, ‘a

punch’, and ‘a flash that hits you’. Yet, despite the

immediacy of this experience, it is short-lasting and decays

rapidly. “When you drink it, you get that bit of a rush. Yes,

it’s basically gone now [P15, sour]. In contrast, other tastes

were described as slowly building up or maintaining

consistent intensities (e.g., high for umami, and low for

salty). Such intensities could be seen to be ‘lingering’,

rather than ‘explosive’, as one participant described it:

“You’ve got this “Whoa” sensation, feels quite strong to

start with. Then it has gone super quick” [P19, sour].

While the dynamics of intensity imply variation (intensity

increasing and decreasing), the vocabulary of movement

animates these changes. Describing the bitter taste, one

participant stated: “I guess it’s not sticky like the first one

[umami]. It’s a bit lively… I feel like it’s moving around”

[P15, bitter]. While certain movements can be attributed to

mouth-feel (e.g., moving left to right across the tongue),

others were externalized (e.g., “I feel it almost into my

sinuses and into the rest of my face” [P14, bitter]). These

expressions were not confined to the temporal

characteristics of taste experiences, but already shed light

on the bodily reactions that can be elicited by tastes.

Movement was also invoked to describe stasis (e.g., ‘stays’)

and repetitive movement (e.g., ‘waves’). “So it is kind of

strong and it also stays. It doesn’t have a peak; it doesn’t

go up and down; it just stays” [P2, umami]. Other tastes

fluctuate rapidly: “Yes, ups and downs, but quite quick.

They’re quite sudden crests and falls…” [P3, sour].

Participants often appealed to similes of physicality in order

to explain their taste experiences (e.g., ‘round’, ‘soft’,

‘heavy’). Such physical experiences are tied to a synchronic

perception of taste. In contrast, the diachronic physicality of

taste experiences is given in the implied and experienced

characteristics of taste as a residual presence (e.g.,

‘lingering’, ‘stays there’): “It just stays in your mouth, so it

kind of keeps developing” [P10, umami] or “it just leaves

its mark in your mouth and doesn’t go” [P7, umami]. Such

experiences are, much like the increasing intensities, those

that ‘build up’, or ‘get a bit stronger”. Such presence is

understood to ‘erode’. Moreover, the implied residual

physicality is associated with experiences of absence. When

tasting sourness, many participants described the

immediate, almost physically imposing intensity followed

by a marked absence. This absence is seen to draw the

taster back into the taste, leaving them wanting more: “it

creates an expectation of sweet flavour, like if you were

biting into a slice of orange or something. … It’s gone now

and actually I’d quite happily have another sip, to be

honest” [P18, sour]. This residual physicality can also be

seen to afford agency to taste experiences, where tastes

‘grab you’, and ‘hit you in the face’. As such, taste

experiences can become reified in exerting influence over

the taster. This can be achieved in the residual physicality

or in absence, for instance, where the marked absence in

sourness is seen as “a forward feeling… It has the feeling of

tartness, your mouth moves forwards” [P14, sour].

Sweetness in contrast is associated with the feeling of

filling the mouth, and when the taste is gone it leaves one

with a kind of stickiness on the teeth.

Figure 2 shows a pictorial representation of the different

types of temporality identified based on the above

descriptions across all five tastes. The intensity is

represented through the thickness of the lines in the bars,

while movement is captured through the frequency of the

lines. Finally, residual physicality as temporal characteristic

is shown through the length of the whole bar. Overall, sour

is the taste delivering the highest intensity, followed by

umami and bitter. Umami presents a high intensity, and is

also characterized by lingering without losing much of its

intensity. Such an extensive residual presence can also be

seen for bitter, however with a lower intensity. Sweet and

salty are also of low intensity and can be characterized by

particular movements. While sweet starts slowly, builds up

and then dies out, salty does not peak at all and is constant

in its perception and moderate in unfolding over time. Sour,

by contrast, is short-lived with a rapid end. Specific to sour

is the sharp beginning followed by the absence of a taste

and the return of it through a forward pulling feeling, which

disappears quickly.

Affective reactions

Affective reactions refer to both the sense of pleasure or

displeasure gained from the taste experience, but also

feelings most often regarding familiarity, such as comfort,

or, by contrast, unfamiliarity, such as surprise and

suspicion. These affective characteristics, to be captured as

pleasant-unpleasant and familiar-unfamiliar, operate not

only as a static attitudinal response to taste experiences

(synchronic structure), but also as evolving characteristics

of the taste experience (diachronic structure).

When sampling the taste stimuli many participants related

their own uncertainty (e.g., I don’t know what to expect).

After one sample, this uncertainty is replaced for familiar

tastes. For unfamiliar tastes, particularly bitter and umami,

the sense of unease pervades and persists. Thus familiarity

produces responses at singular points (e.g., I am/am not

familiar with this), while also producing responses across

time (e.g., I know/do not know what to expect). A recurring

phrase throughout the taste study was “I know what it is, but

I don’t”. While we can at times attribute this to the nature

of the stimuli as water solutions (i.e., those not regularly

experienced by participants), the sentiment expressed also

refers to the lived and felt experiences of the tastes. That is,

while participants on the one hand had the taste ‘on the tip

of their tongue’, those tastes also brought to mind a variety

of known experiences, or, in the absence of known

experiences, feelings of uncertainty or unease. Such

feelings must presumably be associated with evolutionary

causes (considering many bitter foods are poisonous) or in

form of personal memories (e.g., salt, salty water, and the

seaside) and cross-modal experiences (e.g., with color, or

sounds). “If I drink or eat something that leaves that kind of

trace, I always imagine a colour. Glowing…. It’s weird. I

have no idea what this is, but there’s a bitterness that

stays” [P2, bitter]. Participants identified as supertasters

expressed their affective reaction more clearly: “Definitely

bitterness… I don’t like it” [P8, bitter], or “It’s immediately

bitter.… It’s like swallowing medicine” [P18, bitter].

There were few predictable or consistent affective reactions

among participants, and those experienced as pleasurable

by some, were experienced as disgusting or unsettling by

others. The affective response of participants could often be

tied to the participant’s familiarity with the taste. This was

particularly noticeable with umami. Participants who were

familiar with this taste indicated familiarity with savory

Asian cuisine, and could therefore interpret the perceived

taste and experienced it as pleasant. Those who did not eat

Asian cuisine were less familiar with the taste, particularly

in this intensity, and described unease and uncertainty when

tasting it. Such responses also evolved over time, notably

with sweet and sour tastes. While, as mentioned, sour

produced an immediately unpleasant experience, followed

by a refreshingly pleasant experience (e.g., “yes it probably

gets more pleasant as the intensity of the taste dissipates”

[P17, sour]), the taste of sweet was often initially pleasant,

followed by a distinct unpleasantness. This unpleasantness

could be so strongly felt as to produce nausea for some

participants (e.g., “although it’s dying off over time. It’s

quite sickly actually” [P20, sweet]). Such experiences were

tied to the physicality of the taste residing in the mouth, and

were perceived in two extremes for umami, influenced

through the participants’ familiarity/unfamiliarity with this

taste. Participants familiar with this taste perceived the

mouth filling and lingering experiences as comforting

(satisfaction after a full meal), while other participants who

were unfamiliar with it perceived it as disgusting, obtrusive,

and annoying referring to the fact that the taste takes over

control, without the chance to get rid of it quickly.

As with temporality, we created a representation of the

different types affective reactions on the five tastes (see

Figure 3). The pleasant-unpleasant characteristics of the

taste experience are represented through the ‘green’ and

‘red’ colors and in cases of a neutral experience colored as

‘orange’, and finally ‘white’ in case of absence of the taste.

The familiar-unfamiliar characteristics only find an explicit

representation for the umami. The familiarity of the taste

lead to its pleasant perception (upper bar for umami), while

unfamiliarity with the taste was expressed through

unpleasantness (lower bar for umami). Overall, some tastes

are characterized by the change from unpleasant to pleasant

(sour) or the other way around from pleasant to unpleasant

(sweet), while the bitter taste was clearly unpleasant and

salty was described as neutral. For umami, we identified

two separate experiences (participants either love or hate it)

grounded in the familiarity and unfamiliarity of the taste.

Embodiment

Although we would expect food experiences to involve

embodied, textural, responses (such as ‘crunchy’, ‘slimy’),

here each taste stimuli is experienced in the same form (i.e.,

as a colorless and odorless solution), and yet produce varied

embodied responses. Embodiment in relation to the

described diachronic and synchronic taste experiences

refers to the mouth-feel of tastes (how something is felt in

your mouth). Some participants additionally describe whole

body reactions (reactions described beyond the mouth) and

others refer to imagined and disembodied responses

(resulting from the taste stimulation and its associations).

Mouth-feel, referring to the experienced chemical and

physical sensations in the mouth, is frequently used to

describe different characteristics of foods, including coffee,

wine, and textured foods. Such descriptions are offered by

our participants for qualities of texture and viscosity. “It’s

just like a softness, but I guess a little bit more viscosity

even though I’m quite sure it doesn’t have any viscosity. It’s

just sort of the feeling of viscosity, the sweetness and this

cloud is just a bit more mouth feel” [P14, sweet]. The

mouth-feel also relates to a sense of movement, where

tastes evolve in space. Most often these are lateral

movements within the mouth, or commonly tastes are felt to

move backwards. Such experiences can be a feature of the

physical movement of the taste stimuli during the swallow

reflex and also associated with the location of taste

receptors on the tongue. However, in other cases, taste

experiences defied the location of taste receptors and tastes

could be experienced on the teeth, gums, and lips. One

participant goes as far as to describe the absence of mouthfeel:

“I don’t know really. It leaves this numbness in my

mouth like the lemon, but without the initial burst” [P9,

sour]. In addition to the sensations described in mouth,

some participants described bodily reactions that were

opposed to the mouth-feel or isolated taste experiences. “I

think the first part of it, the sour part, is a bit of a shock to

the system. I don’t think you’re expecting it to be like that”

[P16, sour]. Another participant said “I kind of see it from

the moment it enters my mouth and goes down all the way

to my stomach. It’s like I can see where it’s going” [P2,

bitter]. In this sense, participants described tastes as

producing expansive responses, including pleasure, nausea,

and, others including reactions associated with allergy such

as increased body heat (e.g., “If you eat it, it’s like your

body – the heat just changes” [P2, umami]). Feelings of

pleasure were often described as filling, particularly filling

the face or the whole body. A participant describes it as

such: “I feel that my whole face feels pleased with it” [P14,

umami]. Such feelings were not always positive and for

some participants, overwhelming feelings of nausea

accompanied tastes of salt, umami, and sweet. Nausea

could also be experienced in undulating taste experiences –

those taste which were experienced as prone to fluctuations

in intensity, almost mimicking travel or sea sickness.

Participants also described disembodied reactions, which

refer to something experienced that lingers between the

mouth and the body. Rather than experiencing direct bodily

reactions, participants describe an imagined reaction. “It’s

like it’s there but it’s not there” [P2, salty]. Disembodied

reactions could also be seen in expected or caricatured

responses, such as the imagined ‘pucker face’ of the sour

taste. Although few participants actually exhibit such a

reaction, it is an ingrained image of biting into a lemon. “It

feels a little bit, not uncomfortable, but it feels like it makes

you kind of screw you face up a bit” [P11, sour].

Shapes assigned to the overall taste experiences

The usage of the SEI objects (see Figure 4) as a physical

engagement with shapes enriched the description of the

taste experiences. The shapes particularly contributed

descriptors related to the combined temporal, affective, and

embodied experiences of each individual taste. Below, we

summarize the key characteristics and the mapping of the

eight objects for each taste across all participants.

The sweet taste, generally experienced as ‘smooth’ and

‘rounded’, was most reflected in shapes that present

elements of change such as “phases” (shapes 4 and 5) or

have protruding elements (like 7 and 1, or even the halfspiky

shape 2). While typically a pleasant taste, there is a

dynamic modulation of intensity and pleasure in the shapes.

The sour taste produces a ‘sharp’ response and for many is

best characterized by shapes such as 8 or 2. There are,

however, also elements of temporality, a shifting/phasing

associated with shape 4, starting with the big part as an

explosion and then rapidly decaying. The salty taste has a

broad aspect (mapped towards 3 and 6) and a finer

granulated and dynamic experience expressed through the

shape 1. Similar to sweetness and sourness there is a

repeating wave assigned to this taste experience, verbalized

around shapes 4 and 5 though this time associated with an

unpleasant feeling/sickliness as aftertaste. More than other

tastes, salty was associated with a subtlety of the temporal

characteristics, an experience of something moving, not

doing much, but still being there. This made participants

want a shape that they could manipulate (“These [objects]

are kind of too permanent; you’re not able to manipulate

them” [P6]) or something more neutral, such as a flat shape,

or a shape, which can be changed. Despite the fact that the

bitter taste was experienced as unpleasant, the mapping to

the shapes created two distinct experiences. For some

participants, bitter is a spiky but lingering experience

associated with a dull unpleasantness (1, 2, and 7 shapes

selected). For others it is a rounded and smooth taste (these

participants chose shapes 5 and 6), associating it with

medicine (form of pills), which dissolves in the mouth, and

you cannot get rid of it. Similarly to bitter, the mapping for

umami resulted in two distinct experiences. If umami was

experienced as unpleasant, participants tended to describe

the taste as disgusting and chose the shape 8 or 2. In those

cases where umami was perceived as pleasant, participants

described it as a more rounded taste with depth and chose

combinations of the rounded shapes (such as 3 and 1 shape

were used most, and combined with either the 5, 4, or/and 7

shape). This mapping confirms the descriptions of umami

as a full, mouth-filling experience with lots of things to it.

Overall, sweet and sour seem to be the two tastes where

participants show high agreement with respect to mapping

the shapes to taste experiences. Bitter and umami seem to

share some associations and create two different mappings

between shapes and taste experiences, while salty shows a

tendency towards smooth and round shapes, but with the

lack of the ability to change and manipulate the shapes.

Combined representation of the taste experiences

Figure 5 shows the final pictorial representation of all three

characteristics combined for each of the five tastes. The

length of the forms represents the temporal aspects, while

the width captures the mouth-feel. Whole body and

imagined embodiment could not be captured as such, but

are described in detail above. The expression ‘lingering’

was used particularly for sweet, bitter, and umami. When

used for sweet and umami ‘lingering’ is experienced in

combination with a ‘mouth filling’ element (it is filling the

whole mouth), while for bitter there is no filling experience

but it is described as a thin (straight through your mouth to

the back) experience, next to being unpleasant. In the bitter

case, ‘lingering’ thus refers to the residual physicality of

this taste (in the back of your mouth). Sour has an initial

unpleasant taste, dies down quickly, but comes back after a

short absence and leaves one with the feeling of wanting

more. Salty at last is similar to bitter, however with a

shorter life and perceived as less unpleasant. Salty is

perceived as a neutral taste with little consequence.

DISCUSSION AND FUTURE RESEARCH

While sensory researchers and neuroscientists study the

perception of taste and its temporality, their focus is on

quantifying the intensity and perceived changes of intensity

via a wide range of evaluation scales [26] or, in some recent

attempts, by means of time-intensity profiles of fMRI data

 

[19]. Our findings add a semantic level of understanding

underlying the taste experiences, their temporality enhanced

through descriptions of the affective reactions and

embodiment that the five basic tastes provoke.

 

This understanding may be useful when designing for taste

experiences as it provides designers and developers a

vocabulary to talk about taste and the design potentials

related to the different characteristics.

 

First, we discuss the particularities of each taste quality, and then discuss them

with respect to established psychological and behavioral

phenomenon highlighting their design potential for HCI.

design. This claim will be addressed later.

How is taste experienced?

Here we discuss the specific experiences each of the five

basic taste qualities create and can inspire design in HCI.

Sweet: Pleasant but with a bittersweet ending

The sweet taste was consistently described as pleasant,

which turned into something unpleasant. Participants

struggled between the instinctive taste likeability and the

learned taste values and rules (sweet is bad for the teeth),

which can be seen in light of learned associations, discussed

by Schifferstein and Hekkert [32] with respect to taste

experiences of products. Of particular interest with respect

to our findings on crossmodal interactions for sweet

stimulations are the embodied reactions (e.g., “It’s just sort

of the feeling of viscosity, the sweetness and this cloud is

just a bit more mouth feel” [P14]). Such reactions can be

explained through learned associations with sweetened food

and beverages. It is a combination of learned as well as

innate, genetic, and cognitive factors [32]. Sweet sensations

can be used to stimulate and enhance positive experiences,

however, on a limited timescale, as the sweetness is quickly

disappearing leaving one unsatisfied. It’s a pleasant taste

but one that is tinged with a bittersweet ending.

Sour: Unpleasant at first, but with the need for more

In contrast to the sweet taste, the sour taste is described as

short-lived and it often comes as a surprise due to its

explosive and punchy character. This taste overwhelms one

with its rapid appearance and quick decay. It leaves one

with the feeling that there is something missing. Based on

childhood memories, such as for instance of sweet-sour

drops, participants were expecting sweetness, but were left

disappointed, leaving them with the feeling of wanting

more. This phenomena was also observed in the evaluation

of a gustatory gaming interface with children, where sour

was used for negative reinforcement linked to the game

dynamics [17]. Children intentionally failed in the game in

order to get another sour stimulation.

Salty: Not doing much

The salty taste experience was not linked to an extreme

reaction unlike sour, bitter, and umami. This taste is often

described as ‘bland’, ‘discrete’, and ‘just being there and

not doing much’. It is minutely moving around, giving the

feeling of cleansing the mouth, but not being mouth filling

as sweet or umami, and certainty not as unpleasant as bitter,

however lingering almost as long as the bitter taste. The

modesty of saltiness in contrast to all of the other tastes

opens up some interesting questions when looking at the

neuroscience findings. Nakamura’s [19] findings based on

time-intensity fMRI profiles suggest that salty tastes change

more rapidly than do sweet tastes. This is not quite

consistent with how our participants described their

experiences and needs further studies.

Bitter: Unpleasant, not to be experienced again

The perceived intensity of the bitter taste was not the same

for everyone, as confirmed by the supertaster test. While

supertasters felt the experience with more immediacy,

others had to allow the taste to travel to the back of their

mouth to recognize it. After this initial difference, the bitter

experience becomes consistent with respect to its

‘lingering’ features, of ‘staying’ either on the tongue or at

the back of the mouth. Bitter was also described as ‘thin’.

The character of bitter was further revealed through learned

associations referring to ‘biting into a flower’, or

‘medicine’, things you had to take as a child, but after

which you would rather avoid this experience of bitterness.

Bitterness can indicate the presence of toxins [32] and is

found in evolutionary development of humans (e.g., feeling

of suspicion regarding bitter food as poisonous) [7]. It may

be useful for design to make people avoid certain behaviors.

Umami: Like/dislike, but still confusing as a taste

The familiar-unfamiliar characteristics of umami caused

much confusion in our study and participants could not rely

on their intuition. While the ‘like’ or ‘dislike’ of the taste

was decided instantly, the unpicking of the still ‘confusing’

elements of the umami taste was more challenging.

Different word pairs depending on the like/dislike of the

taste were expressed: ‘pleasant–unpleasant’, ‘comforting–

uncomforting’, and ‘liking–disgusting’. We could also see

participants using additional bodily descriptors, in

particular when describing umami as a pleasing experience

(‘face feels pleased’ or ‘body heat changes’). In cases of

dislike, the focus of attention in the verbalizations was the

lingering characteristic of the taste founded in the inability

to get rid of it. In these cases, the residual physicality can be

seen to afford agency. The taste experience becomes reified

in the influence it exerts over the taster. Depending on

personal familiarity/unfamiliarity (which may be defined by

cultural factors) and personal preferences, this taste

experience is quite interesting for design. Umami grabs

one’s attention and initiates a conscious process of

reflection. While judgment on the taste is defined quickly,

the reflective thinking brings to the fore the richness and

variety of the taste. Even when perceived as unpleasant, the

richness is recognized, and linked to the motivation to

remove the taste from the mouth.

How can we design with taste experiences?

Taste experiences can be discussed with respect to their

relevance for design, building on existing psychological and

behavioral phenomenon: rational and intuitive thinking,

anchoring effects, and behavior change.

 

The dual process

theory [14,37], for instance, accounts for two styles of

processing: the intuition based System 1 with associative

reasoning that is fast and automatic with strong emotional

bonds, and reasoning based on System 2 which is slower

and more volatile, being influenced by conscious judgments

and attitudes. Based on our findings, we can see that sweet

is intuitively perceived as pleasant, and bitter as unpleasant,

while sour, salty, and umami cause a reflective process,

confused, for instance, by the surprise appearance and rapid

disappearance of the sour taste. Our findings also give

insights into how to time the presentation of the taste

qualities so that the user can transition from System 1

thinking to System 2 thinking. Figures 2, 3, and 5 can be

used to create the appropriate transitions and time them. For

example, the rapidity of the sour taste experience does not

leave enough time for System 1 to engage with it and

triggers System 2 to reflect on what just happened. Such

reactions when carefully timed can prime users to be more

reason based in their thinking during a productivity task

(e.g., to awaken someone who may be stuck in a loop).

Moreover, an appropriately presented taste can create a

synchronic experience that can lead to stronger cognitive

ease (to make intuitive decisions) or reduce the cognitive

ease to encourage rational thinking. For example, a pleasant

taste can be used to provide achievements across the

workflow, however with the slight hint that there are still

more tasks to do before you are finished (e.g., the slight

unpleasant aftertaste of sweetness). Below, we outline

potential design directions for using taste experiences in

work-related activities and for personal behavior

management. Doing so, we draw on the potential of

different taste qualities and their power to stimulate

intuitive and rational thinking described above.

Managing anchoring effects through taste

A common aspect of everyday activity is interruption. We

are often interrupted by emails, telephone calls, or other

unanticipated events. These interruptions can either be short

(e.g., a quick glance at an email pop-up) or slightly longer

requiring us to change our activity (e.g., a line-manager

walking into your office to ask for something). All these

activities have anchoring effects. In other words, the initial

activity affects our judgments and decision making in the

latter activities. It has also been shown that users often find

it hard to avoid these biases in their judgments [38].

Our study of taste experiences suggests that taste interfaces

can be carefully designed to manage interruptions in such a

way that anchoring effects can be either minimized or

maintained. For example, we know that the salty taste has a

long temporal component with a feeling of “not doing much

but being there”. This taste could be very useful in those

situations where the interruption is small and the user is

expected to return to the initial activity soon. As an

example, when the user notices a pop-up in the bottom left

corner of their desktop (for email or other social media

interruptions) a small salty taste in their mouth which starts

just before the user switches their activity can be useful.

This will prolong their initial experience and remind them

of the initial activity when still checking the social media

page. This could enable smoother transitions back to the

initial activity. Alternatively, however, if the interruption is

a longer activity then it is useful for the user to drop any

priming effect that might transfer to the new activity. In this

case, a sour taste in the mouth would leave the user a quick

sharp taste engaging their rational System 2 but rapidly

decaying helping the user return to a more neutral state by

the time they switch to the new activity. Such management

of anchoring effects is not only useful for productivity

activities but also in other activities, such as gaming. For

example, LOLLio – the taste-based game device described

above [16], currently uses sweet and sour for positive and

negative stimulation during the game play. We suggest that

such a game could be improved based on our framework by

providing fine-grained insights regarding the specific

characteristics of taste experiences, which can be integrated

into the game play. When a person moves between related

levels of a game a continuing taste like bitter or salty is

useful. Whereas when a user is moving to distinct levels or

is performing a side challenge an explosive taste like sour,

sweet, or umami might be useful. The choice of specific

tastes in each category can be tuned by the designer to

create different affective reactions and a sense of agency.

Priming positive behavior through taste

Taste and taste preferences play an important role in our

food choices [24] and food plays a significant role in our

health and wellbeing. The stimulation and manipulation of

taste experiences therefore offers potential to improve a

variety of food behaviors. Using taste stimulation technology

to alter the taste of unpleasant but healthy food is one

obvious route. Expanding the design space for healthy taste

technology, our framework suggests alternative routes.

Taste experiences might be heightened through appeal to

related experiences and sensations. Morphing physical

objects, such as recently suggested shape-changing devices

[29], might also be used to replicate the embodied

expansiveness of the umami taste to stimulate an increased

taste experience for patients receiving chemotherapy who

may suffer from hypogeusia, a decrease in taste sensitivity.

Taste stimulation might also facilitate sustainable food

practices, for instance, linking food waste to taste

experiences. Taste stimuli might thus supplement other

post-actional cues in the effective disturbance of food waste

habits and promote critical reflection. When disposing overripe

bananas, a user might get a sour stimulation for the

waste of food but the immediate reward for waste

separation. Taste stimulation might also reflect various

characteristics of food waste, such as its lengthy impact on

environmental sustainability through the bitter taste. In this

way, the framework for design points to the potential for

taste experiences to be incorporated into timely and

rewarding persuasive messages for positive food behaviour.

CONCLUSIONS

In this paper we presented the results of a user study

exploring the experiential characteristics for each of the five

basic taste qualities. Our analysis of participants’

verbalizations, collected by means of verbal and non-verbal

methods, resulted in three key themes. We provide rich

descriptions on the temporality, affective reactions, and

embodiment of taste experiences. We discuss these themes

for each individual taste elucidating the design potentials

with respect to the specific structure and qualities of sweet,

sour, salty, bitter, and umami tastes.

 

Our findings help to establish a framework for the design of taste experiences in

HCI, enhancing existing technology driven research around

taste, and food interaction design research. Although we do

not provide guidance for the design of a specific interactive

system in this paper, we are convinced that our framework

provides a starting point for designers and developers to

think about design/development potentials for taste in HCI.

Concluding Comment

1. This is an interesting paper, which reports an initial exploration of taste as a potential interactive modality for HCI.

2. The paper reports a taste framework, intended to support HCI design. The question arises, however, of how this framework might be further developed to provide such support. To that end, knowledge can generally be thought to include knowing what and knowing how (declarative and procedural knowledge respectively). Understanding, as used in this paper (see Comment 1), might be similarly expressed. The framework is already declarative (descriptive) knowledge and conceptualises taste experience (see Comment 20). It could be further developed into a (design) model, in which the relationships between the concepts are made explicit.

3. Subsequent development would include the model’s operationalisation, test and generalisation towards the aim of validation (see Comments 6 and 10).

4. To be used by designers, the model would also need a method of application – procedural design knowledge. The model and method would be validated together for their ability to support the diagnosis of design problems and the prescription of design solutions (see Comments 6 and 10).

5.There are, of course, many other possible ways forward. The way forward, suggested here, could be followed; but should otherwise be considered as encouragement for finding a way.

ACKNOWLEDGMENTS

This work is supported by the EU Marie Curie Action (FP7-

PEOPLE-2010-IEF) and RCUK SiDE (EP/G066019/1). We

wish to thank our study participants and Annika Haas for

the audio-visual support, as well as Katerhine Isbister and

Kristina Höök for providing us with a set of the SEI objects.

REFERENCES

1. Bartoshuk, L.M., Duffy, V.B., Fast, K., Green, B.G,

Prutkin, J., Snyder, D.J. Labeled scales (e.g., category,

Likert, VAS) and invalid across-group comparisons:

What we have learned from genetic variation in taste,

Food Qual Pref, 14(2), (2003), 125-138.

2. Bartoshuk, L.M. Comparing sensory experiences across

individuals: Recent psychophysical advances illuminate

genetic variation in taste perception. Chem. Senses,

25(4), (2000) 447-460.

3. Chen, J., Engelen, L. (Eds.). Food oral processing:

Fundamentals of eating and sensory perception. Wiley-

Blackwell: Oxford, UK (2012).

4. Comber, R., Ganglbauer, E., Choi, et al. Food and

interaction design: designing for food in everyday life.

CHI EA (2012), 2767-2770.

5. Desmet, P.M.A., Schifferstein, H.N.J. Sources of

positive and negative emotions in food experience.

Appetite, 50(2-3), (2008), 290-301.

6. Drewnowski, A., Kristal, A., Cohen, J. Genetic taste

responses to 6-n-propylthiouracil among adults: A

screening tool for epidemiological studies. Chem.

Senses 26(5), (2001), 483-489.

7. Glendinning J. Is the bitter rejection response always

adaptive? Physiol. Beh., 56, (1994), 1217-1227.

8. Green, B.G., et al. Evaluating the ‘Labeled Magnitude

Scale’ for measuring sensations of taste and smell.

Chem. Senses, 21(3), (1996), 323-334.

9. Grimes, A., Harper, R. Celebratory technology: new

directions for food research in HCI. CHI (2008), 467-

476.

10. Hoehl, K., Schoenberger, G.U., Busch-Stockfisch, M.

Water quality and taste sensitivity for basic tastes and

metallic sensation. Food Qual Pref 21(2), (2010), 243-

249.

11. Hupfeld, A., Rodden, T. Laying the table for HCI:

uncovering ecologies of domestic food consumption.

CHI (2012), 119-128.

12. Humphries, C. Delicious science. Chefs are teaming up

with researchers to create avantgarde dishes. Is

‘molecular gastronomy’ more than a fad? Nature, 486

(2012), 10-11.

13. Isbister, K., Höök, K., Sharp, M., Laaksolahti, J. The

sensual evaluation instrument: Developing an affective

evaluation tool. CHI (2006), 1163-1172.

14. Kahneman, D. A perspective on judgement and choice.

American Psychologist, 58(9), (2003), 697-720.

15. Light, A. Adding method to meaning: A Technique for

exploring peoples’ experiences with technology. Beh.

and Information Technology, 25(6) (2006), 175-187.

16. Murer, M., Aslan, I., Tscheligi, M. LOLLio: Exploring

taste as playful modality. In Proc. TEI 2013, 299-302.

17. Moser, C., Tscheligi, M. Playful taste interaction. IDC

(2013), 340-343.

18. Narumi, T., Nishizaka, S., Kajinami, T., Tanikawa, T.,

Hirose, M. Augmented reality flavors: gustatory display

based on edible marker and cross-modal interaction.

CHI (2011), 93-102.

19. Nakamura, Y., Goto, T.K., Tokumori, K., et al. The

temporal change in the cortical activations due to salty

and sweet tastes in humans: fMRI and time-intensity

sensory evaluation. Neurorep, 23(6), (2012), 400-404.

20. Ngo, M.K., Velasco, C., Salgado, A., et al. Assessing

crossmodal corresponddences in exotic fruit juices: The

case of shape and sound symbolism. Food Qual Pref 28,

(2013), 361-369.

21. Núnez-Jaramillo, L., Ramirez-Lugo, L., Herrera-

Morales, W., Miranda, M.I. Taste memory formation:

Latest advances and challenges. Behav Brain Res,

207(2), (2010), 232-248.

22. Maynes-Aminzade, D. Edible bits: Seamless interfaces

between people, data and food. CHI EA (2005), 2207-

2210.

23. Obrist, M., Seah, S. A., Subramanian, S. Talking about

tactile experiences. CHI (2013), 1659-1668.

24. Palmer, S. E., Schloss, K. B. An ecological valence

theory of human color preference. Proc. of the National

Academy of Sciences, 107(19) (2010), 8877-8882.

25. Petitmengin, C. Describing one’s subjective experience

in the second person. An interview method for the

science of consciousness. Phen. Cog. Sci., 5(3-4),

(2006), 229-269.

26. Pineau, N., Schlich, P., Cordelle, S., et al. Temporal

dominance of sensations: Construction of the TDS

curves and comparison with time-intensity. Food Qual

Pref, 20(6), (2009), 450-455.

27. Ranasinghe, N., Cheok, A.D., Nakatsu, R. Taste/IP: The

sensation of taste for digital communication. ICMI

(2012), 409-416.

28. Ranasinghe, N., Karunanayaka, K., Cheok, A.D.,

Fernando, O.N.N., Nii, H., Gopalakrishnakone, P.

Digital taste and smell communication. BodyNets

(2011), 78-84.

29. Roudaut, A., Karnik, A., Löchtefeld, M. et al.

Morphees: toward high “shape resolution” in selfactuated

flexible mobile devices. CHI (2013), 593-602.

30. Savage, N. Technology: The taste of things to come.

Nature 486(21), (2012), 18–19.

31. Schifferstein, H.N.J., Fenko, A., Desmet, et al. Influence

of package design on the dynamics of multisensory and

emotional food exp. Food Qual Pref, 27 (2013), 18-25.

32. Schifferstein, H.N.J. Hekkert, P. Product experience.

USA: Elsevier (2007).

33. Smets, G.J.F., Overbeeke, C.J. Expressing tastes in

packages. Design Studies, 16(3), (1995), 349-365.

34. Spence, C. Crossmodal correspondences: A tutorial

review. Atten Percept Psychophys 73, (2011), 971-995.

35. Spence, C., Piqueras-Fiszman, B. Technology at the

dining table. Flavour, (2013), 2-16.

36. Spence, C., Smith, B., Auvray, M. Confusing tastes and

flavours. In M. Matthen and D. Stokes (Eds.). The

senses. Oxford: University Press. (in press, 2014).

37. Stanovich, K.E., West, R.F. Individual difference in

reasoning: implications for the rationality debate?.

Behav. Brain Sciences 23(5), (2000), 645–665.

38. Strack, F., Mussweiler, T. Explaining the enigmatic

anchoring effect: Mechanisms of selective accessibility.

Pers. and Social Psy. 73(3), (1997), 437-446.

39. Tosey, P., Mathison, J. Exploring inner landscapes

through psychophenomenology. Qual. Research in

Organiz. And Management: Inter. J., 5(1) (2010), 63-82.

40. Trivedi, B.P. Hardwired for taste. Research into human

taste receptors extends beyond the tongue to some

unexpected places. Nature, 486, (2012), 7-9.

41. Vermersch P. L’entretien d’explicitation [translated as

Explicitation interview], ESF (1994).

42. Wakeling, I.N., MacFie, H.J. Designing consumer trials

balanced for first and higher orders of carry-over effect

when only a subset of k samples from t may be tested.

Food Qual Pref, 6(4), (1995), 299-308.

————————————————————————————————–

Permission to make digital or hard copies of all or part of this work for personal or

classroom use is granted without fee provided that copies are not made or distributed

for profit or commercial advantage and that copies bear this notice and the full

citation on the first page. Copyrights for components of this work owned by others

than ACM must be honored. Abstracting with credit is permitted. To copy otherwise,

or republish, to post on servers or to redistribute to lists, requires prior specific

permission and/or a fee. Request permissions from Permissions@acm.org.

CHI 2014, April 26 – May 01 2014, Toronto, ON, Canada

Copyright 2014 ACM 978-1-4503-2473-1/14/04…$15.00.

http://dx.doi.org/10.1145/2556288.2557007

——————————————————————————————————-

Applied Approach Illustration – Interacting with the Computer: a Framework 150 150 John

Applied Approach Illustration – Interacting with the Computer: a Framework

Interacting with the Computer: a Framework

J. Morton, P. Barnard, N. Hammond* and J.B. Long

M.R.C. Applied Psychology Unit, Cambridge, England *also IBM Scientific Centre, Peterlee, England

Recent technological advances in the development of information processing systems will inevitably lead to a change in the nature of human-computer interaction. Direct interactions with systems will no longer be the sole province of the sophisticated data processing professional or the skilled terminal user. In consequence, assumptions underlying human-system communication will have to be re-evaluated for a broad range of applications and users. The central issue of the present paper concerns the way in which this re-evaluation should occur.

First of all, then, we will present a characterisation of the effective model which the computer industry has of the interactive process. The shortcoming of the model is that it fails to take proper account of the nature of the user and as such can not integrate, interpret, anticipate or palliate the kinds of errors which the new user will resent making. For remember that the new user will avoid error by adopting other means of gaining his ends, which can lead either to non-use or to monstrously inefficient use. We will document some user problems in support of this contention and indicate the kinds of alternative models which we are developing in an attempt to meet this need.

The Industry’s Model (IM)

The problem we see with the industry’s model of the human-computer interaction is that it is computer-centric. In some cases, as we shall see, it will have designer-centric aspects as well.

To start off with, consider a system designed to operate in a particular domain of activity.In the archetypal I.M. the database is neutralised in much the same kind of way that a statistician will ritually neutralise the data on which he operates, stripping his manipulation of any meaning other than the purely numerical one his equations impose upon the underlying reality. This arises because the only version of the domain which exists at the interface is that one which is expressed in the computer. This version, perhaps created by an expert systems analyst on the best logical grounds and the most efficient, perhaps, for the computations which have to be performed, becomes the one to which the user must conform. This singular and logical version of the domain will, at best, be neutral from the point of view of the user. More often it will be an alien creature, isolating the user and mocking him with its image of the world and its resources to which he must haplessly conform.

Florid language? But listen to the user talking.

“We come into contact with computer people, a great many of whom talk a very alien language, and you have constant difficulty in trying to sort out this kind of mid-Atlantic jargon.”

“We were slung towards what in my opinion is a pretty inadequate manual and told to get on with it”

“We found we were getting messages back through the terminal saying there’s not sufficient space on the machine. Now how in Hell’s name are we supposed to know whether there’s sufficient space on the machine?” .

In addition the industry’s model does not really include the learning process; nor does it always take adequate note of individual’s abilities and experience:

“Documentation alone is not sufficient; there needs to be the personal touch as well . ”

“Social work being much more of an art than a science then we are talking about people who are basically not very numerate beginning to use a machine which seems to be essentially numerate.”

Even if training is included in the final package it is never in the design model. Is there anyone here, who, faced with a design choice asked the questions “Which option will be the easiest to describe to the naive user? Which option will be easiest to understand? Which option will be easiest to learn and remember?”

Let us note again the discrepancy between the I.M. view of error and ours . For us errors are an indication of something wrong with the system or an indication of the way in which training should proceed. In the I.M. errors are an integral part of the interaction. For the onlooker the most impressive part of a D.P. interaction is not that it is error free but that the error recovery procedures are so well practised that it is difficult to recognise them for what they are .

We would not want it thought that we felt the industry was totally arbitrary . There are a number of natural guiding principles which most designers would adhere to.

We do not anticipate meeting a system in which the command DESTROY has the effect of preserving the information currently displayed while PRESERVE had the effect of erasing the operating system. However , the principles employed are intuitive and non-systematic. Above all they make the error of embodying the belief that just as there can only be one appropriate representation of the domain, so there is only one kind of human mind.

A nice example of a partial use of language constraints is provided by a statistical package called GENSTAT. This package permits users to have permanent userfiles and also temporary storage in a workfile. The set of commands associated with these facilities are :

PUT – copies from core to workfile

GET – copies from workfile to core

FILE – defines a userfile

SAVE – copies from workfile to userfile

FETCH – copies from userfile to workfile

The commands certainly have the merit that they have the expected directionality with respect to the user. However to what extent do, for example, FETCH and GET relate naturally to the functions they have been assigned? No doubt the designers have strong intuitions about these assignments. So do users and they do not concur. We asked 40 people here at the A. P.U. which way round they thought the assignments should go: nineteen of these agreed with the system designers, 21 went the 0ther way . The confidence levels of rationalisations were very convincing on both sides!

The problem then, is not just that systems tend to be designer-centric but that the designers have the wrong model either of the learning process or of the non-D.P. users’ attitude toward error. A part-time user is going to be susceptible to memory failure and, in particular, to interference from outside the computer system. du Boulay and O’ Shea [I] note that naive users can use THEN in the sense of ‘next’ rather then as ‘implies’. This is inconceivable to the IM for THEN is almost certainly a full homonym for most D.P. and the appropriate meaning the appropriate meaning thoroughly context-determined .

An Alternative to the Industry Model

The central assumption for the system of the future will be ‘systems match people’ rather than ‘people match systems’. Not entirely so, as we shall elaborate, for in principle, the capacity and perspectives of the user with respect to a task domain could well change through interaction with a computer system.

But the capacity to change is more limited than the variety available in the system . Our task, then, is to characterise the mismatch between man and computer in such a way that permits us to direct the designer’s effort.

In doing this we are developing two kinds of tool, conceptual and empirical. These interrelate within an overall scheme for researching human-computer interaction as shown in Figure 1.

Relating Conceptual and Empirical Tools

 

The conceptual tools involve the development of a set of analytic frameworks appropriate to human computer interaction. The empirical tools involve the development of valid test procedures both for the introduction of new systems and the proving of the analytic tools. The two kinds of tool are viewed as fulfilling functions comparable to the role of analytic and empirical tools in the development of technology. They may be compared with the analytic role of physics, metallurgy and aerodynamics in the development of aircraft on the one hand and the empirical role of a wind tunnel in simulating flight on the other hand.

Empirical Tools

The first class of empirical tool we have employed is the observational field study, with which we aim to identify some of the variables underlying both the occasional user’s perceptions of the problems he encounters in the use of a computer system, and the behaviour of the user at the terminal itself.

The opinions cited above were obtained in a study of occasional users discussing the introduction and use of a system in a local government centre [2]. The discussions were collected using a technique which is particularly free from observer influence [3 ].

In a second field study we obtained performance protocols by monitoring users while they solved a predefined set of problems using a data base manipulation language [4 ]. We recorded both terminal performance and a running commentary which we asked the user to make, and wedded these to the state of the machine to give a total picture of the interaction. The protocols have proved to be a rich source of classes of user problem from which hypotheses concerning the causes of particular types of mismatch can be generated.

There is thus a close interplay between these field studies, the generation of working hypotheses and the development of the conceptual frameworks. We give some extracts from this study in a later section.

A third type of empirical tool is used to test specific predictions of the working hypothesis. The tool is a multi-level interactive system which enables the experimenter to simulate a variety of user interfaces, and is capable of modeling and testing a wide range of variables [5]. It is based on a code-breaking task in which users perform a variety of string-manipulation and editing functions on coded messages.

It allows the systematic evaluation of notational, semantic and syntactic variables. Among the results to be extensively reported elsewhere is that if there is a common argument in a set of commands, each of which takes two arguments, then the common argument must come first for greatest ease of use. Consistency of argument order is not enough: when the common argument consistently comes second no advantage is obtained relative to inconsistent ordering of arguments [6].

Conceptual Tools

Since we conceive the problem as a cognitive one, the tools are from the cognitive sciences.

Comment 1

HCI, then, applies analytic tools taken or derived from the cognitive sciences.

Also we define the problem as one with those users who would be considered intellectually and motivationally qualified by any normal standards. Thus we do not admit as a potential solution that of finding “better” personnel, or simply paying them more, even if such a solution were practicable.

The cognitive incompatibility we describe is qualitative not quantitative and the mismatch we are looking for is one between the user’s concept of the system structure and the real structure: between the way the data base is organised in the machine and the way it is organised in the head of the user: the way in which system details are usually encountered by the user and his preferred mode of learning.

The interaction of human and computer in a problem-solving environment is a complex matter and we cannot find sufficient theory in the psychological literature to support our intuitive needs. He have found it necessary to produce our own theories, drawing mainly on the spirit rather than the substance of established work.

Comment 2

It is assumed that the theories referenced here would be cognitive science theories. See also Comment 1.

Further than this, it is apparent that the problem is too complex for us to be able to use a single theoretical representation. The model should not only be appropriate for design, it should also give a means of characterising errors – so as to understand their origins and enable corrective measures to be taken.

Comment 3

Cognitive science theories (Comments 1 and 2) are intended to be applied to the design of humans interacting with computers. Therein lies the applied approach. Human performance can be expressed as errors.

Take the following protocol.

The user is asked to find the average age of entries in the block called PEOPLE.

“I’ll have a go and see what happens” types: *T <-AVG(AGE,PEOPlE)

machine response: AGE – UNSET BLOCK

“Yes, wrong, we have an unset block. So it’s reading AGE as a block, so if we try AGE and PEOPLE the other way round maybe that’ll work.”

This is very easy to diagnose and correct. The natural language way of talking about the target of the operation is mapped straight into the argument order. The cure would be to reverse the argument order for the function AVG to make it compatible.

The next protocol is more obscure. The task is the same as in the preceding one.

“We can ask it (the computer) to bring to the terminal the average value of this attribute.”

types: *T -AVG( AGE)

machine response: AVG(AGE) – ILLEGAL NAME

“Ar.d it’s still illegal. .. ( … ) I’ve got to specify the block as well as the attribute name.”

Well of course you have to specify the block. How else is the machine going to know what you’re talking about? A very natural I.M. response. How can we be responsible for feeble memories like this.

However, a more careful diagnosis reveals that the block PEOPLE is the topic of the ‘conversation’ in any case.

The block has just been used and the natural language conventions are quite clear on the point.

We have similar evidence for the importance of human-machine discourse structures from the experiment using the code-breaking task described above. Command strings seem to be more ‘cognitively compatible’ when the subject of discourse (the common argument) is placed before the variable argument. This is perhaps analogous to the predisposition in sentence expression for stating information which is known or assumed before information which is new [7]. We are currently investigating this influence of natural language on command string compatibility in more detail.

The Block Interaction Model

Systematic evidence from empirical studies, together with experience of our own, has led us to develop a conceptual analysis of the information in the head of the user (see figure 2). Our aim with one form of analysis is to identify as many separable kinds of knowledge as possible and chart their actual or potential interactions with one another. Our convention here is to use a block diagram with arrows indicating potential forms of interference. This diagram enables us to classify and thus group examples of interference so that they could be counteracted in a coordinated fashion rather than piecemeal. It also enables us to establish a framework within which to recognise the origin of problems which we haven’t seen before. Figure 2 is a simplified form of this model. The blocks with double boundaries, connected by double lines, indicate the blocks of information used by the ideal user. The other lines indicate prime classes of interference. The terminology we have used is fairly straightforward: Domain – the range of the specific application of a system. This could be a hospital, a city’s buildings, a set of knowledge such as jobs in ~n employment agency. Objects – the elements in the particular data base. They could be a relational table, patients’ records. I Representation of domain I Representa ti on of work-base version of domain domain Representation of problem Operations – the computer routines which manipulates the objects. Labels – the letter sequences which activate operators which, together with arguments and syntax, constitute the commands. Work base – in general, people using computer systems for problem solving have had experience of working in a non-computerised work environment either preceding the computerisation or at least in parallel with the computer system. The representation of this experience we call the work-base version. There will be overlap between this and the users representation of the computer’s version of the domain; but there will be differences as well, and these differences we would count as potential sources of interference. There may be differences in ·the underlying structure of the data in the two cases, for example, and will certainly be differences in the objects used. Thus a user found to be indulging in complex checking procedures after using the command FILE turned out to be perplexed that the material filed was still present on the screen. With pieces of paper, things which are filed actually go there rather than being copied. Here are some examples of interference from one of our empirical studies [4]:

Interference on the syntax from other languages. Subject inserts necessary blanks to keep the strings a fixed length.

“Now that’s Matthewson, that’s 4,7, 10 letters, so I want 4 blanks”

types: A+<:S:NAME = ‘MATTHEWSON ‘:>PEOPLE

Generalised interference

“Having learned how reasonably well to manipulate one system, I was presented with a totally different thing which takes months to learn again.”

Interference of other machine characteristics on machine view

“I’m thinking that the bottom line is the line I’m actually going to input. So I couldn’t understand why it wasn’t lit up at the bottom there, because when you’re doing it on (another system) it’s always the bottom line.”

The B.I.M. can be used in two ways. We have illustrated its utility in pinpointing the kinds of interference which can occur from inappropriate kinds of information. We could look at the interactions in just the opposite way and seek ways of maximising the benefits of overlap. This is, of course, the essence of ‘cognitive compatibility’ which we have already mentioned. Trivially, the closer the computer version of the domain maps onto the user’s own version of the domain the better. What is less obvious is that any deviations should be systematic where possible.

In the same way, it is pointless to design half the commands so that they are compatible with the natural language equivalents and use this as a training point if the other half, for no clear reason, deviate from the principle. If there are deviations then they should form a natural sub-class or the compatibility of the other commands will be wasted.

Comment 4

The example shows how cognitive science theory might be applied to design in practice.

Information Structures

In the block interaction model we leave the blocks ill-defined as far as their content is concerned. Note that we have used individual examples for user protocols as well as general principles in justifying and expanding upon the distinctions we find necessary. What we fail to do in the B. I .M. is to characterise the sum of knowledge which an individual user carries around with him or brings to bear upon the interaction. We have a clear idea of cognitive compatibility at the level of an individual. If this idea is to pay then these structures must be more detailed.

There is no single way of talking about information structures. At one extreme there is the picture of the user’s knowledge as it apparently reveals itself in the interaction; the view, as it were, that the terminal has of its interlocutor. From this point of view the motivation for any key press is irrelevant. This is clearly a gross oversimplification.

The next stage can be achieved by means of a protocol. In it we would wish to separate out those actions which spring from the users concept of the machine and those actions which were a result of him being forced to do something to keep the interaction going. This we call ‘heuristic behaviour’. This can take the form of guessing that the piece of information which is missing will be consistent with some other system or machine. “If in doubt, assume that it is Fortran” would be a good example of this. The user can also attempt to generalise from aspects of the current system he knows about. One example from our study was where the machine apparently failed to provide what the user expected. In fact it had but the information was not what he had expected. The system was ready for another command but the user thought it was in some kind of a pending state, waiting with the information he wanted. In certain other stages – in particular where a command has produced a result which fills up the screen – he had to press the ENTER key – in this case to clear the screen. The user then over-generalised from this to the new situation and pressed the ENTER key again, remarking

“Try pressing ENTER again and see what happens.”

We would not want to count the user’s behaviour in this sequence as representing his knowledge of the system – either correct knowledge or incorrect knowledge. He had to do something and couldn’t think of anything else. When the heuristic behaviour is eliminated we are left with a set of information relevant to the interaction. With respect to the full, ideal set of such information, this will be deficient with respect to the points, at which the user had to trust to heuristic behaviour.

Note that it will also contain incorrect information as well as correct information; all of it would be categorised by the user as what he knew, if not all with complete confidence, certainly with more confidence than his heuristic behaviour. The thing which is missing from B.I.M. and I.S. is any notion of the dynamics of the interaction. We find we need three additional notations at the moment to do this. One of these describes the planning activity of the user, one charts the changes in state of user and machine and one looks at the general cognitive processes which are mobilised.

Goal Structure Model

The user does some preparatory work before he presses a key. He must formulate some kind of plan, however rudimentary. This plan can be represented, at least partially, as a hierarchical organisation. At the top might be goals such as “Solve problem p” and at the bottom “Get the computer to display Table T”. The Goal Structure model will show the relationships among the goals.

Comment 5

The user here is understood to be using the computer for a purpose, that is, to do something.

This can be compared with the way of structuring the task imposed by the computer. For example, a user’s concept of editing might lead to the goal structure:

Two problems would arise here. Firstly the new file has to be opened at an ‘unnatural’ place. Secondly the acceptance of the edited text changes from being a part of the editing process to being a part of the filing process.

The goal structure model, then, gives us a way of describing such structural aspects of the user’s performance and the machines requirements. Note that such goals might be created in advance or at the time a node is evaluated. Thus the relationship of the GSM to real time is not simple.

The technique for determining the goal structure may be as simple as asking the user “What are you trying to do right now and why?” This,may be sufficient to reveal procedures which are inappropriate for the program being used.

State Transition Model

In the course of an interaction with a system a number of changes take place in the state of the machine. At the same time the user’s perception of the machine state is changing. It will happen that the user misjudges the effect of one command and thereafter’ enters others which from an outside point of view seem almost random. Our point is, as before, that the interaction can only be understood from the point of view of the user.

 

This brings us to the third of the dynamic aspects of the interaction: the progress of the user as he learns about the system.

Let us explore some ways of representing such changes. Take first of all the state of the computer. This change is a result of user actions and can thus be represented as a sequence of Machine States (M.S.) consequent on user action. If the interaction is error free, changes in the representations would follow changes in the machine states in a homologous manner. Errors will occur if the actual machine state does not match its representation.

We will now look at errors made by a user of an interactive data enquiry system. We will see errors which reveal both the inadequate knowledge of the particular machine state or inadequate knowledge of the actions governing transitions between states. The relevant components of the machine are the information on the terminal display and the state of a flag shown at the bottom right hand corner of the display which ‘informs the user of some aspects of the machine state (ENTER … or OUTPUT … ). In addition there is a prompt, “?”, which indicates that the keyboard is free to be used, there is a key labelled ENTER. In the particular example the user wishes to list the blocks of data he has in his workspace. The required sequence of machine states and actions is:

The machine echoes the command and waits with OUTPUT flag showing.

User: “Nothing happening. We’ve got an OUTPUT there in the corner I don’ t know what that means.

The user had no knowledge of MS2: we can hypothesise his representation of the transition to be:

This is the result of an overgeneralisation. Commands are obeyed immediately if the result is short, unless the result is block data of any size. The point of this is that the data may otherwise wipe everything from the screen. With block data the controlling program has no lookahead to check the size and must itself simply demand the block, putting itself in the hands of some other controlling program. We see here then a case where the user needs to have some fairly detailed and otherwise irrelevant information about the workings of the system in order to make sense of (as opposed to learn by rote) a particular restriction.

The user was told how to proceed, types ENTER, and the list of blocks is displayed together with the next prompt. However, further difficulties arise because the list of blocks includes only one name and the user was expecting a longer listing. Consequently he misconstrues the state of the machine. (continuing from previous example)

User types ENTER

Machine replies with block list and prompt.

Flag set to ENTER …

“Ah, good, so we must have got it right then.

A question mark: (the prompt). It doesn’t give me a listing. Try pressing ENTER again and see what happens.”

User types ENTER

“No? Ah, I see. Is that one absolute block, is that the only blocks there are in the workspace?”

This interaction indicates that the user has derived a general rule for the interaction:

“If in doubt press ENTER”

After this the user realises that there was only one name in the list. Unfortunately his second press of the ENTER key has put the machine into Edit mode and the user thinks he is in command mode. As would be expected the results are strange.

At this stage we can show the machine state transitions and the user’s representation together in a single diagram, figure 3.

This might not be elegant but it captures a lot of features of the interaction which might otherwise be missed.

 

The final model we use calls upon models currently available in cognitive psychology which deal with the dynamics of word recognition and production, language analysis and information storage and retrieval. The use of this model is too complex for us to attempt a summary here.

Comment 6

Although a summary cannot be provided here, it is clear that the cognitive psychology models, to which reference is made, are intended to be applied to the design of humans interacting with computers.

 

Conclusion

We have stressed the shortcomings of what we have called the Industrial Model and have indicated that the new user will deviate considerably from this model. In its place we have suggested an alternative approach involving both empirical evaluations of system use and the systematic development of conceptual analyses appropriate to the domain of person-system interaction. There are, of course, aspects of the I.M. which we have no reason to disagree with, for example, the idea that the computer can beneficially transform the users view of the problems with which he is occupied. However, we would appreciate it if someone would take the trouble to support this point with clear documentation. So far as we can see it is simply asserted.

Finally we would like to stress that nothing we have said is meant to be a solution – other than the methods. We do not take sides for example, on the debate as to whether or not interactions should be in natural language – for we think the question itself is a gross oversimplification. What we do know is that natural language interferes with the interaction and that we need to understand the nature of this interference and to discover principled ways of avoiding it.

And what we know above all is that the new user is most emphatically not made in the image of the designer.

References

[1 ] du Boulay, B. and O’Shea, T. Seeing the works: a strategy of teaching interactive programming. Paper presented at Workshop on ‘Computing Skills and Adaptive Systems’, Liverpool, March 1978.

[2] Hammond, N.V., Long, J.B. and Clark, l.A. Introducing the interactive computer at work: the users’ views. Paper presented at Workshop on ‘Computing Skills and Adaptive Systems’, Liverpool, March 1978.

[3] Wilson. T. Choosing social factors which should determine telecommunications hardware design and implementation. Paper presented at Eighth International Symposium on Human Factors in Telecommunications, Cambridge, September 1977.

[4] Documenting Human-computer Mismatch with the occasional interactive user. APU/IBM project report no. 3, MRC Applied Psychology Unit. Cambridge, September 1978.

[5] Hammond, N.V. and Barnard, P.J. An interactive test vehicle for the investigation of man-computer interaction. Paper presented at BPS Mathematical and Statistical Section Meeting on ‘Laboratory Work Achievable only by Using a Computer’, London, September 1978.

[6] An interactive test vehicle for the study of man-computer interaction. APU/IBM project report no. 1,MRC Applied Psychology Unit, Cambridge, September 1978.

[7] Halliday, M.A.K. Notes on transitivity and theme in English. Part 1. Journal of Linguistics, 1967, 3, 199-244.

FIGURE 3: STATE TRANSITION EXAMPLE

5. HCI Engineering Research Exemplar 150 150 John

5. HCI Engineering Research Exemplar

The HCI/E(U) approach is based on a Conception for HCI/E and associated Frameworks for HCI research, comprising – Discipline; Design Problem; Design Knowledge; Design Practices (as set out above) and Design Research Exemplar – as set out below. The latter encapsulates all the HCI/E components.

The design research exemplar specifies a complete HCI/E design research cycle, which, once implemented, constitutes a case-study of an engineering approach to HCI.

The diagram, which follows, presents the HCI/E(U) design research exemplar for HCI/E.

 

Key: EP – Empirical Practice EK – Empirical Knowledge as: design guidelines; models and methods

SFP – Specific Formal Practice GFP – General Formal Practice

SFK Specific Formal Knowledge as: Specific Design Principle (Declarative and Methodological)

GFK – General Formal Knowledge as: General Design Principle (Declarative and methodological)

The HCI/E design research exemplar is described below by level, starting at the lowest level:

Level 1: User Requirements are transformed into an Interactive System by means of implicit design research, implicit design knowledge and implicit design practices of implement and test. The knowledge and practices at this level are not explicit and so are not addressed here. Hence, they do not appear in the diagram. If the design, however, is ‘for performance’ the knowledge and practices might be considered ‘Craft Engineering or some-such’.

As an illustration, User Requirements for e-shopping might be transformed into an Interactive e-shopping System to the satisfaction of the client. The knowledge and practices would comprise the experience and best practice of the interactive system designers. The research would comprise the enhancement of their design experience.

Level 2: Design Problems are explicitly, but empirically, derived from and validated, against User Requirements. Design Problems must, at least in principle, be soluble. User Requirements, however, may be impossible to be satisfied by an Interactive System. Likewise, Design Solutions are explicitly, but empirically, derived from and validated against the Interactive System. Design Research explicitly, but empirically, acquires and validates Design Knowledge. The latter supports the explicit, but empirical, Design Practices of Specification and Implementation and Test of Design Solutions from Design problems and from Design Solutions to Design Problems.

As an illustration, the e-shopping ‘check-out’ Design Problem of the slow and inaccurate billing of goods (ineffective performance) might be solved by a virtual shopping cart, which cumulates the costs of ordered goods (effective performance). The empirical Design Knowledge supporting the Design Practices might be expressed as: heuristics; guidelines etc.

Level 3: Specific Principle Design Problems are explicitly, but empirically, derived from and validated against Design Problems. Likewise, Specific Principle Design Solutions are explicitly, but empirically, derived from and validated against Design Solutions. Specific Principle Design Research explicitly, but empirically, acquires and validates Specific Principles. The latter support the explicit, formal Design Practices of Derivation and Verification of Specific Principle Design Solutions from Specific Principle Design Problems and from Specific Principle Design Solutions to Specific Principle Design Problems. That is to say, specify, then implement.

As an illustration, the e-shopping check-out ‘goods costs against client financial budget’ Specific Principle Design Problem of the slow and inaccurate client assessment of ordered goods’ costs against their financial budget (ineffective performance) might be solved by a virtual shopping cart, which deducts the financial costs of ordered goods against a client-specified financial budget (effective performance). The Specific Principle Formal Design Knowledge supporting the Specific Principle Formal Design Practices would be expressed as a Formal Specific Principle.

Level 4: General Principle Design Problems are formally derived from and validated against Specific Principle Design Problems. Likewise, General Principle Design Solutions are formally derived from and validated against Specific Principle Design Solutions. General Principle Design Research , formally acquires and formally validates General Principles. The latter support the formal Design Practices of Derivation and Verification of general Principle Design Solutions from General Principle Design Problems and from General Principle Design Solutions to General Principle Design Problems. That is to say, specify, then implement.

As an illustration, the e-shopping check-out ‘goods costs against client resources (that is, budget, calory’ Specific Principle Design Problem of the slow and inaccurate client assessment of ordered goods’ costs against their budget (ineffective performance) might be solved by a virtual shopping cart, which deducts the costs of ordered goods against a client-specified budget (effective performance). The Specific Principle Formal Design Knowledge supporting the Specific Principle Formal Design Practices would be expressed as a Formal Specific Principle.

As an illustration, the e-shopping check-out ‘goods costs against client financial and calorie budget’ General Principle Design Problem of the slow and inaccurate client assessment of ordered goods’ costs against their financial and calorie budget (ineffective performance) might be solved by a virtual shopping cart, which deducts the financial and calorie costs of ordered goods against a client-specified financial and calorie budget (effective performance). The General Principle Formal Design Knowledge supporting the general Principle Formal Design Practices would be expressed as a Formal General Principle.

Design Research Exemplar Illustrations

Towards Engineering principles for Human-Computer Interaction

Engineering Design Principles: Validating Successful HCI Design Knowledge to Support its Re-use

Auto Draft 150 150

Auto Draft