Introduction

 

Review

Videogame Engagement as a Process of Seeking Cultural Value

 

Empirical investigations of videogame play and videogame engagement are often delimited along demographic or genre lines. This paper summarizes an attempt to generate a theory of videogame play and engagement which is not restricted to arbitrary factors of types of players or types of games. In order to achieve this theory a version of Classic Grounded Theory (Glaser 1978) was employed.

The result reveals a highly generalized theory: that players engage with games if they can find a sense of net personal cultural value as they select, play and reflect on their play experiences. The theory is presented and explained and the contributing hypotheses are also presented and explained.

In conclusion it is felt that the methodology has produced a theory with reasonable fit and relevance, suggesting some utility to the fields of Game Design and Videogame Research. Further work is suggested which will clarify and possibly modify the theory to increase the perceived fit, relevance, and utility.

Categories and Subject Descriptors: K.8.0 [Personal Computing]: General

General Terms: Games

Additional Key Words and Phrases: engagement, qualitative analysis, flow, fun, videogames, identity, culture, Pragmatism, Grounded Theory Methodology

1.    INTRODUCTION

This paper summarises the findings of a research programme that set out to empirically create a theory relating to individuals’ experiences of videogame playing.

John Long Comment 1

With the perspective that many contemporary empirical theories are too narrow in focus (e.g. Malone 1981 studied only elementary school children) , methodologically inappropriate (e.g. Sweetser and Johnson 2004 report having based part of their work on a “Grounded Theory” analysis of a 4 participant focus group), or place undue weight on folk developed assumptions (e.g. Brown and Cairns 2004 seem to take as their starting point that “immersion” is the ultimate objective of videogame players) and that a data driven approach with minimal a priori assumptions relating to types of games, types of players, or proposed engagements and objectives might have a chance of arriving at a useful theory with broad applicability, a data-driven methodology was selected, interpreted and employed.

Comment 2

The methodology employed was an interpretation of Glaser’s (1978) Classic Grounded Theory Methodology (CGT), as Glaser positions CGT as a methodology that, if applied correctly, should produce a global dependant variable or central hypothesis supported by contributing variables and sub-hypotheses in a data-driven or empirical manner, that should account for most of the variation found in data related to the domain of study. As this methodology was employed then the resulting theory is a highly generalised concept accounting for players’ reported experiences of engaging with videogames, but with a systematic connection to sub-hypotheses and ultimately data related to the domain. It is hoped that in ‘grounding’ the hypotheses in information about our chosen domain that the theory developed can clearly account for the domain rather than account for arbitrary or ‘grand’ theory.

Comment 3

Sections later in this paper summarise the resulting global hypothesis and the sub-hypotheses that contribute to it, and in order that the reader is clear about how this theory was derived the following section explains the particular interpretation of Classic Grounded Theory (CGT) employed. The concluding sections of this paper explore if the utility of the theory with respect to the fields of videogame design and player research, and if such a general theory or the supporting hypotheses can be further modified or reformulated to be of greater utility to interested audiences and if so how. These concluding sections also attempt to place the theory in a broader theoretical context.

 

Ultimately, the contribution of this work is felt to sit in 2 areas. The first is in applying CGT in a domain quite different from those it might normally be applied to.

 

Comment 4

 

The other main contribution is to forcefully express the general hypothesis that players are seeking culturally expressed value though cycles of positive and negative identification with videogame play experiences, and that this value sum drives engagement.

Comment 5

 

2.    INTERPRETATION OF CGT METHODOLOGY AS EMPLOYED

2.1  Overview of Grounded Theory

For various reasons the term Grounded Theory (GT) is applied to multiple research perspectives, including a form of analysis applied to qualitative data (e.g. Sweetser and Johnson 2004) or a means of analysing the behavior of individuals relative to a specific hypothesis (Brown and Cairns 2004; Fabricatore, Nussbaum, and Rosas 2002), delimiting the domain according to a priori hypotheses about what is important. Early in the programme of research which forms the basis of this paper the decision was made to understand the methodology in the broadest sense, and hopefully to develop a theory inductively (though perhaps more accurately abductively) derived from the domain of people playing videogames. This ‘inductive’ approach is most forcefully expressed by one of the co-originators of the term ‘Grounded Theory’, Barney Glaser (BG Glaser and Strauss 1967; BG Glaser 1978; B Glaser 1992).

Comment 6

In this version of GT no a priori hypotheses are formed, as the objective of the methodology is to form hypotheses based on available data rather than to validate existing theory. In order to achieve hypotheses about the domain a methodology encompassing data collection, data analysis, and theory formulation is proposed.

Comment 7

 

There are several methods within the methodology, and the understanding of those methods as they have been applied in this research are summarized here in order that the reader can both understand where the theoretical concepts came from and how this research might be differentiated from other similarly labeled work.

 

 

There are 5 methods or activities which make up the CGT methodology:

•            Data collection

•            Comparative coding

•            Theoretical ‘memoing’

•            Sorting

•            Writing

 

Each method is intended to move the research from information about a domain to a coherent theory about what is going on in that domain.

Comment 8

These methods are not linear, sequential activities but methods which apply in different proportion at different times. The following subsections will describe how and when they are used while also describing how these methods were employed in the research described in this paper.

For reasons of brevity no attempt will be made in the following text to explore the merits of the methodology from an epistemological basis, rather the following subsections are provided to allow the reader a means of evaluating how the theory was derived in order to differentiate this research from other similar attempts. For a critique of the Grounded Theory methodology see Bryant (2007).

Comment 9

 

2.2  Data Collection Method

Any information which is directly collected from the domain of study or is unequivocally concerned with that domain is useful and should be included. So where the thoughts and actions of people are concerned we might include formal interviews, informal conversations, focus groups, overheard conversations, diaries, and possibly observations, or applicable statistics, while other less textual sources might also provide useful insights. Deciding what to collect and when to use it is determined by the shape and direction of the current theory and progress of coding and memoing (see below). This ‘theoretical sampling’ approach helps to provide a degree of parsimony in the amount of data collected, as in linking data collection to data analysis and theory formation helps to ensure that only as much data will be collected as required. Relative to the process of coding (below) there are essentially two types of targets for sampling: ‘new’ kinds of case (by which we hope to generate new codes) and ‘similar’ kinds of case (by which we hope to flesh out the properties of existing codes).

 

The research reported in this research started by interviewing by opportunity (friends, relatives and colleagues), attempted to explore diary and observation data, further interviewed specific individuals (non-players, more ‘casual’ or more ‘hardcore’ players, and an increasing number of strangers with disparate tastes ), and included a few field noted observations about overheard and informal conversations. The total number of individuals who contributed either distinct codes or an illustration for a particular memo (post coding) was in the order of the mid 30s. The data was in the form of transcribed interviews, recorded but un-transcribed interviews, recorded observations, and field notes (the diaries proved unproductive).

Comment 10

 

2.3  Comparative Coding Method

The GT methodology grew out of research by Barney Glaser and Anselm Strauss (1967) which utilized a process they knew as ‘Constant Comparison’. This process is advocated as the coding method for CGT by Glaser (1978). The codes generated in CGT then are initially derived from comparing data to data. If an apparent part of the data appears to have a relationship with some other part then the nature of this relationship is noted as a code. Most simply then, codes are categories of data or properties of already identified categories. At a more complex level codes can be compared themselves producing meta or ‘theoretical’ codes. Coding is an attempt to reframe raw data, making the theory fit multiple cases rather than single interesting occurrences.

 

Codes were created in two ways in the research reported here, early in the programme transcriptions of the data were marked with subjective observations, which further into the programme (once the constant comparative emphasis was more clearly understood) were clustered into categories and properties which were subsequently added to in further iterations in the constant comparative manner described  above. Few theoretical codes were created, but rather theoretical memos were created which accounted for the comparisons between existing codes. This way of coding with memos rather than specific theoretical codes was due in part to the software employed (Atlas.ti Anon. 1993), which made comparative codes or properties, and especially theoretical codes of codes, a little tricky, but in using memos to create theoretical codes rather than a specific ‘theoretical code’ facility of the supporting software, the result is assumed to be the same.

 

2.4  Theoretical Memoing Method

In the jargon of GT ‘memoing’ is the activity of interest, the main material of the methodology if you will. As the researcher iteratively collects data and codes it, and as they sort and write their outputs they should be constantly capturing each hypothesis they have about what it all means and how it all fits together. This central act of memoing drives every other activity. The researcher finds what to sample for next based on their theoretical observations about the codes they have generated, when to stop collecting and coding data based on how their memos are filling out, and it is the memos which are arranged (and further complemented) in the sorting process, which then yields a structured collection of memoed theoretical ideas to be written.

 

Memos are critical to two milestones found in the methodology. At some point the researcher will come to believe that their data collection and coding seem to be about a particular code. As the memos coalesce about this code the researcher will come to  conclude that they may have identified what the domain may be about (in the jargon of the methodology they have identified the ‘core category’). Once this milestone is met the research moves from collecting data and coding openly for all possibilities and starts collecting data and coding specifically to selectively generate theoretical ideas about this code. The stopping rule for these selective iterations of coding is that once the researcher is no longer generating any new theoretical ideas they might be said to have theoretically saturated the core code. That is not to say that new codes might not be being generated by further iterations, but that as the researcher continues to sample, in accordance with their emerging theory new codes are interchangeable with old. Thus listing out all possible types of subject, perspective, context, tool, strategy, or whatever is not the point and developing categories of only those things that contribute to the emerging theory in terms of new theoretical ideas are important features in ensuring that the theory is developed parsimoniously. We might also say that in recognizing that not every case can be included we are leaving opportunity for any resulting hypothesis to be logically falsifiable.

 

This research recorded theoretical memos in the appropriate function of the software employed. Generally these memos consisted of short notes about what the codes might represent, as well as relationships between codes and possible targets for data collection. The core category selected for saturation by selective coding related to players’ felt identities and how these identities manifested as roles through which the player ascribed value to different game features. Memos were also raised relating these ideas to general theories drawn primarily from Social Psychology where appropriate, especially during selective coding and sorting. As explained below, the sorting process showed this concept of valorization of game features according to a player’s self sense to be somewhat inadequate in accounting for all the theoretical ideas raised, and as such was duly extended.

Comment 11

 

2.5  Memo Sorting Method

Once the core category is felt to be suitably saturated, the collection of memos is not expected to be in a state that would allow for immediate publication, rather while most of the memos are expected to implicitly relate to the core category or how the core category explicitly relates to others, these relations are likely to lack a structure suitable for writing up into a clear publication. There are likely to be gaps and inconsistencies which will need to be dealt with before writing can happen, if one intends to present a coherent theory rather than an incoherent collection of observations. Sorting then is the process of creating a framework for the intended dissemination of the research findings and is performed in order to make as many of the theoretical ideas work towards explaining the derivation of the core hypothesis as possible.

 

It is likely that new comparisons will be noticed in the act of sorting and as such the process of memoing continues throughout. It is also possible that gaps exist that require some further rounds of data collection and selective coding. It is also possible that the core hypothesis may well need modifying in order to account for more of the theoretical ideas and codes than previously realised. In this sense sorting is critical and is not entirely equivalent to the process of expounding generalised observations which often occurs in ethnographic work (e.g. Carr 2005).

 

In this research the pre-sort core category which related to a players sense of identity and assumed socio-cultural role relative to game features was felt to be somewhat inadequate, in that that concept failed to account for the mass of data, codes and thus theoretical ideas relating to the cyclical process of engagement. As such the sort revealed that it was more reasonable to talk of players’ cycles of identifying with games at a feature level, which is the theory presented here. The sort was physically accomplished by printing out the electronically captured  memos, complimented by hand written memos which were raised during the sort, which were repeatedly placed into piles until almost every memo was included and a writable structure of chapters and subdivisions was visible.

 

2.6  Theory Writing Method

After sorting, writing then is not the process of structuring an argument as much as it is the process of laying out the sorted theoretical memos in a text, ensuring that the connections and derivations are made clear for the reader. Also in this process other theories are related to the presented theory (which is also possible in the sort, where general theoretical ideas might help to contextualise the saturated theory).

 

As such the reader can assume that the sections of this paper that set out the theory are in fact directly representative of the sorted memos expanded upon and linked; this paper being a summary of a much more comprehensive thesis which literally contains all the expounded memos.

Comment 12

 

Comment 13

 

3.    The Developed theory

Comment 14

As proposed above, the following subsections represent the theoretical memos as an integrated text, with reference to specific data where necessary (and as space allows). Starting with the contributing hypotheses and leading to the composite or core hypothesis will hopefully allow the reader a means to evaluate the theory clearly that a top down presentation might obscure.

 

3.1  Process of Engagement

A major observation to make about player engagement is that it apparently does not happen as a singular event. The following subsection expands on the interim report published XXXXXX in which an early understanding of the methodology and early findings was published. The following differs from that published work in that the phases or stages were slightly different in the XXXX publication. What is common is that there is a phase of engagement that occurs before play, and the difference between the two presentations is due to greater saturation and borne out of a formal sorting process.

 

Essentially this sub-hypothesis is that there are 3 indistinct phases of engagement: Selection (before hands-on interaction); Play (actual hands-on interaction); and Reflection.

 

3.1.1    Selection

The mechanisms employed to select games are complex and depend on the particular individual and their sense of identifications. Tying the cycle of engagements to the sense of identity will be explored later in this report, in the section dealing with the core hypothesis. This sub-section and the sub-sections relating to playing and reflecting will focus on generalized patterns and procedures employed by individuals as they engage with a proposition.

 

Selection itself can be broken down into broad strategies, situated within contexts:

3.1.1.1  Selection of the singular activity of ‘videogame play’

Firstly we can talk of prospective players selecting videogame play, in current forms, as a potentially agreeable activity. This global point of selection can be best seen in the attitudes of those who reject videogame playing outright. Such individuals expressed attitudes suggesting that for some videogaming represents a male, juvenile, sedentary and solitary activity which is not for them, seeing themselves as variously adult, active, social and not male individuals. While some interest was expressed in novel developments in the products which militate the existing perceptions of gaming (primarily Nintendo’s attempts at introducing motion control and marketing which focused on social settings and players who were not necessarily male or juvenile), the non-gamer subjects had not made the investment of time, money, or effort in exploring these possibilities.

 

For those individual who had not rejected videogame play outright the data reveals a number of strategies employed and perspectives on what videogame activities they might actively seek. These selection criteria reach into a huge range of potentials for play, and are not simply the user finding an agreeable narrative or representation which is might be an easy assumption to make (Juul 2010). The following subsections explore some of the ‘whats’ or pre-play engagements made based on activities sought and some of the ‘hows’ or strategies employed in ascertaining these potentials. These factors will be revisited when discussing the derivation of the core hypothesis, later in this paper.

3.1.1.2  Selecting for an explicit context

Games are not played in a laboratory environment; they are played in a real-world context. Potential players often account for potential contexts of play and select games based upon those contexts. The data relating to the ways players recognize possible contexts of play before actual play occurs seem to be driven by primarily social factors.

 

That isn’t to say that prospective players are always seeking experiences which they can share with others, though this is not uncommon. Prospective players also recognize that there may be occasions when they might want an involving experience requiring an extensive commitment in terms of time and concentration possibly during unavoidable periods of solitude. In this sense a player might be looking to become ‘immersed’ in a game (though the term ‘immersion’ was only used by a single individual in this research) as a means of passing the time or avoiding boredom. These ‘anti-social’ sentiments are not shared by all; other subjects suggested that recognizing the potential commitment necessary in order to play certain types of games is the reason that they reject many videogaming activities, preferring to invest these resources in more ‘productive’ pursuits; a sentiment which will also be covered in more depth in later sections.

 

More social contexts are selected for when a player can imagine playing a game in the presence of or along with other players. As such a prospective player might select a game with performance or multi-player features. While a player might never actually get chance to play the game as a performance, or collaborate or compete with their peers, that a game provides the possibility is often a positive factor. Recognizing the possible tastes of witnesses or co-players is important in helping the prospective player determine the value of the game for social play, which will also be explored in the section of this paper which deals with ‘kinds of players’.

 

3.1.1.3  Selecting Specific Features

In selecting for a specific context we might expect a prospective player to be investigating the purported features of a game. Features which have a bearing on suitable contexts are not the only ones noticed. Prospective players explicitly or implicitly consider a great many design features. While ‘surface’ features are commonly attended to as suggested by Juul’s suggestion that a prospective player is first drawn to the ‘fiction’ of a game (2010), respondents also discussed ‘deeper’ features such as the type of challenge offered. One specific subject explicitly stated that he would eschew any game which might test his dexterity, preferring to engage in intellectual puzzles instead. Interviewees expressed such targets as graphical style and quality, game mechanics, activities including any overarching story or narrative, and challenge type. In fact it seems that any designed feature may be noted by a prospective player and used as a means of differentiation between possible offerings.

 

3.1.1.4                Selecting the Familiar

Selecting games according to familiarities seems to operate in two ways, selecting familiar game related features and selecting according to features not immediately related to games.

When a prospective player is selecting features based on their past experience of playing other games they are clearly drawing on their reflections about past experiences of play. This construction of predispositions is also noted by Carr (2005), and might be said to have been predicted by Pragmatist theories of engaging with pleasurable artifacts such as those of Dewey (1934). This act of selecting a game which promises experiences similar to those enjoyed in the past might account for the success of sequels, though obviously not all reflections are positive and can thus turn a prospective player off a certain set of features as well as on to them.

Another interesting facet of selecting according to the player’s past experiences is where a prospective player selects a videogame based on factors external to their videogaming experience. This is usually in the prospective player identifying with the subject matter (or fiction) offered by the game and may account for the successes of sports related properties and games based on films and television series.  That is if a player feels that they are a fan of Football or Batman say, then they are more likely engage with games which include such themes. This principle can also act in the opposite direction, with familiar themes that the individual does not identify with acting to drive down the degree of engagement a prospective player has with the concept. For example, one interviewee expressed a dislike of Boxing as a justification for not liking beat-em-up style games. In fact he expressed that he was not someone who enjoys watching Boxing and so wouldn’t be someone who would like fighting games, which seems to be a sophisticated expression of identity, which will be covered later in this paper.

 

3.1.1.5  Selection Based on Trusted Opinion

The previous sub-sections dealt with what kinds of things prospective players might be evaluating as they select games to play. The following subsections deal with how prospective players get their impressions of games they haven’t yet played.

 

A clear source of information about what a game is like to play is to consult the opinions of those that have already played it. These opinions could be obtained from peers, reviews in the media, or other ‘expert’ opinion. In social groups where games and game play was seen as a valid topic of conversation information gleaned from the opinions of peers was most valued. However several subjects suggested that gaming was not often a valid topic of conversation, and so one of these subjects had formed a relationship with a clerk in his local game shop where the clerk had learned his tastes to such a degree that he trusted the clerk’s recommendations. Where media reviews were concerned, among the subjects that suggested that they did read such things there was a general impression that they were not as well trusted as peer recommendation, but were never the less used as a source of information about the features and overall quality of a game.

 

3.1.1.6  Selections Based on Marketing

Information sourced directly from the producers or publishers of videogames is another means by which potential players find out if a game might offer a suitable play experience. This could be information from the company websites of the game producing or publishing companies, media advertisements, media preview editorials, or even the game packaging. The amount of information sources consulted seems to loosely correlate with how much the prospective player identifies themselves as a game player. ‘Hardcore’ players may be aware of release dates and proposed features at a fairly fine grained level, while data from players at the other end of the hobby/commitment spectrum suggests that these players might only consult the game packaging as they browse games in a retail outlet.

The extent to which the player has investigated the promised features of a game may well influence their commitment at later phases of the engagement process. For example a player who might describe themselves as ‘hardcore’ who has tracked the development of a game from announcement through to sale, and who may well have engaged in the online fan community concerned with that specific offering, discussing hopes and fears for the final product, is less likely to give the game 2 minutes of their time before permanently deleting it from their hard disk (as the subject who downloaded games based on their title alone suggested he would).

That some less hobbyist players select games based on packaging, more often than not, suggests that they have very little understanding of the features of a game other than a theme and the positive description of the features commonly summarized on the packaging. The subjects who suggested that packaging information was their primary source of information seemed to select games by their theme (or ‘fiction’) more than any other features, even though this had in the past lead to disappointing play experiences.

 

3.1.1.7  Selection by Provenance

Where a game comes from can provide important information to a prospective player in helping them determine if it might be engaging. That is information about who made or published the game or who owns a copy of the game or gaming product can push up or pull down the engagement a player has with a game before they play it. If the game was developed by a team responsible for games that the prospective player is fond of, or the game is found in the collection of a friend the prospective player considers to have good taste, then the player is more likely to be engaged by the prospect of the game.  Conversely if the game is developed or published by a company the prospective player considers to be producers of bad games or the game is found in the collection of someone considered to have a poor taste in games, then this provenance might act to drive down the individual’s engagement with the prospect of playing the game. Those prospective players who might describe themselves as gamers are more likely to know who produced a game and judge it on this knowledge, but such knowledge is also held, to some degree, by those who play more casually. For example one ‘casual’ subject suggested that Nintendo are more likely to produce games which are more aligned to what they are personally seeking than other publishers. Less hobbyist or ‘hardcore’ players are likely to trade games amongst their peers as a means of determining quality, essentially pooling agreeable games.

 

3.1.1.8  Selection by Availability

Often players might make no conscious decision to obtain a game; it is simply there. In this case the only decision the prospective player must make is whether to ‘have a go’ or not. In these cases many of the material costs are removed (such as time, effort or money spent to obtain the game) and the decision then only rests on whether the user feels that there might be other costs involved (embarrassment at playing a performance game in public say) relative to the benefits of playing (using our performance game example they might feel that in playing they become more socially connected to the other players). Where the context is less social (for example where the game is obtained cheaply, maybe as a bundled software product with a new device), the low cost of entry might have the player ‘give it a go’ where otherwise they might not. Judgment and engagement then rests on the later phases of engagement.

 

3.1.1.9  Selection by Trying

All other selection methods and criteria considered, there will be a point where the player starts playing the game. At this point engagement seems to go through a period of evaluation. Does the game meet up to expectations?

 

There is no clear cut off between a player’s initial evaluation and when they might be said to be playing ‘properly’, but there is enough evidence in the data to suggest that on occasion players have tried a game, decided that it wasn’t for them and stopped playing forever. Sometimes this is because they have encountered a game in a context that is not conducive to them seeing the benefits of continued play (such as one subject feeling that a game was far too hard to bother with having encountered it with players who were far more skilled than themselves and thus he became frustrated with his lack of skill), but more often it is simply that a game didn’t deliver what the prospective player imagined it might before they actually sat down to play it. Sometimes a player has minimal expectations and finds pleasure in their initial encounters. Sometimes though this pleasure is context dependant (such as individuals who wouldn’t normally play games, joining in with a group playing a game conducive to multi-player, party like activities), and once that context doesn’t exist anymore nor will they play anymore.

 

Occasionally though the player will find enough of what they thought they might get from the experience to remain engaged and to continue playing.

 

3.1.2    Play

While researchers such as Aarseth (Aarseth 2003) have argued that play must be the central object of study for games research, this project has essentially settled on a study of the conditions supporting engagement in play. That is the actual act of playing is bound into a social psychological praxis which informs the conditions of engagement; the actual engagement itself being a successful realization of the supporting factors of identification, expectation, context and so on. This is due in part to the differences in methodology, where the methodology used here deals with the heterophenomenology of reported player experiences Aarseth has traditionally focused on the artifact and their imputed meanings explored via personal play. That is much games research deals with the game and how it facilitates play while this research has developed a theory of how and why players make the choices they do; what experiences do games provide vs what kinds of experiences are players seeking to engage in. These are two sides of the same question.

 

As part of the process of selection, play and then reflection, the actual playing of the game is most simply stated as the period where a player considers them self to be an active player of the game. What factors hold them there for a session, or has the individual return for another session of play, are dealt with more completely in other sections of this paper. In terms of the phased process of engagement similar factors to those involved in game selection are constantly evaluated against the specific variable context during play, and if the weight of those factors becomes insufficiently positive then the player will stop playing. For example if the social situation changes to one that is insufficiently agreeable then the play may well stop to accommodate this change. Similarly other less dramatic changes might amass to stop play such as fatigue or hunger, or a player might have other concerns such as chores or work the time for which the play activities might be eating into. This is also alongside the possible changes within the game. The game might become too repetitive or too challenging for the player’s current state of mind, and this too will drive down the motivation to continue to play.

 

The conception of the motivations and de-motivations to play a specific game presented here is different from other conceptions which focus on such motivations as ‘immersion’ (Brown and Cairns 2004) or Flow (Cowley et al. 2008) as the ideas presented here should also account for players who are not looking for such deeply engaging experiences, as well as those that are. Indeed the data collected suggests that some players who have experienced the effects of ‘immersion’ or Flow like experiences in the past now reject many games or gameplaying contexts as they feel that losing track of time (say) with a game is more destructive and futile than beneficial and productive. As such some players deliberately seek out games which are not likely to take hours of their life at a time; games which are easy to pick up and put down.

 

3.1.3    Reflection

A player who has selected a game to play and has played it may continue to engage with the game afterwards. This engagement will take the form of explicit or implicit reflection. The player will be considering if playing that game was a positive or negative experience. They might even discuss the merits of the game amongst their social group. Indeed much of the data used in this research is essentially the reflections of players. It is apparent that while a player may select a game experience and play it, they might decide, on reflection that the experience overall is not worth repeating. Other non-negative reflections are related to the relative merits of particular games and will result in realizations which are fed back into future selections. Precisely what is being reflected on is explored in the following sections.

 

 

3.2  Identification with features

In the previous sections regarding the cycle of engagement a few hints are given as to what drives player engagement in this process. In general terms it seems that for each feature at each phase of the engagement the individual is determining if they are the ‘kind of person’ who might engage in such a game with such a feature. This identification operates for multiple pertinent features and seems to be summed or massed for any whole product. This sub-hypothesis then should help us understand how different people engage in different games, as if a player feels at any point that their perception of the fiction, graphical presentation, challenge type, and other features results in an overall positive engagement then they will be likely to play, where as if the same features are perceived, in summation, negatively then they are not likely to play.

 

One powerful example of how features are perceived in a socially relative personal way is of the adolescent subject who extensively played a certain JRPG (or Japanese Role-Playing Game) but felt the ‘super-deformed’ graphical style employed in much of the game was ‘babyish’. That is he seemed to feel that the graphical style was more suited to an audience younger than himself, but in summation the other features of the experience were sufficiently aligned to his cultural understanding of who he was and what he should be playing, to allow him to engage with the game despite the ‘babyish’ graphics.

 

What is also apparent from the data is that different individuals perceive the importance of features differently in terms of the weight they ascribe to these features. For example while one individual finds a degree of difficulty which will challenge their skills off-putting other players will deliberately play a game at its hardest setting as a personal challenge and would become fed up with a game at which they were always successful. I have related these weighted positives and negatives to a loose conception of ‘costs’ and ‘returns’. Costs might be loosely separated into material costs (money, space, portability, time commitment required) and social or cultural costs (is there a sense that in playing this I will perceive myself badly or I will be perceived badly by others in this context). Returns might be that a player is obtaining a ‘fun’ experience, whatever the particular user deems an acceptably fun experience to be (getting some exercise, inspiration, obtaining knowledge about the state of the art, experiencing an interesting narrative, and immersing oneself in an alternative world were examples encountered). Material returns are less difficult to suggest that they used to be. With the relatively recent introduction of motion control it seems that some players are interested in the fitness aspect which is used to market some products. Similarly self improvement and mental agility training game types are also apparently popular, suggesting that some players are looking for extrinsic returns such as enhanced mental fitness. This cost/benefit aspect of engagement suggests that the degree of overall engagement with a product could be said to be an aggregation of feature relative positions, or a summation of costs and returns to a net sum of overall ‘cultural’ (socially relative, personally expressed) value.

 

This socially developed sense of ‘kinds of people’ and what behavior is acceptable for such, which feeds into the cost/benefit sum, could be related to such Pragmatist ideas as  Cooley’s ‘Looking Glass Self’ (1902); the theory that an individual’s sense of self is constructed by subjective reasoning about how the individual imagines they might be perceived by others in their society or immediate social context. We might say that so constructed the individual will behave in a way that seeks to reinforce this identity and seek to minimize any possibilities that they might be perceived poorly. We could also suggest that these expressions are expressions of cultural values (where an individual has learned suitable modes of conduct from their social interactions). That is not to say that an individual will embody all socially dictated grand cultural values (such as an abhorrence of murder say), though some of these types of value might impinge on some players’ engagement (and as such some players will state that they are uncomfortable playing a game where one plays at murder) but personally acquired, fine grained values (such as playing a game with cartoony graphics will reflect badly on an adolescent boy, but may reflect less badly on a woman in her 20s, or even that owning the latest videogame console reflects badly on a male dancer in his late 20s who believes that games are for dullards or a fashionable, female student in her teens who believes that games are for boys, but a technology savvy male, computing student in his 20s will feel remiss if he didn’t live in a house with all the latest hardware).

 

3.3  Value Seeking Process

In combination the hypotheses set out in the above sections suggest that players are engaging with games as collections of features with ascribed cultural value which are constantly evaluated and negotiated and the values summed throughout a course of engagement, from before the game is actively played, through active playing, to reflecting on the experience. The sense of cultural value is realized as a type of socially relative personal identification. So when an individual asks “Am I the kind of person who would play this game?” they are also asking “If I play this game, what does that say about me?” and “If I saw someone playing this game, what would that tell me about them?”, quite similar to Cooley’s ‘looking Glass self. So at each phase of engagement these implicit questions are being asked in slightly different ways.

 

3.3.1    Selection as investigating and finding potential positive cultural value

The space of potential gameplay offerings is not fully known by any individual, rather they form impressions of what offerings exist and what the nature of those offerings are from a variety of sources. These impressions are then contrasted with their sense of identification to determine if this activity is possibly one in which the individual feels that they can engage. The source of the information also helps to form this sense of identification, and the impression of the offering might not be formed simply on surface features such as themes, graphics, or characters, but at this stage, for many individuals, these features are more important here than they are at other phases of the engagement. The sources of information and methods used are those discussed in the relevant sub-sections above.

 

Essentially if an individual is sufficiently engaged by the prospect and can reconcile the investments required to play the game then they might seek it out and play it. If the investments are too great then they will play it if the investments are reduced, but otherwise will not seek it out (they are the kind of person who would play a game with those features in principle, but there is not enough time, it would be a waste of money, or it’s not worth upgrading hardware for are stated examples). If an individual is not engaged by the prospect of playing that game, then they will not seek it out or be inclined to play it without a context where the previously considered features become less relevant (not wanting to seem a ‘kill joy’ if everyone else in a social setting is playing together, that is they are not normally the kind of person who would play this game, but in this context they might as well participate and would then find it to be fun for example).

 

3.3.2    Engaging in play as long as a sense of positive cultural value persists

Once a player has reached the point of accepting an offering as suitable or agreeable (that they are likely to be the kind of person who would play such a game or with such a device), they will then be disposed to play it. This engagement as a state of disposition is not fixed, in that it is not such that a player who is engaged by the idea of playing will automatically then set about playing the game ‘fully’ (as the designer intended); rather it is such that the negotiations between the player’s sense of identification, the imagined reactions of their social context, and the actual experience of playing the game are fully initiated. Initially there is a sense of traversal from wanting to play the game to ‘actually’ or ‘really’ playing. This phase might be seen as ‘giving the game a chance’ and lacks a clear end unless the match between expectations and the actual experience of play shows that the game dramatically disappoints the player, at which point the value sum will be negative and the individual will stop being engaged and thus stop playing. We could say that for every new element that is introduced throughout the playing of a game the player will be ‘giving it a chance’, but this is increasingly subtle with the player also having extra investments in play (having spent the time to gain skill, develop characters, engage with the narrative or similar).

 

Once a player has selected an offering and then encountered that offering without being ‘put off’ by a negative sum of identification, they may be said to have recognized it as a game that they would be disposed to playing. However many games are not a simple interaction repeated over and over again but often progressively introduce new elements to the player as the player gains skill, tokens, or progresses through the story or different challenges and levels. As such for many games the player will be constantly evaluating the offering as they go; shifting their sense of value in light of new elements. Even though the terrain of the game is shifting, the player must always feel that they are engaged in an activity of positive net worth or they will stop playing or will not return to play in future sessions.

It is likely that as players move from the negotiated factors of selection to factors associated with play there is a shift of emphasis away from surface factors (thematic, graphics and such) toward ludic factors (game mechanic, challenge and such). That is players might find that they feel that they are the kind of person who would play a game with a particular graphical style say, and as they play the game become less concerned with the particular graphical style and more concerned with the actualities of playing the game; meeting the challenges or progressing through the story for example. This is the position of Aarseth (2004) who argues that the nature of any avatar is likely to fade into the background as a player focuses on ludic aspects of the game as they play. Likewise Juul’s (2010) assertion that the ‘fiction’ of a game is the first factor encountered and engaged with before other elements are considered is not completely rejected.

 

The degree to which they have already formed an identification will influence a player’s degree of perseverance, such that offerings with which the player has formed a strong personal connection (by developing characters or other ‘actors’ and objects, engaging with a story, or developing skill) will be much more resilient to problems such as a particularly difficult challenge, a bug, a displeasing plot direction, or any other unexpected negative experience. That is players can become more or less the kind of person who would play such a game as they acquire or lose any sense of personal connection. Similarly as a player invests resources to make progress in a game they become less likely to disengage until the player feels that this investment has resulted in a payoff, few people like to feel that they are ‘quitters’, however players do not like to feel that they are being forced to repeat gameplay elements they have already mastered or understood. This sense of progress and growth toward an arbitrary goal (a higher score, a new level, the next part of the story) can be related to an interpretation of Flow theory that suggests that ‘skill matching’ does not create engagement without the individual feeling that progress is being made toward a personally meaningful end (Csikszentmihalyi 1990).

 

Another type of development which is particularly true of multiplayer games is that of a social nature. Especially where online play might be concerned, the amount of socialization for some types of games (i.e. Massively Multiplayer Online games or MMOs) acts as a means of reinforcing some users’ engagements or a means of driving down engagement for others. To some players the appeal of having a set of trusted and decent playmates is apparent where the facility to play against others online is available. However, some are likely to find that they are not the kind of person who wants to micromanage other people in terms of competition schedules and training sessions, or that they are not the kind of person who wants to spend a large amount of time engaged in activities to support the play activity they identify with, and will become less engaged as the amount of required social interaction increases.

 

Another factor which might influence a player’s sense of engagement is a shift in context. External factors might have the player become less likely to feel that they are the kind of person who would engage in a game as other life pressures impinge on the experience. Take for example a player who is engaged by the degree to which the game facilitates social play; if the social context changes, for example by play mates ceasing to play, then the game will become less engaging as this feature or factor is less well supported.

 

3.3.3    Reinforcing the degree of sense of value by reflection

It is apparent that past experiences are fed back into future selections. This feedback does not seem to loop directly from putting one game aside to openly selecting the next, rather there appears to be an ongoing period of reflection, which seems to summarize the pros and cons of past experiences resulting in more carefully considered future selections.

 

It seems there are two types of reflection, implicit reflection and explicitly expressed reflection. That is there are times when an individual appears to be forming an opinion that can only be based on their past experiences without necessarily consciously analyzing their experience, and there are times when individuals’ experiences with various offerings can be heard being openly discussed respectively. The act of tacit reflection is difficult to demonstrate other than in the player, when quizzed, relating their preference (or dislike) of new propositions to past experiences, but struggling to put their finger on why they have this value position other than in relation to those same past experiences. It is when a user tries to relate the qualities of an offering to others that the reflective player must make value judgements as to what factors to highlight and espouse or reject. A number of observations could be made about the nature of reflection, but suffice it to say that much of the data used in this programme of research was based on interviews where the interviewer implied that the interviewee should explicitly reflect on their past gaming experiences and engagements. While this might seem introspective and thus a collection of possibly poorly realized subjective reports on interviewees’ tacit knowledge, hopefully the methodology employed has heterophenomenologically arrived at an account with some utility.

 

The less formal interviews in the data (along with some field notes on casual observations made) reveal that in discussing which experiences individuals found engaging, there is a degree of rhetoric involved. Individuals expounding the merits of the experiences they engage with and those that they do not; occasionally attempting to convince the other of the merits or faults of games they have played, which serves to amplify the sense of identification and hence degree of engagement.

Comment 15

 

4.    conclusions

Hopefully, the summary of the theory provided above gives the reader enough information to be able to decide if these hypotheses make sense, and if the main hypothesis of a cyclical process of seeking cultural value fits the explored domain. Obviously the methodology employed focuses on certain aspects of a player’s experience. So no sense of a player’s emotion is considered explicitly, for example. Many apparent omissions are likely due to them being expected in the data due to a priori positions, as for example little mention was made by subjects of their preferred emotional states, other than a game should be fun or interesting and not boring.

 

A small (9 respondent) survey of interested parties asked to review a very brief summary of the results reveals that the degree of fit and relevance is good with two caveats. These caveats are that the short presentation of the theory only seemed to account for extensive cycles of engagement and not one off experiences which might still be deemed engaging, and that the result is an obvious truism. Hopefully this more extensive presentation of the theory helps to demonstrate that to some extent the one-off  interaction is included as an extreme case (where the individual is the kind of person who would play that game in that context, but not necessarily in others), while the charge that the theory is an obvious truism is not supported by the empirical literature, as there seems to be very little which deals with concepts of a process of finding engagement in videogames by a constantly evaluated or negotiated sense of cultural value. So if this theory is a truism it seems to have little impact on much empirical research into player experiences, maybe because it has not been stated clearly.

5.    discussion

A pertinent observation about player engagement is made by Carr (2005). In attempting to account for an observation that girls in a specifically convened female only gaming club Carr notes that “Different people will accumulate particular gaming skills, knowledge and frames of reference, according to the patterns of access and peer culture they encounter – and these accumulations will pool as predispositions, and manifest as preferences.” while “Preferences are an assemblage, made up of past access and positive experiences, and subject to situation and context.”, which seems to be quite closely related to the theory presented here. This observation is substantively different from say those of Malone (1981) and subsequent multi factor theories as it does not state that players engage with Challenge, Curiosity and Fantasy (or some other factors such as novelty and spectacle; excitement of combat; game characters; persistence; exploration; advancement; unraveling of puzzles; building, creating and controlling; humour; relation to one’s hobby or interest; audiovisual quality; imaginary world; and winning (Ermi and Mäyrä 2003)), but gives us some indication of what process a player goes through in arriving at the specific combination of factors that engages them.

 

In terms of related work from other domains McCarthy and Wright (2004) have taken a theoretically driven approach which has arrived at similar conclusions drawing for Pragmatic theory. That is the research presented in this paper was empirical abductive research, where as McCarthy and Wright seem to have used a theoretical deductive approach, but we have arrived at a position which neatly fits into a Pragmatic position. Where the theory in this paper relates the engagement with videogames to Cooley’s ‘Looking glass self’ and notes that engagements are cyclical and sit within a social context McCarthy and Wright take theories from Dewey (1934) and Bakhtin (1993) to arrive at related conclusions about the felt experience of the use of technology in general. One point of similarity is that they also recognise that an engagement is not simply a single (or repeated) instance of use, but also contains expectation and reflection, placing the experiencing subject and the object of experience in the broadest context. The breadth of this context extending out into personal conception of personal meaning and cultural value of the individual’s life, not just their life related to the technological artifact. This consideration of the overall felt experience (history, reflection, context) of the individual then determines the degree of meaning and value they then apply to the experience they are having as they have it.

 

So is a theory that players must be able to find the value in playing as a kind of investment of cultural negatives in order to make net cultural gains useful?

As a means of reframing the problem of Game Design to consider the broad cultural context of different player perspectives, some surveyed games professionals have shown an interest. However it seems apparent that some sectors of the industry are well aware that users must identify themselves as potential players and are already extending their thinking to traditionally under exploited kinds of players (e.g. Nintendo’s successes with products  that purport to promote mental and physical fitness, and play in a social context). Certainly recent attempts at marketing games seem to focus on the player as much as the game, in a seeming attempt to demonstrate to the user what kinds of people would be players of the games and devices being released.

 

As a means of framing understanding of what engages players, it could be suggested that the core hypothesis is too broad; so inclusive as to describe all human experience of entertainment products at a macro level, with little to say about individual cases of subjective engagement. However the supporting hypotheses and how they interrelate could be said to provide a meso level description of engagement which hopefully helps us understand individual cases more clearly.

 

It is obvious that more work is required. The summary presented in this paper is necessarily quite brief. It is immediately apparent that each contributing factor or hypothesis could be explained in much more detail, with reference to the data from which it was derived. An effort will be made to disseminate these detailed descriptions at a later date. Similarly while an attempt was made to saturate a small number of critical hypotheses, other hypotheses which might be of significant interest to the fields of design and research are relatively under saturated; as such further research to flesh out these ideas will be needed. An attempt to translate these theoretical findings into some practical artifacts would both help to validate the theory to some extent and to provide further operational information for design practitioners.

6.    REFERENCES

AARSETH, ESPEN. 2004. ‘Genre Trouble: Narrativism and the Art of Simulation.’ In FirstPerson: New Media as Story, Performance, and Game. Cambridge, MA: MIT Press.

ATLAS.TI SCIENTIFIC SOFTWARE DEVELOPMENT GMBH. 1993. Atlas.ti. Windows. Atlas.

BAKHTIN, M. 1993. Toward a Philosophy of the Act. 1st ed. Austin: University of Texas Press.

BROWN, EMILY, AND PAUL CAIRNS. 2004. ‘A Grounded Investigation of Game Immersion’. In CHI  ’04 Extended Abstracts on Human Factors in Computing Systems, 1297–1300. Vienna, Austria: ACM.

BRYANT, ANTONY. 2007. The SAGE Handbook of Grounded Theory. Los Angeles ;;London: SAGE.

CARR, D. 2005. ‘Contexts, Gaming Pleasures, and Gendered Preferences’. Simulation & Gaming 36 (4) (December): 464–482. doi:10.1177/1046878105282160.

COOLEY, CHARLES. 1902. Human Nature and the Social Order. New Brunswick (U.S.A.): Transaction Books.

COWLEY, BEN, DARRYL CHARLES, MICHAELA BLACK, AND RAY HICKEY. 2008. ‘Toward an Understanding of Flow in Video Games’. Comput. Entertain. 6 (2): 1–27.

CSIKSZENTMIHALYI, MIHALY. 1990. Flow : the Psychology of Optimal Experience. 1st ed. New York: Harper & Row.

DEWEY, JOHN. 1934. Art as Experience. Perigee Trade pbk. ed. New York: Perigee Books.

ERMI, LAURA, AND FRANS MÄYRÄ. 2003. ‘Power and Control of Games: Children as the Actors of Game Cultures’. In Level Up: Digital Games Research Conference, 234–244. Utrecht.

FABRICATORE, CARLO, MIGUEL NUSSBAUM, AND RICARDO ROSAS. 2002. ‘Playability in Action Videogames: A Qualitative Design Model’. Human-Computer Interaction 17 (4) (December): 311–368. doi:10.1207/S15327051HCI1704_1.

GLASER, BARNEY. 1978. Theoretical Sensitivity: Advances in the Methodology of Grounded Theory. Sociology Press, Mill Valley, CA.

———. 1992. Basics of Grounded Theory Analysis. [S.l.]: Sociology Press.

GLASER, BARNEY, AND ANSELM STRAUSS. 1967. Discovery of Grounded Theory. Sociology Press, Mill Valley, CA.

JUUL, JESPER. 2010. A Casual Revolution : Reinventing Video Games and Their Players. Cambridge  MA: MIT Press.

MALONE, THOMAS. 1981. ‘Toward a Theory of Intrinsically Motivating Instruction’. Cognitive Science 5 (4) (October): 333–369. doi:10.1207/s15516709cog0504_2.

MCCARTHY, JOHN, AND PETER C. WRIGHT. 2004. Technology as Experience. Cambridge  Mass.: MIT Press.

SWEETSER, PENELOPE, AND DANIEL JOHNSON. 2004. ‘Player-Centered Game Environments: Assessing Player Opinions, Experiences, and Issues’. In Third International Conference, Proceedings. Eindhoven: Springer Berlin / Heidelberg.

 

Introduction

Temporal, Affective, and Embodied Characteristics of

Taste Experiences: A Framework for Design

Marianna Obrist1,2, Rob Comber1, Sriram Subramanian3, Betina Piqueras-Fiszman4, Carlos Velasco4, Charles Spence4

 m.obrist@sussex.ac.uk | rob.comber@ncl.ac.uk | sriram@cs.bris.ac.uk |

betina.piqueras-fiszman@psy.ox.ac.uk | carlos.velasco@psy.ox.ac.uk | charles.spence@psy.ox.ac.uk

1Culture Lab, School of Computing Science

Newcastle University, UK

2School of Engineering and Informatics

University of Sussex, UK

3Deptartment of Computer Science,

University of Bristol, UK

4Department of Experimental Psychology,

University of Oxford, UK

ABSTRACT

We present rich descriptions of taste experience through an

analysis of the diachronic and synchronic experiences of

each of the five basic taste qualities: sweet, sour, salt, bitter,

and umami. Our findings, based on a combination of user

experience evaluation techniques highlight three main

themes: temporality, affective reactions, and embodiment.

We present the taste characteristics as a framework for

design and discuss each taste in order to elucidate the

design qualities of individual taste experiences. These

findings add a semantic understanding of taste experiences,

their temporality enhanced through descriptions of the

affective reactions and embodiment that the five basic tastes

elicit. These findings are discussed on the basis of

established psychological and behavioral phenomena,

highlighting the potential for taste-enhanced design.

Author Keywords

Taste; user experience; taste experiences; sensory research;

explicitation interview technique; sensual evaluation tool.

ACM Classification Keywords

H.5.2 Information interfaces and presentation (e.g., HCI):

Miscellaneous.

INTRODUCTION

Experts in taste perception agree on at least five basic tastes

[40]. Beyond this, however, we lack insights into the rich

experience of these tastes. This lack of experiential

understanding

Comment 1

extends beyond HCI, as sensory researchers

have also acknowledged that: What is not well researched

is the link between the food that goes into our mouth and

what we think of it [12]. There is a growing interest in taste

within the HCI community [e.g., 16,17,18,22,27,28],

particularly relating to technical challenges in designing for

taste stimulation and one-off designs to enhance user

experiences through the manipulation of taste.

There is a need for a more systematic study of people’s

taste experiences and their specific characteristics in order

to make a fuller use of this sense in future taste-enhanced

technologies. This paper stands as a first step in addressing

this need. Drawing on neuroscience and sensory research in

combination with user experience evaluation techniques,

we investigated how all five basic tastes are experienced at

a given time (synchronic) and how they evolve over time

(diachronic). We used pure tastants (i.e., that have no smell

or visual qualities) with an explicitation interview technique

[41] designed to encourage the participants to verbalize

their experiences. Additionally, we used physical objects

from the Sensual Evaluation Instrument [13] to elicit

affective responses, and create a flexible, non-verbal

channel of communication between the user and designers.

This paper makes a number of contributions: First, we

provide a rich description of subjective taste experiences

along both the diachronic and synchronic characteristics of

the five basic tastes.

Comment 2

Second, these taste characteristics

establish a framework for taste experiences and elucidate

the potential design qualities of individual tastes.

Comment 3

We demonstrate how each quality can be described along three

main themes: temporality, affective reactions, and

embodiment. Third, our findings extend human-computer

interaction research on taste through a user experience

perspective. Overall, our findings provide interaction

designers and user experience researchers with a richer

understanding of taste experiences and their specific power

to influence human behavior and decision-making. The

framework presented here enables the HCI community to

think and talk about taste in the design of interactive

systems in a fine-grained manner.

Comment 4

RELATED WORK

This section provides an overview of the human sense of

taste and its relevance for HCI based on ongoing research.

The sense of taste

Sensory researchers and neuroscientists agree on five basic

tastes (sweet, sour, salty, bitter and umami), and a

‘gustotopic map’ linking these classes of receptors with

particular brain areas is currently being developed [40].

However, despite breakthroughs in understanding the sense

————————————————————————————————–

Permission to make digital or hard copies of all or part of this work for personal or

classroom use is granted without fee provided that copies are not made or distributed

for profit or commercial advantage and that copies bear this notice and the full

citation on the first page. Copyrights for components of this work owned by others

than ACM must be honored. Abstracting with credit is permitted. To copy otherwise,

or republish, to post on servers or to redistribute to lists, requires prior specific

permission and/or a fee. Request permissions from Permissions@acm.org.

CHI 2014, April 26 – May 01 2014, Toronto, ON, Canada

Copyright 2014 ACM 978-1-4503-2473-1/14/04…$15.00.

http://dx.doi.org/10.1145/2556288.2557007

——————————————————————————————————-

of taste, scientists have still not approached the

phenomenology of taste nor developed a semantic

understanding of how taste is experienced [30].

Comment 5

 

Although a wide body of sensory research has studied the temporal

evolution of taste perception using labeled intensity scales

[e.g., 1,8] and more specific time-intensity sensory

evaluation scales [26], insights are limited to the

quantification of temporal responses to perceived taste

intensities. Such scale-based evaluations leave us

uninformed as to the subjective qualities that lie behind the

ratings of the perceived taste experience over time.

Recently, neuroscientists have studied taste-specific

temporal profiles by comparing sensory evaluation scales

with functional MRI (fMRI) data [19]. Their results

suggested that salty tastes change more rapidly than sweet

tastes in the cerebral cortex, and confirm the same patterns

that have been observed using time–intensity sensory

evaluation [19]. While such results are intriguing, they

cannot explain the differences in experienced tastes.

Comment 6

To account for subjective differences, the ‘taster status’

measure has been introduced [2,6]. By means of such tests,

it is possible to identify participants’ subjective sensitivity

to bitter tastes and to distinguish between supertasters (25%

of population), medium tasters (50%), and non-tasters

(25%) [3]. Taster status has been considered to partially

explain why some consumers like certain foods more than

others and how they describe the way they experience them.

Comment 7

 

Food-interaction design

The last few years have seen increasing interest in

designing human-food interaction in HCI [e.g., 4,9,11,33].

Such research looks to position human-food interaction

within the wider spectrum of social, environmental, and

physiological influences on our food practices. In this area,

there is a growing realization of the potential for new

technologies to support pleasurable experiences around

food [20,35], and the potential for designers to draw on the

extensive research on multisensory experiences (i.e.,

auditory, tactile, visual, olfactory, and gustatory). Despite

this increased interest in food experience, we know little

about the richness of people’s taste experiences. The

majority of the studies on food experience combine taste

with other modalities, where taste is but one component

[e.g., 18,27,28].

For instance, Schifferstein et al. [31] elicited emotional

experiences across the different stages of food product

usage, from choosing a product in the supermarket through

to cooking and eating [31]. Taste experience is interwoven

with vision, touch, and olfaction, which, in combination

create multisensory food experiences. Desmet and

Schifferstein [5] also explored the emotions elicited through

eating and tasting food. They describe variables related to

food-evoked emotions, such as sensory features, product

type, food-related activities, context, and the agent (who

consumes, prepares, or produces). Due to the wide range of

influencing variables, it is not clear how well these findings

translate beyond the specific context of their studies.

Taste-enhanced technology

Technological advances in creating taste stimulations

[27,28] and one-off applications exploiting taste in games

[16] and other scenarios [18,22] demonstrate a growing

interest in the use of taste in interactive applications. For

instance, Ranasinghe et al. [27,28] developed a tongue

interface that creates taste through the combination of

electrical and thermal stimulation. They use electrical

pulses applied to the tongue. Verbal descriptors provided by

participants were, for instance, a ‘refreshing taste’ or ‘minty

taste’ in relation to the change in temperature. The authors

call for future work to understand the particularities of such

taste (flavor) experiences.

Comment 8

 

They focused on the introduction

of taste in digital communication to enhance long-distance

family relations and create remote co-presence and coliving

experiences (e.g., remote dining) [28].

Comment 9

 

Murer et al. [16] designed a gustatory game device,

LOLLio, which consists of an interactive lollipop that

serves as a haptic input device that dynamically changes its

taste between sweet and sour. Remote triggering of taste

while motion sensing with accelerometers allows LOLLio

to be used as an input modality. The authors identify

various ways in which taste could be used in an interaction,

such as to provide reward or punishment or else to provide

hidden information through taste stimuli. LOLLio was

evaluated in a game context with children [17]. Sweetness

was constantly used in the game session and sour stimuli

were used in combination with game mechanics to provide

‘negative reinforcement’. Their findings suggest an

enhanced playing experience through taste stimulation

motivating further explorations of such taste-enhanced

interaction experiences.

Comment 10

STUDY METHOD AND PROCEDURE

Sensory research provides important information regarding

the objective measures of taste perception, temporality, and

subjective sensitivity levels. Yet, an understanding of the

subjective understanding of taste experiences is missing.

Comment 11

 

This study explores the diachronic and synchronic structure

(explained below) of each of the five basic tastes.

Methodology

For our study, we combine two verbal and non-verbal user

experience and elicitation methods, the explicitation

interview technique (verbal method) and the ‘Sensual

Evaluation Instrument’ (non-verbal method).

The explicitation interview technique [41] is used to elicit

verbalizations of subjective experiences. This technique

helps to explore the unfolding of an experience over time,

the ‘diachronic’ dimension, and examines the specific

facets of the experience at a particular moment, the

‘synchronic’ structure (see also [24,39]). The value of this

interview technique lies in helping participants to express

their experiences at a specific moment. Participants are

encouraged to talk about the experiential (cognitive,

perceptive, sensory, and affective) aspects of the moment

without building on rational comments and explanations

[24].

Comment 12

Questions related to the diachronic structure help to

understand how the description of an experience unfolds

over time (e.g. “What happened after you opened the

door?” and “What did you perceive next?”). With respect to

the synchronic structure of an experience, the participant is

questioned about a particular moment (e.g. “At the moment

when you pushed the handle down, how did it feel?” or

“What else came in your mind?”). In comparison to open

questioning approaches, this technique is non-inducive but

directive [24] in the sense that it keeps the participant

talking about the experience without inducing any content;

it focuses on the structure of the experience, and directive,

as it keeps the participant focused on the singular

experience being explored. Although it is typically used

retrospectively to support the reconstruction of an

experience, it has also been used in-situ (e.g., [15,23]).

The Sensual Evaluation Instrument (SEI) is a non-verbal

tool that can be used to elicit users’ affective reactions [13].

SEI is composed of sculpted objects that can be held in the

hand, used by a person to indicate how they are feeling as

they interact with a system. The SEI includes eight objects

with different shapes, which represent various levels of

arousal and valence (positive and negative). Isbister et al.

[13] describe SEI objects as evoking and expressing a range

of emotions; they do not claim a direct mapping between

the objects and the mentioned emotions,

Comment 13

 

but emphasize the

benefit of the objects for stimulating expressiveness. The

value of the SEI is to elicit real-time, affective responses,

and to create a flexible, non-verbal channel of communication

between user and designers. The latter defines a key

advantage compared to other methods that are often limited

to verbalizations or visualizations that lack physicality.

Taste stimuli

The stimuli used for each taste are specified in Table 1.

Each stimulus was prepared as an odorless and colorless

water solution using a stock solution as specified in ISO

3972. We prepared the solutions according to the

specifications detailed by Hoehl et al. [10] and used

deionized water for the tastants. These compounds

standardised stimulus features and controlled for sensory

differences, such as texture, vision, etc. All of the solutions

were prepared the day before each study day. The

participants received 20 ml of each stimulus in a disposable

40 ml cup. A Latin square design was used to avoid order

bias [42].

Table 1. Stimuli used for the five main tastes, including the

stock solution (indicating the threshold specified in ISO 3972).

Participants

The study was conducted with 20 participants (nine female)

aged between 21-38 years (M=29.4, SD=5). Participants

were recruited based on the following criteria: not having

any food allergies, being non-smokers, not being pregnant,

and not having any sensory dysfunction (e.g., dysguesia, a

taste disorder), by self-report. The participants were

recruited through the staff list within the lead university. 16

participants were native English speakers, and the

remaining four were fluent in English. All participants gave

informed consent prior to the study.

Study set up and procedure

The participants were instructed and reminded 2 days prior

to the study not to eat spicy food 24 hours before the study

and not to drink or eat 1 hour before attending the study.

The study had 2 parts (see Figure 1): In the first part, we

applied the explicitation interview technique for all five

tastes; in the second part we introduced the SEI objects to

enhance the verbalizations for each taste.

In the first part, participants were given 5 minutes per

stimulus. They could take as many sips as they wanted of

the stimulus and were prompted with specific questions

about their taste experience (e.g., Could you describe what

you perceive? How does it feel in your mouth?). The aim

was to receive insights regarding the diachronic and

synchronic structure of the taste experience. We used this

technique in-situ in order to account for the rapidly

decaying sensory memory trace related to the human sense

of taste [21]. Before continuing with the next stimulus, the

participants were asked to have a sip of the deionized water

in order to cleanse their mouth. The same procedure was

repeated for all stimuli.

In the second part of the study, the participants were

instructed to match each taste experience to one or more of

the eight shapes inside the box. The participants could only

feel, and not see, the objects, to exclude any visual

influences and to focus on the mapping between ‘taste and

shape’ via the sense of touch. The participants were

instructed to select one or more or none of the shapes (they

could also reuse shapes for different tastes). Before going

through each taste stimulus again, the participants were

given the chance to put their hands into the box and

familiarize themselves with the 8 shapes.

Next they were asked to take a sip of water and start with

the first taste stimulus. They were asked to express the

thoughts they had in mind and to describe their choices or

lack thereof (if none of the shapes was selected). Finally,

the participants were asked to rate the pleasantness/

unpleasantness of the shapes on a four-point Likert scale

from ‘very pleasant’ to ‘very unpleasant’. They were also

asked about their personal favorites amongst the 5 taste

stimuli and their personal food preferences to support the

interpretation of the data.

In a final step, we tested the participants for their taster

status, which classified participants into supertaster, normal

tasters, and non-tasters. Overall, the study lasted one hour

and was audio/video recorded with the consent of the

participants. No incentives were paid to the participants.

Data analysis

All 20 tasting sessions were transcribed and a qualitative

analysis based on the transcripts was conducted. Two

researchers independently performed an open thematic

coding based on 5 cases (25%). The resulting themes were

discussed and an initial coding scheme was established.

Two more cases (10%) were coded independently leading

to a final coding scheme consisting of three main themes

(described in the next section), which were then applied to

the remaining 13 cases by both researchers. We also

performed a qualitative analysis of the mapping between

the SEI objects (see Figure 4) and the taste experiences,

captured through the transcripts and the visual material

from the recorded hand movements in the second part of the

study. Based on participants’ ratings of the shapes (their

physical pleasantness/unpleasantness) we could confirm

previous ratings of Isbister et al. [13] – the more spiky

shapes were rated as ‘unpleasant to slightly unpleasant’

(shapes 8,7,2), the more rounded shapes were rated ‘very

pleasant to pleasant’ (shapes 3,4,5,6), and only one shape

was perceived as ‘neutral’ (shape 1). Finally, the supertaster

test provided us with insights on the different taste

sensibility of participants and ensures a good distribution of

taster statuses in our study. Overall, we identified 5 nontasters,

11 normal taster (4 tending towards the upper edge

of bitterness sensitivity), and 4 supertasters. These results

are consistent with the known distribution amongst the

general population [3].

STUDY FINDINGS

The description of taste experiences is based on both parts

of the study. We describe the characteristics of taste

experiences across all five tastes along three identified

themes: (1) temporality, (2) affective reactions, and (3)

embodiment (see overview in the supplementary material).

We also discuss the particularities of each individual taste

in order to elucidate the potential design qualities of single

tastes. Each identified theme is represented in a pictorial

visualization of its key characteristics based on the

identified patterns across participants’ verbalizations.

Temporality

While taste experiences have expected elements of

changing intensity (e.g., strong taste, weak taste), the tastes

were also perceived as being mobile (e.g., moving within

the mouth, moving intensities), and occasionally exerted a

physical presence (e.g., building up, eroding, lingering).

These temporal characteristics are intertwined in the

unfolding of the experiences from its initial stimulation

(diachronic structure) and set the stage for the different

taste journeys (synchronic structure). Below, we describe

the different time-intensity profiles of taste experiences.

Taste intensities are generally experienced as being

dynamic and participants’ verbalizations offer a lexicon of

growth and decline. The diachronic nature of taste

experience is also revealed in the immediacy or longevity

of dynamic intensities. For instance, all participants agree

on the immediacy of the sour taste. Such immediacy is

expounded in similes such as ‘a firework in the mouth’, ‘a

punch’, and ‘a flash that hits you’. Yet, despite the

immediacy of this experience, it is short-lasting and decays

rapidly. “When you drink it, you get that bit of a rush. Yes,

it’s basically gone now [P15, sour]. In contrast, other tastes

were described as slowly building up or maintaining

consistent intensities (e.g., high for umami, and low for

salty). Such intensities could be seen to be ‘lingering’,

rather than ‘explosive’, as one participant described it:

“You’ve got this “Whoa” sensation, feels quite strong to

start with. Then it has gone super quick” [P19, sour].

While the dynamics of intensity imply variation (intensity

increasing and decreasing), the vocabulary of movement

animates these changes. Describing the bitter taste, one

participant stated: “I guess it’s not sticky like the first one

[umami]. It’s a bit lively… I feel like it’s moving around”

[P15, bitter]. While certain movements can be attributed to

mouth-feel (e.g., moving left to right across the tongue),

others were externalized (e.g., “I feel it almost into my

sinuses and into the rest of my face” [P14, bitter]). These

expressions were not confined to the temporal

characteristics of taste experiences, but already shed light

on the bodily reactions that can be elicited by tastes.

Movement was also invoked to describe stasis (e.g., ‘stays’)

and repetitive movement (e.g., ‘waves’). “So it is kind of

strong and it also stays. It doesn’t have a peak; it doesn’t

go up and down; it just stays” [P2, umami]. Other tastes

fluctuate rapidly: “Yes, ups and downs, but quite quick.

They’re quite sudden crests and falls…” [P3, sour].

Participants often appealed to similes of physicality in order

to explain their taste experiences (e.g., ‘round’, ‘soft’,

‘heavy’). Such physical experiences are tied to a synchronic

perception of taste. In contrast, the diachronic physicality of

taste experiences is given in the implied and experienced

characteristics of taste as a residual presence (e.g.,

‘lingering’, ‘stays there’): “It just stays in your mouth, so it

kind of keeps developing” [P10, umami] or “it just leaves

its mark in your mouth and doesn’t go” [P7, umami]. Such

experiences are, much like the increasing intensities, those

that ‘build up’, or ‘get a bit stronger”. Such presence is

understood to ‘erode’. Moreover, the implied residual

physicality is associated with experiences of absence. When

tasting sourness, many participants described the

immediate, almost physically imposing intensity followed

by a marked absence. This absence is seen to draw the

taster back into the taste, leaving them wanting more: “it

creates an expectation of sweet flavour, like if you were

biting into a slice of orange or something. … It’s gone now

and actually I’d quite happily have another sip, to be

honest” [P18, sour]. This residual physicality can also be

seen to afford agency to taste experiences, where tastes

‘grab you’, and ‘hit you in the face’. As such, taste

experiences can become reified in exerting influence over

the taster. This can be achieved in the residual physicality

or in absence, for instance, where the marked absence in

sourness is seen as “a forward feeling… It has the feeling of

tartness, your mouth moves forwards” [P14, sour].

Sweetness in contrast is associated with the feeling of

filling the mouth, and when the taste is gone it leaves one

with a kind of stickiness on the teeth.

Figure 2 shows a pictorial representation of the different

types of temporality identified based on the above

descriptions across all five tastes. The intensity is

represented through the thickness of the lines in the bars,

while movement is captured through the frequency of the

lines. Finally, residual physicality as temporal characteristic

is shown through the length of the whole bar. Overall, sour

is the taste delivering the highest intensity, followed by

umami and bitter. Umami presents a high intensity, and is

also characterized by lingering without losing much of its

intensity. Such an extensive residual presence can also be

seen for bitter, however with a lower intensity. Sweet and

salty are also of low intensity and can be characterized by

particular movements. While sweet starts slowly, builds up

and then dies out, salty does not peak at all and is constant

in its perception and moderate in unfolding over time. Sour,

by contrast, is short-lived with a rapid end. Specific to sour

is the sharp beginning followed by the absence of a taste

and the return of it through a forward pulling feeling, which

disappears quickly.

Affective reactions

Affective reactions refer to both the sense of pleasure or

displeasure gained from the taste experience, but also

feelings most often regarding familiarity, such as comfort,

or, by contrast, unfamiliarity, such as surprise and

suspicion. These affective characteristics, to be captured as

pleasant-unpleasant and familiar-unfamiliar, operate not

only as a static attitudinal response to taste experiences

(synchronic structure), but also as evolving characteristics

of the taste experience (diachronic structure).

When sampling the taste stimuli many participants related

their own uncertainty (e.g., I don’t know what to expect).

After one sample, this uncertainty is replaced for familiar

tastes. For unfamiliar tastes, particularly bitter and umami,

the sense of unease pervades and persists. Thus familiarity

produces responses at singular points (e.g., I am/am not

familiar with this), while also producing responses across

time (e.g., I know/do not know what to expect). A recurring

phrase throughout the taste study was “I know what it is, but

I don’t”. While we can at times attribute this to the nature

of the stimuli as water solutions (i.e., those not regularly

experienced by participants), the sentiment expressed also

refers to the lived and felt experiences of the tastes. That is,

while participants on the one hand had the taste ‘on the tip

of their tongue’, those tastes also brought to mind a variety

of known experiences, or, in the absence of known

experiences, feelings of uncertainty or unease. Such

feelings must presumably be associated with evolutionary

causes (considering many bitter foods are poisonous) or in

form of personal memories (e.g., salt, salty water, and the

seaside) and cross-modal experiences (e.g., with color, or

sounds). “If I drink or eat something that leaves that kind of

trace, I always imagine a colour. Glowing…. It’s weird. I

have no idea what this is, but there’s a bitterness that

stays” [P2, bitter]. Participants identified as supertasters

expressed their affective reaction more clearly: “Definitely

bitterness… I don’t like it” [P8, bitter], or “It’s immediately

bitter.… It’s like swallowing medicine” [P18, bitter].

There were few predictable or consistent affective reactions

among participants, and those experienced as pleasurable

by some, were experienced as disgusting or unsettling by

others. The affective response of participants could often be

tied to the participant’s familiarity with the taste. This was

particularly noticeable with umami. Participants who were

familiar with this taste indicated familiarity with savory

Asian cuisine, and could therefore interpret the perceived

taste and experienced it as pleasant. Those who did not eat

Asian cuisine were less familiar with the taste, particularly

in this intensity, and described unease and uncertainty when

tasting it. Such responses also evolved over time, notably

with sweet and sour tastes. While, as mentioned, sour

produced an immediately unpleasant experience, followed

by a refreshingly pleasant experience (e.g., “yes it probably

gets more pleasant as the intensity of the taste dissipates”

[P17, sour]), the taste of sweet was often initially pleasant,

followed by a distinct unpleasantness. This unpleasantness

could be so strongly felt as to produce nausea for some

participants (e.g., “although it’s dying off over time. It’s

quite sickly actually” [P20, sweet]). Such experiences were

tied to the physicality of the taste residing in the mouth, and

were perceived in two extremes for umami, influenced

through the participants’ familiarity/unfamiliarity with this

taste. Participants familiar with this taste perceived the

mouth filling and lingering experiences as comforting

(satisfaction after a full meal), while other participants who

were unfamiliar with it perceived it as disgusting, obtrusive,

and annoying referring to the fact that the taste takes over

control, without the chance to get rid of it quickly.

 

As with temporality, we created a representation of the

different types affective reactions on the five tastes (see

Figure 3). The pleasant-unpleasant characteristics of the

taste experience are represented through the ‘green’ and

‘red’ colors and in cases of a neutral experience colored as

‘orange’, and finally ‘white’ in case of absence of the taste.

The familiar-unfamiliar characteristics only find an explicit

representation for the umami. The familiarity of the taste

lead to its pleasant perception (upper bar for umami), while

unfamiliarity with the taste was expressed through

unpleasantness (lower bar for umami). Overall, some tastes

are characterized by the change from unpleasant to pleasant

(sour) or the other way around from pleasant to unpleasant

(sweet), while the bitter taste was clearly unpleasant and

salty was described as neutral. For umami, we identified

two separate experiences (participants either love or hate it)

grounded in the familiarity and unfamiliarity of the taste.

Embodiment

Although we would expect food experiences to involve

embodied, textural, responses (such as ‘crunchy’, ‘slimy’),

here each taste stimuli is experienced in the same form (i.e.,

as a colorless and odorless solution), and yet produce varied

embodied responses. Embodiment in relation to the

described diachronic and synchronic taste experiences

refers to the mouth-feel of tastes (how something is felt in

your mouth). Some participants additionally describe whole

body reactions (reactions described beyond the mouth) and

others refer to imagined and disembodied responses

(resulting from the taste stimulation and its associations).

Mouth-feel, referring to the experienced chemical and

physical sensations in the mouth, is frequently used to

describe different characteristics of foods, including coffee,

wine, and textured foods. Such descriptions are offered by

our participants for qualities of texture and viscosity. “It’s

just like a softness, but I guess a little bit more viscosity

even though I’m quite sure it doesn’t have any viscosity. It’s

just sort of the feeling of viscosity, the sweetness and this

cloud is just a bit more mouth feel” [P14, sweet]. The

mouth-feel also relates to a sense of movement, where

tastes evolve in space. Most often these are lateral

movements within the mouth, or commonly tastes are felt to

move backwards. Such experiences can be a feature of the

physical movement of the taste stimuli during the swallow

reflex and also associated with the location of taste

receptors on the tongue. However, in other cases, taste

experiences defied the location of taste receptors and tastes

could be experienced on the teeth, gums, and lips. One

participant goes as far as to describe the absence of mouthfeel:

“I don’t know really. It leaves this numbness in my

mouth like the lemon, but without the initial burst” [P9,

sour]. In addition to the sensations described in mouth,

some participants described bodily reactions that were

opposed to the mouth-feel or isolated taste experiences. “I

think the first part of it, the sour part, is a bit of a shock to

the system. I don’t think you’re expecting it to be like that”

[P16, sour]. Another participant said “I kind of see it from

the moment it enters my mouth and goes down all the way

to my stomach. It’s like I can see where it’s going” [P2,

bitter]. In this sense, participants described tastes as

producing expansive responses, including pleasure, nausea,

and, others including reactions associated with allergy such

as increased body heat (e.g., “If you eat it, it’s like your

body – the heat just changes” [P2, umami]). Feelings of

pleasure were often described as filling, particularly filling

the face or the whole body. A participant describes it as

such: “I feel that my whole face feels pleased with it” [P14,

umami]. Such feelings were not always positive and for

some participants, overwhelming feelings of nausea

accompanied tastes of salt, umami, and sweet. Nausea

could also be experienced in undulating taste experiences –

those taste which were experienced as prone to fluctuations

in intensity, almost mimicking travel or sea sickness.

Participants also described disembodied reactions, which

refer to something experienced that lingers between the

mouth and the body. Rather than experiencing direct bodily

reactions, participants describe an imagined reaction. “It’s

like it’s there but it’s not there” [P2, salty]. Disembodied

reactions could also be seen in expected or caricatured

responses, such as the imagined ‘pucker face’ of the sour

taste. Although few participants actually exhibit such a

reaction, it is an ingrained image of biting into a lemon. “It

feels a little bit, not uncomfortable, but it feels like it makes

you kind of screw you face up a bit” [P11, sour].

Shapes assigned to the overall taste experiences

The usage of the SEI objects (see Figure 4) as a physical

engagement with shapes enriched the description of the

taste experiences. The shapes particularly contributed

descriptors related to the combined temporal, affective, and

embodied experiences of each individual taste. Below, we

summarize the key characteristics and the mapping of the

eight objects for each taste across all participants.

The sweet taste, generally experienced as ‘smooth’ and

‘rounded’, was most reflected in shapes that present

elements of change such as “phases” (shapes 4 and 5) or

have protruding elements (like 7 and 1, or even the halfspiky

shape 2). While typically a pleasant taste, there is a

dynamic modulation of intensity and pleasure in the shapes.

The sour taste produces a ‘sharp’ response and for many is

best characterized by shapes such as 8 or 2. There are,

however, also elements of temporality, a shifting/phasing

associated with shape 4, starting with the big part as an

explosion and then rapidly decaying. The salty taste has a

broad aspect (mapped towards 3 and 6) and a finer

granulated and dynamic experience expressed through the

shape 1. Similar to sweetness and sourness there is a

repeating wave assigned to this taste experience, verbalized

around shapes 4 and 5 though this time associated with an

unpleasant feeling/sickliness as aftertaste. More than other

tastes, salty was associated with a subtlety of the temporal

characteristics, an experience of something moving, not

doing much, but still being there. This made participants

want a shape that they could manipulate (“These [objects]

are kind of too permanent; you’re not able to manipulate

them” [P6]) or something more neutral, such as a flat shape,

or a shape, which can be changed. Despite the fact that the

bitter taste was experienced as unpleasant, the mapping to

the shapes created two distinct experiences. For some

participants, bitter is a spiky but lingering experience

associated with a dull unpleasantness (1, 2, and 7 shapes

selected). For others it is a rounded and smooth taste (these

participants chose shapes 5 and 6), associating it with

medicine (form of pills), which dissolves in the mouth, and

you cannot get rid of it. Similarly to bitter, the mapping for

umami resulted in two distinct experiences. If umami was

experienced as unpleasant, participants tended to describe

the taste as disgusting and chose the shape 8 or 2. In those

cases where umami was perceived as pleasant, participants

described it as a more rounded taste with depth and chose

combinations of the rounded shapes (such as 3 and 1 shape

were used most, and combined with either the 5, 4, or/and 7

shape). This mapping confirms the descriptions of umami

as a full, mouth-filling experience with lots of things to it.

Overall, sweet and sour seem to be the two tastes where

participants show high agreement with respect to mapping

the shapes to taste experiences. Bitter and umami seem to

share some associations and create two different mappings

between shapes and taste experiences, while salty shows a

tendency towards smooth and round shapes, but with the

lack of the ability to change and manipulate the shapes.

Combined representation of the taste experiences

Figure 5 shows the final pictorial representation of all three

characteristics combined for each of the five tastes. The

length of the forms represents the temporal aspects, while

the width captures the mouth-feel. Whole body and

imagined embodiment could not be captured as such, but

are described in detail above. The expression ‘lingering’

was used particularly for sweet, bitter, and umami. When

used for sweet and umami ‘lingering’ is experienced in

combination with a ‘mouth filling’ element (it is filling the

whole mouth), while for bitter there is no filling experience

but it is described as a thin (straight through your mouth to

the back) experience, next to being unpleasant. In the bitter

case, ‘lingering’ thus refers to the residual physicality of

this taste (in the back of your mouth). Sour has an initial

unpleasant taste, dies down quickly, but comes back after a

short absence and leaves one with the feeling of wanting

more. Salty at last is similar to bitter, however with a

shorter life and perceived as less unpleasant. Salty is

perceived as a neutral taste with little consequence.

DISCUSSION AND FUTURE RESEARCH

While sensory researchers and neuroscientists study the

perception of taste and its temporality, their focus is on

quantifying the intensity and perceived changes of intensity

via a wide range of evaluation scales [26] or, in some recent

attempts, by means of time-intensity profiles of fMRI data

Comment 14 [19]. Our findings add a semantic level of understanding

underlying the taste experiences, their temporality enhanced

through descriptions of the affective reactions and

embodiment that the five basic tastes provoke.

Comment 15

 

This understanding may be useful when designing for taste

experiences as it provides designers and developers a

vocabulary to talk about taste and the design potentials

related to the different characteristics.

Comment 16

 

First, we discuss the particularities of each taste quality, and then discuss them

with respect to established psychological and behavioral

phenomenon highlighting their design potential for HCI.

design. This claim will be addressed later.

Comment 17

How is taste experienced?

Here we discuss the specific experiences each of the five

basic taste qualities create and can inspire design in HCI.

Comment 18

Sweet: Pleasant but with a bittersweet ending

The sweet taste was consistently described as pleasant,

which turned into something unpleasant. Participants

struggled between the instinctive taste likeability and the

learned taste values and rules (sweet is bad for the teeth),

which can be seen in light of learned associations, discussed

by Schifferstein and Hekkert [32] with respect to taste

experiences of products. Of particular interest with respect

to our findings on crossmodal interactions for sweet

stimulations are the embodied reactions (e.g., “It’s just sort

of the feeling of viscosity, the sweetness and this cloud is

just a bit more mouth feel” [P14]). Such reactions can be

explained through learned associations with sweetened food

and beverages. It is a combination of learned as well as

innate, genetic, and cognitive factors [32]. Sweet sensations

can be used to stimulate and enhance positive experiences,

however, on a limited timescale, as the sweetness is quickly

disappearing leaving one unsatisfied. It’s a pleasant taste

but one that is tinged with a bittersweet ending.

Sour: Unpleasant at first, but with the need for more

In contrast to the sweet taste, the sour taste is described as

short-lived and it often comes as a surprise due to its

explosive and punchy character. This taste overwhelms one

with its rapid appearance and quick decay. It leaves one

with the feeling that there is something missing. Based on

childhood memories, such as for instance of sweet-sour

drops, participants were expecting sweetness, but were left

disappointed, leaving them with the feeling of wanting

more. This phenomena was also observed in the evaluation

of a gustatory gaming interface with children, where sour

was used for negative reinforcement linked to the game

dynamics [17]. Children intentionally failed in the game in

order to get another sour stimulation.

Salty: Not doing much

The salty taste experience was not linked to an extreme

reaction unlike sour, bitter, and umami. This taste is often

described as ‘bland’, ‘discrete’, and ‘just being there and

not doing much’. It is minutely moving around, giving the

feeling of cleansing the mouth, but not being mouth filling

as sweet or umami, and certainty not as unpleasant as bitter,

however lingering almost as long as the bitter taste. The

modesty of saltiness in contrast to all of the other tastes

opens up some interesting questions when looking at the

neuroscience findings. Nakamura’s [19] findings based on

time-intensity fMRI profiles suggest that salty tastes change

more rapidly than do sweet tastes. This is not quite

consistent with how our participants described their

experiences and needs further studies.

Bitter: Unpleasant, not to be experienced again

The perceived intensity of the bitter taste was not the same

for everyone, as confirmed by the supertaster test. While

supertasters felt the experience with more immediacy,

others had to allow the taste to travel to the back of their

mouth to recognize it. After this initial difference, the bitter

experience becomes consistent with respect to its

‘lingering’ features, of ‘staying’ either on the tongue or at

the back of the mouth. Bitter was also described as ‘thin’.

The character of bitter was further revealed through learned

associations referring to ‘biting into a flower’, or

‘medicine’, things you had to take as a child, but after

which you would rather avoid this experience of bitterness.

Bitterness can indicate the presence of toxins [32] and is

found in evolutionary development of humans (e.g., feeling

of suspicion regarding bitter food as poisonous) [7]. It may

be useful for design to make people avoid certain behaviors.

Umami: Like/dislike, but still confusing as a taste

The familiar-unfamiliar characteristics of umami caused

much confusion in our study and participants could not rely

on their intuition. While the ‘like’ or ‘dislike’ of the taste

was decided instantly, the unpicking of the still ‘confusing’

elements of the umami taste was more challenging.

Different word pairs depending on the like/dislike of the

taste were expressed: ‘pleasant–unpleasant’, ‘comforting–

uncomforting’, and ‘liking–disgusting’. We could also see

participants using additional bodily descriptors, in

particular when describing umami as a pleasing experience

(‘face feels pleased’ or ‘body heat changes’). In cases of

dislike, the focus of attention in the verbalizations was the

lingering characteristic of the taste founded in the inability

to get rid of it. In these cases, the residual physicality can be

seen to afford agency. The taste experience becomes reified

in the influence it exerts over the taster. Depending on

personal familiarity/unfamiliarity (which may be defined by

cultural factors) and personal preferences, this taste

experience is quite interesting for design. Umami grabs

one’s attention and initiates a conscious process of

reflection. While judgment on the taste is defined quickly,

the reflective thinking brings to the fore the richness and

variety of the taste. Even when perceived as unpleasant, the

richness is recognized, and linked to the motivation to

remove the taste from the mouth.

How can we design with taste experiences?

Taste experiences can be discussed with respect to their

relevance for design, building on existing psychological and

behavioral phenomenon: rational and intuitive thinking,

anchoring effects, and behavior change.

Comment 19

 

The dual process

theory [14,37], for instance, accounts for two styles of

processing: the intuition based System 1 with associative

reasoning that is fast and automatic with strong emotional

bonds, and reasoning based on System 2 which is slower

and more volatile, being influenced by conscious judgments

and attitudes. Based on our findings, we can see that sweet

is intuitively perceived as pleasant, and bitter as unpleasant,

while sour, salty, and umami cause a reflective process,

confused, for instance, by the surprise appearance and rapid

disappearance of the sour taste. Our findings also give

insights into how to time the presentation of the taste

qualities so that the user can transition from System 1

thinking to System 2 thinking. Figures 2, 3, and 5 can be

used to create the appropriate transitions and time them. For

example, the rapidity of the sour taste experience does not

leave enough time for System 1 to engage with it and

triggers System 2 to reflect on what just happened. Such

reactions when carefully timed can prime users to be more

reason based in their thinking during a productivity task

(e.g., to awaken someone who may be stuck in a loop).

Moreover, an appropriately presented taste can create a

synchronic experience that can lead to stronger cognitive

ease (to make intuitive decisions) or reduce the cognitive

ease to encourage rational thinking. For example, a pleasant

taste can be used to provide achievements across the

workflow, however with the slight hint that there are still

more tasks to do before you are finished (e.g., the slight

unpleasant aftertaste of sweetness). Below, we outline

potential design directions for using taste experiences in

work-related activities and for personal behavior

management. Doing so, we draw on the potential of

different taste qualities and their power to stimulate

intuitive and rational thinking described above.

Managing anchoring effects through taste

A common aspect of everyday activity is interruption. We

are often interrupted by emails, telephone calls, or other

unanticipated events. These interruptions can either be short

(e.g., a quick glance at an email pop-up) or slightly longer

requiring us to change our activity (e.g., a line-manager

walking into your office to ask for something). All these

activities have anchoring effects. In other words, the initial

activity affects our judgments and decision making in the

latter activities. It has also been shown that users often find

it hard to avoid these biases in their judgments [38].

Our study of taste experiences suggests that taste interfaces

can be carefully designed to manage interruptions in such a

way that anchoring effects can be either minimized or

maintained. For example, we know that the salty taste has a

long temporal component with a feeling of “not doing much

but being there”. This taste could be very useful in those

situations where the interruption is small and the user is

expected to return to the initial activity soon. As an

example, when the user notices a pop-up in the bottom left

corner of their desktop (for email or other social media

interruptions) a small salty taste in their mouth which starts

just before the user switches their activity can be useful.

This will prolong their initial experience and remind them

of the initial activity when still checking the social media

page. This could enable smoother transitions back to the

initial activity. Alternatively, however, if the interruption is

a longer activity then it is useful for the user to drop any

priming effect that might transfer to the new activity. In this

case, a sour taste in the mouth would leave the user a quick

sharp taste engaging their rational System 2 but rapidly

decaying helping the user return to a more neutral state by

the time they switch to the new activity. Such management

of anchoring effects is not only useful for productivity

activities but also in other activities, such as gaming. For

example, LOLLio – the taste-based game device described

above [16], currently uses sweet and sour for positive and

negative stimulation during the game play. We suggest that

such a game could be improved based on our framework by

providing fine-grained insights regarding the specific

characteristics of taste experiences, which can be integrated

into the game play. When a person moves between related

levels of a game a continuing taste like bitter or salty is

useful. Whereas when a user is moving to distinct levels or

is performing a side challenge an explosive taste like sour,

sweet, or umami might be useful. The choice of specific

tastes in each category can be tuned by the designer to

create different affective reactions and a sense of agency.

Priming positive behavior through taste

Taste and taste preferences play an important role in our

food choices [24] and food plays a significant role in our

health and wellbeing. The stimulation and manipulation of

taste experiences therefore offers potential to improve a

variety of food behaviors. Using taste stimulation technology

to alter the taste of unpleasant but healthy food is one

obvious route. Expanding the design space for healthy taste

technology, our framework suggests alternative routes.

Taste experiences might be heightened through appeal to

related experiences and sensations. Morphing physical

objects, such as recently suggested shape-changing devices

[29], might also be used to replicate the embodied

expansiveness of the umami taste to stimulate an increased

taste experience for patients receiving chemotherapy who

may suffer from hypogeusia, a decrease in taste sensitivity.

Taste stimulation might also facilitate sustainable food

practices, for instance, linking food waste to taste

experiences. Taste stimuli might thus supplement other

post-actional cues in the effective disturbance of food waste

habits and promote critical reflection. When disposing overripe

bananas, a user might get a sour stimulation for the

waste of food but the immediate reward for waste

separation. Taste stimulation might also reflect various

characteristics of food waste, such as its lengthy impact on

environmental sustainability through the bitter taste. In this

way, the framework for design points to the potential for

taste experiences to be incorporated into timely and

rewarding persuasive messages for positive food behaviour.

Comment 20

CONCLUSIONS

In this paper we presented the results of a user study

exploring the experiential characteristics for each of the five

basic taste qualities. Our analysis of participants’

verbalizations, collected by means of verbal and non-verbal

methods, resulted in three key themes. We provide rich

descriptions on the temporality, affective reactions, and

embodiment of taste experiences. We discuss these themes

for each individual taste elucidating the design potentials

with respect to the specific structure and qualities of sweet,

sour, salty, bitter, and umami tastes.

”Comment

Our findings help to establish a framework for the design of taste experiences in

HCI, enhancing existing technology driven research around

taste, and food interaction design research. Although we do

not provide guidance for the design of a specific interactive

system in this paper, we are convinced that our framework

provides a starting point for designers and developers to

think about design/development potentials for taste in HCI.

Concluding Comment

ACKNOWLEDGMENTS

This work is supported by the EU Marie Curie Action (FP7-

PEOPLE-2010-IEF) and RCUK SiDE (EP/G066019/1). We

wish to thank our study participants and Annika Haas for

the audio-visual support, as well as Katerhine Isbister and

Kristina Höök for providing us with a set of the SEI objects.

REFERENCES

1. Bartoshuk, L.M., Duffy, V.B., Fast, K., Green, B.G,

Prutkin, J., Snyder, D.J. Labeled scales (e.g., category,

Likert, VAS) and invalid across-group comparisons:

What we have learned from genetic variation in taste,

Food Qual Pref, 14(2), (2003), 125-138.

2. Bartoshuk, L.M. Comparing sensory experiences across

individuals: Recent psychophysical advances illuminate

genetic variation in taste perception. Chem. Senses,

25(4), (2000) 447-460.

3. Chen, J., Engelen, L. (Eds.). Food oral processing:

Fundamentals of eating and sensory perception. Wiley-

Blackwell: Oxford, UK (2012).

4. Comber, R., Ganglbauer, E., Choi, et al. Food and

interaction design: designing for food in everyday life.

CHI EA (2012), 2767-2770.

5. Desmet, P.M.A., Schifferstein, H.N.J. Sources of

positive and negative emotions in food experience.

Appetite, 50(2-3), (2008), 290-301.

6. Drewnowski, A., Kristal, A., Cohen, J. Genetic taste

responses to 6-n-propylthiouracil among adults: A

screening tool for epidemiological studies. Chem.

Senses 26(5), (2001), 483-489.

7. Glendinning J. Is the bitter rejection response always

adaptive? Physiol. Beh., 56, (1994), 1217-1227.

8. Green, B.G., et al. Evaluating the ‘Labeled Magnitude

Scale’ for measuring sensations of taste and smell.

Chem. Senses, 21(3), (1996), 323-334.

9. Grimes, A., Harper, R. Celebratory technology: new

directions for food research in HCI. CHI (2008), 467-

476.

10. Hoehl, K., Schoenberger, G.U., Busch-Stockfisch, M.

Water quality and taste sensitivity for basic tastes and

metallic sensation. Food Qual Pref 21(2), (2010), 243-

249.

11. Hupfeld, A., Rodden, T. Laying the table for HCI:

uncovering ecologies of domestic food consumption.

CHI (2012), 119-128.

12. Humphries, C. Delicious science. Chefs are teaming up

with researchers to create avantgarde dishes. Is

‘molecular gastronomy’ more than a fad? Nature, 486

(2012), 10-11.

13. Isbister, K., Höök, K., Sharp, M., Laaksolahti, J. The

sensual evaluation instrument: Developing an affective

evaluation tool. CHI (2006), 1163-1172.

14. Kahneman, D. A perspective on judgement and choice.

American Psychologist, 58(9), (2003), 697-720.

15. Light, A. Adding method to meaning: A Technique for

exploring peoples’ experiences with technology. Beh.

and Information Technology, 25(6) (2006), 175-187.

16. Murer, M., Aslan, I., Tscheligi, M. LOLLio: Exploring

taste as playful modality. In Proc. TEI 2013, 299-302.

17. Moser, C., Tscheligi, M. Playful taste interaction. IDC

(2013), 340-343.

18. Narumi, T., Nishizaka, S., Kajinami, T., Tanikawa, T.,

Hirose, M. Augmented reality flavors: gustatory display

based on edible marker and cross-modal interaction.

CHI (2011), 93-102.

19. Nakamura, Y., Goto, T.K., Tokumori, K., et al. The

temporal change in the cortical activations due to salty

and sweet tastes in humans: fMRI and time-intensity

sensory evaluation. Neurorep, 23(6), (2012), 400-404.

20. Ngo, M.K., Velasco, C., Salgado, A., et al. Assessing

crossmodal corresponddences in exotic fruit juices: The

case of shape and sound symbolism. Food Qual Pref 28,

(2013), 361-369.

21. Núnez-Jaramillo, L., Ramirez-Lugo, L., Herrera-

Morales, W., Miranda, M.I. Taste memory formation:

Latest advances and challenges. Behav Brain Res,

207(2), (2010), 232-248.

22. Maynes-Aminzade, D. Edible bits: Seamless interfaces

between people, data and food. CHI EA (2005), 2207-

2210.

23. Obrist, M., Seah, S. A., Subramanian, S. Talking about

tactile experiences. CHI (2013), 1659-1668.

24. Palmer, S. E., Schloss, K. B. An ecological valence

theory of human color preference. Proc. of the National

Academy of Sciences, 107(19) (2010), 8877-8882.

25. Petitmengin, C. Describing one’s subjective experience

in the second person. An interview method for the

science of consciousness. Phen. Cog. Sci., 5(3-4),

(2006), 229-269.

26. Pineau, N., Schlich, P., Cordelle, S., et al. Temporal

dominance of sensations: Construction of the TDS

curves and comparison with time-intensity. Food Qual

Pref, 20(6), (2009), 450-455.

27. Ranasinghe, N., Cheok, A.D., Nakatsu, R. Taste/IP: The

sensation of taste for digital communication. ICMI

(2012), 409-416.

28. Ranasinghe, N., Karunanayaka, K., Cheok, A.D.,

Fernando, O.N.N., Nii, H., Gopalakrishnakone, P.

Digital taste and smell communication. BodyNets

(2011), 78-84.

29. Roudaut, A., Karnik, A., Löchtefeld, M. et al.

Morphees: toward high “shape resolution” in selfactuated

flexible mobile devices. CHI (2013), 593-602.

30. Savage, N. Technology: The taste of things to come.

Nature 486(21), (2012), 18–19.

31. Schifferstein, H.N.J., Fenko, A., Desmet, et al. Influence

of package design on the dynamics of multisensory and

emotional food exp. Food Qual Pref, 27 (2013), 18-25.

32. Schifferstein, H.N.J. Hekkert, P. Product experience.

USA: Elsevier (2007).

33. Smets, G.J.F., Overbeeke, C.J. Expressing tastes in

packages. Design Studies, 16(3), (1995), 349-365.

34. Spence, C. Crossmodal correspondences: A tutorial

review. Atten Percept Psychophys 73, (2011), 971-995.

35. Spence, C., Piqueras-Fiszman, B. Technology at the

dining table. Flavour, (2013), 2-16.

36. Spence, C., Smith, B., Auvray, M. Confusing tastes and

flavours. In M. Matthen and D. Stokes (Eds.). The

senses. Oxford: University Press. (in press, 2014).

37. Stanovich, K.E., West, R.F. Individual difference in

reasoning: implications for the rationality debate?.

Behav. Brain Sciences 23(5), (2000), 645–665.

38. Strack, F., Mussweiler, T. Explaining the enigmatic

anchoring effect: Mechanisms of selective accessibility.

Pers. and Social Psy. 73(3), (1997), 437-446.

39. Tosey, P., Mathison, J. Exploring inner landscapes

through psychophenomenology. Qual. Research in

Organiz. And Management: Inter. J., 5(1) (2010), 63-82.

40. Trivedi, B.P. Hardwired for taste. Research into human

taste receptors extends beyond the tongue to some

unexpected places. Nature, 486, (2012), 7-9.

41. Vermersch P. L’entretien d’explicitation [translated as

Explicitation interview], ESF (1994).

42. Wakeling, I.N., MacFie, H.J. Designing consumer trials

balanced for first and higher orders of carry-over effect

when only a subset of k samples from t may be tested.

Food Qual Pref, 6(4), (1995), 299-308.

 

 

 

Introduction

Opportunities for Odor:

Experiences with Smell and Implications for Technology

Marianna Obrist1,2, Alexandre N. Tuch3,4, Kasper Hornbæk4

m.obrist@sussex.ac.uk | a.tuch@unibas.ch | kash@diku.dk

1Culture Lab, School of Computing Science

Newcastle University, UK

2School of Engineering and Informatics

University of Sussex, UK

3Department of Psychology,

University of Basel, CH

4Department of Computer Science,

University of Copenhagen, DK

Permission to make digital or hard copies of all or part of this work for personal or

classroom use is granted without fee provided that copies are not made or distributed

for profit or commercial advantage and that copies bear this notice and the full

citation on the first page. Copyrights for components of this work owned by others

than ACM must be honored. Abstracting with credit is permitted. To copy otherwise,

or republish, to post on servers or to redistribute to lists, requires prior specific

permission and/or a fee. Request permissions from Permissions@acm.org.

CHI 2014, April 26 – May 01 2014, Toronto, ON, Canada

Copyright 2014 ACM 978-1-4503-2473-1/14/04…$15.00.

http://dx.doi.org/10.1145/2556288.2557008

ABSTRACT

Technologies for capturing and generating smell are

emerging, and our ability to engineer such technologies and

use them in HCI is rapidly developing. Our understanding

of how these technologies match the experiences with smell

that people have or want to have is surprisingly limited. We

therefore investigated the experience of smell and the

emotions that accompany it. We collected stories from 439

participants who described personally memorable smell

experiences in an online questionnaire. Based on the stories

we developed 10 categories of smell experience. We

explored the implications of the categories for smellenhanced

technology design by (a) probing participants to

envision technologies that match their smell story and (b)

having HCI researchers brainstorm technologies using the

categories as design stimuli. We discuss how our findings

can benefit research on personal memories, momentary and

first time experiences, and wellbeing.

Author Keywords

Smell; smell experiences; odor; olfaction; user experience;

smell-enhanced technology; narratives; smell stories;

crowdsourcing; design brainstorming; designing for smell.

ACM Classification Keywords

H.5.2 Information interfaces and presentation (e.g., HCI):

Miscellaneous.

General Terms

Experimentation, Human Factors, Design.

INTRODUCTION

Smell plays an important role for memories and emotions.

Compared to other modalities, memories evoked by smell

give stronger feelings of being brought back in time, are

more emotionally loaded, are experienced more vividly,

feel more pleasant, and are autobiographically older

(ranging back to childhood) [15,33]. Smell is incredibly

powerful in connecting humans to past events and

experiences.

Comment 1

Matsukura et al. [22] recently proposed the Smelling

Screen, an olfactory display system that can distribute

smells. Earlier work in HCI has proposed other systems that

capture and generate smells. For example, Brewster et al.

[5] developed a smell-based photo-tagging tool, and Bodnar

et al. [4] showed smell to be a less disruptive notification

mechanism than visual and auditory modalities.

Comment 2

 

Thus, smell technologies are already emerging.

Our understanding of how these technologies match the

experiences with smell that people have or want to have is

surprisingly limited.

Comment 3

First, while technologies such as those

mentioned above are often evaluated, the results mainly

concern the perception of smell. The evaluations say little

about the general potential of smell technologies for

humans or their ability to generate particular experiences.

 

Comment 4

Second, whereas earlier work states that the subjective

experience of smell stimulation is crucial for the success of

a system (e.g., [5]), we are unaware of work in HCI that

studies the subjective experience of smell (though see [17]).

Third, several hundred receptors exist for smell and we

cannot rely on any primary smells to stimulate a particular

experience, as might be imagined for other human senses

[2].

Comment 5

 

Taken together, these points suggest that we can only

link smell tenuously to particular experiences or emotions.

This limits our ability to design for a spectrum of

experiences.

Comment 6

The present paper focuses instead on experiences and

emotions related to smell and links them to potential

technologies. Inspired by work on user experience [14,34],

we concentrate on personal memorable smell experiences

and their links to emotion. From the focus on experience we

developed design guidance for smell-enhanced

technologies. The goal is to contribute knowledge on

subjective smell experiences and their potential for design.

 

Comment 7

We collected 439 smell stories, that is, descriptions of

personal memorable experiences involving smell.We

distributed a questionnaire through crowdsourcing, ensuring

a large-scale coverage and variety of smell stories. We

analyzed the stories and identified 10 main categories and

36 sub-categories. Each category was described with

respect to its experiential and emotional characteristics and

specific smell qualities. Besides smell stories associated

with the past (e.g., memory of loved people, places, life

events) we identify stories where smell played an important

role in stimulating action, creating expectations, and

supporting change (e.g., of behavior, attitude, mood). Smell

can sometimes also be invasive and overwhelming, and can

affect people’s interaction and communication. Within the

categories, we identify common smell qualities and

emotions, which support the exploration of opportunities

for design. In particular, we discuss the implications for

technology based on feedback from participants and on a

brainstorming session with HCI researchers working on

smell technologies.

The main contributions of this paper are

(1) an experiencefocused understanding of smell experiences grounded in a

large sample of personal smell stories, which allowed us

Comment 8

 

(2) to establish a systematic categorization and description scheme for smell experiences, leading to

Comment 9

 

(3) the identification of technology implications by participants,and

Comment 10

(4) the exploration of design potentialities by HCI

researchers.

Comment 11

THE HUMAN SENSE OF SMELL

The sense of smell is the most complex and challenging

human sense.

 

Comment 12

Hundreds of receptors for smell exist and the

mixing of the sensations, in particular with our sense of

taste, is immense [2]. The sense of smell is further

influenced by other senses such as vision, hearing, and

touch; plays a significant role for memory and emotion; and

shows strong subjective preferences. Willander and Larsson

[33] showed that autobiographical memories triggered by

smell were older (mostly from the first decade of life) than

memories associated with verbal and visual cues (mostly

from early adulthood). Moreover, smell-evoked memories

are associated with stronger feelings of being brought back

in time, are more emotionally loaded, and are experienced

more vividly than memories elicited through other

modalities [15,33]. No other sensory system makes the

direct and intense contact with the neural substrates of

emotion and memory, which may explain why smellevoked

memories are usually emotionally potent [15].

The emotion-eliciting effect of smell is not restricted to the

context of autobiographical memories. Smell is particularly

useful in inducing mood changes because they are almost

always experienced clearly as either pleasant or unpleasant

[8]. For instance, Alaoui-Ismaïli et al. [1] used ‘vanilla’ and

‘menthol’ smells to trigger positive emotions in their

participants (mainly happiness and surprise) and ‘methyl

methacrylate’ and ‘propionic acid’ to trigger negative

emotions (mainly disgust and anger). Interestingly, Herz

and Engen [15] pointed out that almost all responses to

smell are based on associative learning principles. They

argued that only smells learned to be positive or negative

can elicit the corresponding hedonic response and that

people, therefore, should not have any hedonic preference

for novel smells. The only exceptions are smells of

irritating quality that strongly stimulate intranasal

trigeminal structures. Such smells often indicate toxicity.

While neuroscientists and psychologists have established a

detailed understanding of the human sense of smell, insight

into the subjective characteristics of smell and related

experiences is lacking.

Comment 13

The exploration of this subjective layer of smell is often understood as going beyond the

interest of these disciplines, but is highly relevant for HCI

and user experience research.

 

Comment 14

SMELL IN HUMAN-COMPUTER INTERACTION

Ten years ago, Kaye [17] encouraged the HCI community

to think about particular topics that need to be studied and

understood about smell. While some attempts have been

made to explore smell during recent years, the potential of

smell in HCI remains underexplored.

 

Comment 15

 

 

Most work on smell in HCI focuses on developing and

evaluating smell-enhanced technologies.

Comment 16

Brewster et al. [5]

used smell to elicit memories, and developed a smell-based

photo-tagging tool (Olfoto). Bodnar et al. [4] showed smell

to be less disruptive as a notification mechanism than visual

and auditory modalities. Emsenhuber et al. [9] discussed

scent marketing, highlighting the technological challenges

for HCI and pervasive technologies. Ranasinghe et al. [24]

further investigated the use of smell for digital

communication, enabling the sharing of smell over the

Internet. More examples of smell-enhanced technologies

can be found in multimedia applications [13], games [16],

online search interfaces [19], health and wellbeing tools

(e.g., http://www.myode.org/), and ambient displays [22].

The exploration of smell-enhanced technologies is mostly

limited to development efforts and the evaluation of users’

smell perception of single smell stimuli. The smells used

are often arbitrary and not related to experiences. This is

because of the lack of knowledge pertaining to the

description and classification of smells required for HCI

[17]. Kaye points out that “There are specific ones

[classification and description schemes] for the perfume,

wine and beer industries, for example, but these do not

apply to the wide range of smells that we might want to use

in a user interface” (p. 653). Thus, previous work has a

general and quite simple usage of smell.

Comment 17

THE POTENTIAL OF STUDYING SMELL EXPERIENCES

In contrast to the work reported above, the present paper

focuses instead on experiences with smell and links them to

potential technologies. We do so through stories of

experiences with smell. Stories are increasingly used within

user experience research to explore personal memories of

past experiences, but also to facilitate communication in a

design process [3,34]. Stories are concrete accounts of

particular people and events, in specific situations [10],

 

Comment 18

and

are more likely to stimulate empathy and inspire design thinking than, for example, scenarios.

 

Comment 19

STUDY METHOD

We asked a large sample of participants to report smell

experiences that were personal and meaningful.

Comment 20

We refer to the description of these experiences as smell stories. These

stories were captured through a questionnaire described

below, which included inspirational examples of smellenhanced

technologies at its end. Based on the examples we

asked participants to reflect on their experience and future

technologies. The rationale of this approach was to begin

from smell experiences that matter to participants, instead

of starting from an application or a particular technology.

 

Comment 21

Questionnaire

We created a web-based questionnaire consisting of six

parts. We started with an open question to stimulate the

report of a personal memorable smell experience. This was

followed by closed questions aiming to elucidate the

relevant emotional and experiential characteristics, as well

as the smell qualities. Participants could freely choose the

story to report. The questionnaire was administered through

a crowdsourcing platform to obtain a large sample of smell

stories. Crowdsourcing provides valid and reliable data [20]

and has been used for capturing user experiences [31].

Comment 22

Part 1: Smell Story

The smell stories were elicited through an initial exercise,

where participants were asked to think about situations and

experiences where smell played an important role. The aim

was to get participants into the right frame of mind and

sensitize them to smell. Next, participants were asked to

describe one memorable smell experience in as much detail

as possible, inspired by the questioning approach used in

explicitation interviews [23]. This questioning technique is

used to reconstruct a particular moment and aims to place a

person back in a situation to relive and recount it. Part 1 of

the survey was introduced as follows: Bring to your mind

one particular memorable moment of a personal smell

experience. The experience can be negative or positive.

Please try to describe this particular smell experience in as

much detail as possible. You can use as many sentences as

you like, so we can easily understand why this moment is a

memorable experience involving smell for you.

Participants were asked to give a title to their story

(reflecting its meaning) and indicate if the experience was

positive, negative, or ambivalent (i.e., equally positive and

negative). They were also asked to indicate how personally

relevant the experience was (from ‘not personally relevant

at all’ to ‘very personally relevant’).

Comment 23

Part 2: Context

Part 2 asked participants to give further details of their

reported experience via open and closed follow-up

questions. There were four questions on the context of the

described experience, including the social context (who else

was present), the place (based on the categories used by

[26]), the location (as an open field), and the time when the

reported experience took place (days, weeks, months, or

years ago).

Comment 24

Part 3: The smell

Specific questions on the characteristics and qualities of the

smell were asked in Part 3. Participants characterized the

smell itself using a list of 72 adjectives (i.e., affective and

qualitative terms) derived from the ‘Geneva Emotion and

Odor Scale’ (GEOS) [7]. Participants could also add

descriptions to characterize the smell in an open feedback

box. In addition, they rated the smell with respect to its

perceived pleasantness, intensity, and familiarity.

Part 4: Experienced emotions

In Part 4 participants had to describe how they felt about

the experience as a whole, using a list of affective terms

(101 in total). They could go through the list and tick the

words that best described their emotions during the

experience. The words were derived from Scherer [27].

Participants could also add their own words in a free-text

field.

Part 5: Smell technologies

After the participants had selected, thought about, and

described a particular smell episode, Part 5 linked their

personal experience to technology. The participants were

engaged in a envisioning exercise inspired by work on

mental time travel [30]. They were shown six inspirational

examples of smell technologies, namely: Olfoto: searching

and tagging pictures (CHI, [5]); Smelling screen: ambient

displays (IEEE, [22]); Digital smell: Sharing smell over the

Web (ICST, [24]); Scent dress: interactive fabric with smell

stimulation (http://www.smartsecondskin.com/); Mobile

smell App: iPhone To Detect Bad Breath and Other Smells

(BusinessInsider 01/2013), and Smell-enhanced cinema:

Iron Man 3 Smell-Enhanced Screening (Wired 04/2013).

These six technologies cover areas of relevance for HCI

(mobile, ambient, wearable, personal, and entertainment

computing), give realistic examples of smell technologies

from research, and include recent, commercial examples.

We asked the participants to imagine any desirable change

that future smell technology might make (or not) with

respect to their personal smell experience.

Comment 25

We asked them the following questions: (1) How could your experience be

enhanced? (2) What technology are you thinking about? (3)

Why would such a combination of your experience and the

technology be desirable, or why would it not?

Comment 26

Finally, the participants could express any other ideas for smell

technology in a free-text field.

Part 6: Personal background

At the end of the questionnaire, participants answered

questions on their socio-demographic and cultural

background. The goal was to try to identify any

geographical and cultural influences on smell attitudes (as

found by Seo et al. [29]). The participants were also asked

to assess their own smell sensitivity.

All the questions, except for those on demographics, were

mandatory. On average, the survey took 16 minutes to

complete (SD = 7.57 minutes). Participants received US$

1.50 for completing the questionnaire, corresponding to an

hourly salary of 5.63 dollars.

Comment 27

Collected data and participants

A total of 554 participants began the questionnaire. Of

these, 480 completed the questionnaire and answered three

verification questions at its end. These questions required

participants to describe the purpose of the study without

being able to go back and look at the earlier questions or

guidelines. After data cleaning, 41 stories were excluded.

Fake entries (n = 11) were identified immediately, while

repeated entries (n = 10), incomplete stories (unfinished

sentences; n = 6), and incomprehensible stories (which did

not make sense on their own; n = 14), were excluded

iteratively throughout the coding process. This left us with

439 smell stories.

Comment 28

At the time of the study, all 439 participants (52.8% female)

lived in the US; most had grown up in the US (95%). The

participants’ age ranged from 18 to 67 years (M = 31.5, SD

= 10.0). A majority of participants (84%) indicated being

sensitive to smell (rating 4 or higher on a scale from 1 to 5).

Data analysis

The analysis process followed an open and exploratory

coding approach [25]. Two researchers conducted the

qualitative coding process. After coding an initial 25% of

the stories, two more coding rounds (to reach 33% and then

50% of the data), led to the establishment of an agreed

coding scheme. The coding scheme contained 10 main

categories and 36 sub-categories, and a category entitled

‘not meaningful’ for cases where smell did not seem to

have any relevance in the described experience. Based on

this coding scheme, one researcher coded the remaining

50% of the data, and the second researcher coded a subsample

of 25% of that data, resulting in a good inter-coder

agreement (Cohen’s kappa of κ = .68) [12].

Comment 28

Follow up design brainstorming

In addition to the feedback from our participants,

Comment 29

we also explored the design value of the smell stories with experts

in the field. We organized a two-hour design brainstorming

session with three HCI researchers, two working on smell

technologies and one working on advanced interface and

hardware design. None of them were from the same

organization as the authors and none were familiar with the

details of the study before the session.

The brainstorming session aimed to share and interpret the

smell stories and followed four stages [11]: (1) prompting,

(2) sharing, (3) selecting, and (4) committing. We selected

36 stories (one representative story for each sub category)

as brainstorming prompts. All 36 stories were printed on A6

sheets (including the story title, the smell story, context

information, and personal background). Each researcher

was asked to read through the stories individually before

discussing them together. They were asked the same

questions as our participants (e.g., how they might imagine

a connection between the experience and technology). Each

researcher chose the most interesting/inspiring stories to

share with the group, then they generated ideas as a group,

and selected three to four ideas to be developed in more

detail. The outcome of the brainstorming session is

presented in the implication section, after the description of

the findings from the smell stories.

 

Comment 30

FINDINGS ON EXPERIENCES WITH SMELL

In the following sections we present our findings according

to the 10 identified categories. The 439 smell stories were

organized via their primary category, as agreed by the

coders. This categorization does not define a strict line

between the categories, as they are not wholly independent,

but it does enable us to organize the material and generate a

useful dataset for design.

Comment 31

Below we provide for each category a rich description of the particularities of the

stories, excerpts from example stories, and their associated

smell qualities and emotions. Each category also contains

information about the participants’ own rating of the stories

as positive, negative, or ambivalent.

Category 1: Associating the past with a smell

This category is the largest and contains 157 stories. In

these stories, the participants described a past experience in

which a smell was encountered during a special event in life

(e.g., ‘Wedding Day’, ‘New House’), at a special location

(e.g., ‘The Smells of Paris’, ‘Grandma’s House’), or as part

of a tradition (e.g., ‘The Smell of Thanksgiving’ or

‘Christmas Eve’). In these stories the smell was described

as having a strong association to those particular moments

in the past, with no actual smell stimulus in the present. A

particularity of this category is the distinction between

stories describing personal memorable events versus

personal life events (e.g., ‘Disneyland’ versus ‘When my

mother died’). Smells were also associated with personal

achievement/success (e.g., ‘Scent of Published Book’,

‘New Car Smell’) and other important episodes of change,

such as “‘Fresh Start’: I was taking a job in a new city. …. I

took a plane trip across the country and the moment I took

a step off the plane and took a deep breath will always stick

with me. It felt so clean and the air actually smelled fresh

and new” [#488]. Within this story, the qualities of the

smell were for instance described as fresh, energetic, and

invigorating. Some of the emotions experienced at this

moment were courageousness and excitement. Although

this category is dominated by positive experiences (n =

127), negative experiences were also reported (n = 27),

such as ‘Car Crash’.

Category 2: Remembering through a smell

The 40 stories in this category described a recent

experience of a smell, which reminded participants about

past events, people, locations, or specific times in their life.

In contrast to the previous category (where stories describe

a direct link from the recollected past smell to the present;

e.g., the smell of ‘Grandma’s House’), this category

contains stories that describe an indirect link from the

present experienced smell stimulus

 

Comment 32

to the past event, person or place (e.g., the smell of chocolate cookies as sudden

reminder about grandma). Most stories in this category

contain reminders of childhood described as ‘sweet’,

‘reassuring’ and ‘nostalgic’ with respect to the qualities of

the smell. A sub-set of stories in this category (n = 10) also

highlight the particular power of smells to take a person

back in time. The description of such a flashback caused by

a sudden smell stimulus was described as: “‘My first love’:

It was the next day, when I was walking through the local

Macy’s that I smelled something that threw me back into

that situation, I could feel and see everything that had

happened the day before when I smelled a perfume in the

store” [#630]. Some of the qualities used to describe the

smell were attractive, erotic, and fresh. The experienced

emotions were described as amorous, aroused, excited,

hopeful, and interested. The stories in this category were

mainly positive (n = 37), except for three.

Comment 33

Category 3: Smell perceived as stimulating

The 62 stories in this category described experiences with a

unique, mostly unknown smell (all stories, except one, were

positive). The smells arose from different sources, such as

perfume, food, and nature. A particularity of this category is

the quality of ‘first time’ encounters with a smell across all

origins. One participant described the first time he was at a

beach: “The smell was very different from anything I had

ever experienced before. At first I was kind of grossed out

by the smell, but I grew to love it” [#921]. Another

participant described the smell of a tornado experienced for

the first time: “It was similar to the smell before rain but

had a certain sharpness to it, as if to warn of the incoming

danger. I felt like I knew this smell but at the same time, it

felt foreign to me. It wasn’t a bad smell, it was just slightly

unfamiliar” [#713]. The smell qualities and experienced

emotions were often described with mixed attributes (e.g.,

heavy, imitating, and stimulating; attentive, serious, and

calm), but still rated as positive experiences by participants.

Most of the other stories in this category reported on the

first experiences with food (e.g., ‘Slice of Heaven’) and

nature (e.g., ‘Grass’), and were described as desirable,

fresh, or pure, and provoked feelings of happiness at the

moment they occurred. Although specific memories were

established, including unique new associations (e.g.,

‘Tornado smell’), the stories in this category did not evoke

the kind of strong connections to the past as described in

Category 1 and 2.

Category 4: Smell creating desire for more

This category contains 48 stories (45 positive). Key to these

stories is that the smell grabbed the persons’ attention

unexpectedly. The smell was either associated with food

(triggering appetite), nature (triggering curiosity), or the

scent of other people (triggering attractiveness), which

motivated one to do or get something. In some stories smell

was described in relation to the sensation of newness (e.g.,

“‘The sweet smell of CPU’: …There was the smell of the

cardboard boxes it all arrived in, the smell of new metal–

perhaps it was a combination of these and other things, but

when the building was complete there was just a singular

smell that was unique to a new computer built by my own

hands” [#685]). The qualities of the smell in this story

included beneficial, heavy, sophisticated, energetic, and

pleasantly surprising. The experienced excitement was

expressed through words such as confident, delighted,

enthusiastic, impressed, or triumphant. This category also

contained one story where the smell at a funeral stimulated

reflection in the moment (e.g., ‘The scent of moving on’).

The story was rated as a positive experience and at the same

time the smell was described as clean, penetrating, and

persistent, and the participant indicated that she was afraid,

anxious, discontented, sad, tired, and uncomfortable.

Despite the negative situation described in this story, the

smell gave hope and a desire to live and move on, looking

into the future in contrast to the stories in Category 1 and 2.

Category 5: Smell allowing identification and detection

This category captures the enabling role of smell in certain

situations, such as allowing one to identify or detect a smell

(e.g., “‘Gas leak’: I was cooking something on a gas stove

and went out for a few minutes. When I came back, the fire

was extinguished but the gas was still on. My roommate

was sat at the table doing schoolwork, completely oblivious

to the poisonous gas that was filling the room. I told him to

get the hell up and open the windows and doors” [#951]).

The qualities used to describe the smell were distinguished,

penetrating, dirty, and light. The emotions related to this

situation were described as anxious, conscientious,

confident, and serious. Although the category is rather

small (n = 11), the lesson to be learnt from the shared

stories was the immediacy of the smell, allowing the

participant to act or prevent something.

Category 6: Overwhelming power of smell

This category includes 37 stories where the smell

overwhelmed the person in a positive way (n = 5; e.g., ‘The

Chocolate Factory’) but predominately in a negative way (n

= 30; e.g., ‘The Smell of Death’). In the latter case, people

described the smell as something disturbing, as something

that hit them suddenly on their way or during an activity. A

subset of the stories was recounted as traumatizing, so that

the person could still vividly remember the particular

moment in the past although years have passed and no

recent similar smell stimulation had occurred unlike in

Category 2 (e.g., “‘Visit to a local county jail’: My guide

warned me ahead of the time that it was going to be a little

foul in there, but nothing could have prepared me for the

obscenely acrid stench of hundreds of men crammed into

every available space of the jail, right down to windowless

storage rooms converted into more cells. … For days

afterwards, I couldn’t shake the smell…. There weren’t

enough showers to take it away. It’s been several years

since then, and my memory of that smell is just a strong as

ever” [#604]). In this category, the qualities of the smell

were described as heavy, penetrating, dirty, or sickening.

Amongst others, the experienced emotions were described

as alarmed, anxious, distressed, frustrated, or

uncomfortable. In contrast to Category 1 and 2 (where the

smell was associated with an event from the past or

triggered a specific memory), Category 6 is about the smell

as such during the experience and not about the memory

associated with this smell. As opposed to the first two

categories, in most stories forgetting – not remembering –

the smell was the key element.

Category 7: Smell invading private and public spaces

All the stories in this category (n = 32) described an

experience where one could not get rid of the smell. The

smell invaded private and public spaces. In contrast to the

previous category, the smell entered the person’s personal

space (the person did not enter the space where the smell

already existed) and took over the space. The loss of control

over the smell was linked to the lingering quality of the

smell (e.g., “‘Don’t want to smell that twice!’: I woke up

one morning suddenly confused and was hit with an odor so

horrible I couldn’t figure out what it was. … It was not like

the smell you get a whiff of when a skunk stinks up the

outdoors” [#530]). In the story the power of the smell,

causing them to leave the house for several hours, was

described with qualities such as foul, nauseous, penetrating,

and persistent. One of the experienced emotions was

surprisingly ‘amused’, however it was overruled by other

emotions including annoyed, anxious, disgusted, taken

aback, and uncomfortable. Despite the glimpse of humor in

some stories, this category mainly contains negative

experiences and underlines the power of the smell with its

sudden and lingering qualities.

Category 8: Social interaction is affected by the smell

Within this category, smell was related to a person’s own

smell or to the smell of others. Smell negatively affected

the interaction among people and their togetherness (e.g.,

“‘Dragon breath teacher’: Once a teacher yelled at me

during class. She got so close up into my face that I could

smell her bad breath. This made the experience much worse

because I wanted to get up and walk away but she was

grabbing me to keep me focused on her while she was

talking” [#744]). The smell qualities were described as

nauseous, penetrating, and sickening, and caused negative

emotions experienced as bitter, distressed, or insulted.

Despite frequent interactions among people, this category

only contains 11 stories. This set of stories (overall negative

experiences, apart from two) contains interesting elements

with respect to a person’s own awareness of body smell and

the overbearing effect of other peoples’ smell on one’s

comfort.

Category 9: Smell changes mood, attitude and behavior

This category contains 23 stories, which underlined the

power of smells to change a person’s mood, attitude, or

behavior. Stories reported the active regulating effect of

smells with respect to mood, but mostly (n = 14) the change

of behavior due to smells (e.g., ‘Accidental vegetarian’ or

‘Saved by the Smell!’). One story showed the active usage

of smells to change one’s mood. A participant had recently

been divorced and reported on the day her husband had

moved out: “‘White Lilac Sheets’: “I made the bed with my

lilac sheets and the atmosphere changed. I still remember

that scent and how I felt on that day. I was going to be

okay. The room didn’t look or feel or smell lonely anymore.

It looked and smelled fresh and clean and lovely and a bit

romantic and it was mine” [#526]. The qualities of the

smell were described as fresh, reassuring, and spring-like,

while the experienced emotions were determined, hopeful,

longing, tense, but also worried. Overall, the stories in this

category were reported as mainly positive (n = 12)

experiences, but also as negative (n = 7) and four stories

were rated ambivalent, neither positive nor negative.

Category 10: Smell builds up and changes expectations

This category shows the potential of smell to build up

expectations and to surprise. In the former case (11 stories)

the smell was building up expectations until the actual

contact with the trigger, such as food or a perfume (e.g.,

“‘The Smell of Hungry Anticipation’: “I was trying a new

soup for the first time. When it was brought to the table, the

soft smell of rosemary immediately hit my nostrils. …It

complimented the taste of the soup and built anticipation”

[#585]). The smell was described as mouthwatering,

healthy, and pleasantly surprising, and was further related

to emotions such as conscientious, expectant, and relaxed.

In other stories (n = 7), expectations were exceeded to the

extent that they surprised and diverted anticipations (e.g.,

‘PomVinegar Surprise’: “I could smell the pomegranate

and vinegar from about 10 steps away, and it was a very

pungent (thought not unpleasant) odor. I almost felt my

nose becoming runny and took out a tissue. When I tasted

the dish, however, the taste wasn’t nearly as sour as I

expected it to be from the smell” [#542]). The smell in this

story was described as distinguished and penetrating, and

was associated with emotions such as attentive and excited.

Key quantitative facts behind the smell stories

While the above-described categories can be used as an

inspiration and as a starting point for exploring design

opportunities for smell in HCI,

Comment 34

our quantitative data provides additional background information. Below, the

key quantitative information across all the collected smell

stories is summarized. The majority of the 439 collected

stories were positively valenced (n = 296), 112 were

negative, and 31 were ambivalent. On average, negative

stories tended to be slightly longer (M = 90 words) than

positive stories (M = 79 words), but the difference is

statistically not significant (U = 14600, p = .063, r = .09).

Contextual information

Most stories occurred in a context where one or more familiar persons were present (64.2%)

or where participants were alone (21.6%). The presence of

one or more strangers was reported less frequently (8.7%).

With regard to location, most of the experiences happened

at the participant’s or a friend’s home (38.1%) or in a public

building (20.7%). Quite a few participants reported that

their experience took place in the streets or another public

space (14.4%), in a natural setting (7.3%), or at work

(6.4%). The remaining participants (13.2%) indicated other

places (e.g., stranger’s home). On average the reported

experiences occurred 8.7 years ago (SD = 10.3), ranging

from 1 day to 58 years ago.

The qualities of smell

The most frequent smell qualities

reported in positive stories were pleasant (60%), fresh

(42%), sweet (38%), clean (31%), and mouthwatering

(30%). Smells in negative stories were described as

unpleasant (62%), penetrating (55%), heavy (54%), foul

(53%), and nauseous (51%). In ambivalent stories the smell

was perceived as fresh (39%), pleasant (32%), mouthwatering

(32%), attractive (26%), and penetrating (23%).

Experienced emotions

When asked to describe how they felt during their experience, participants’ used the affective

terms happy (63%), pleased (53%), joyous (42%), delighted

(41%), and excited (39%) in positive stories and

uncomfortable (55%), disgusted (51%), distressed (43%),

miserable (42%), and taken aback (29%) in negative stories.

Ambivalent experiences were most frequently described as

happy (42%), excited (39%), enthusiastic (35%), joyous

(32%), and serene (29%).

An overview of all 10 categories and 36 sub-categories

including qualitative and quantitative information

(including a full example for each sub-category, used in the

design brainstorming session) is provided as supplementary

material. All smell stories and related qualities of smell,

experienced emotions, and context information, are also

available at www.multisensory.info for further exploration.

IMPLICATIONS FOR TECHNOLOGY

This study focused first on experiences and second on the

implications for technology. This section turns to

technology. Below we summarize the feedback from the

participants on how technology would fit with their

experience, and describe input from a brainstorming session

with HCI researchers working on smell technologies and

advanced interface and hardware design, based on a sub-set

of the smell stories (one from each sub-category).

Ideas for technology from participants

Below we summarize the six areas and ideas for desirable

future smell technologies mentioned by participants in Part

5 of the questionnaire:

(1) To share smells with family/friends: allow one to

participate in a family event through remote smelling; share

smells of special moments such as the smell of a newborn

baby with distant relatives; share smells with people who

they know would appreciate it (such as through social

media); allow capturing and sharing of smells to create a

common understanding where you can’t explain it.

Participants also desired to be able to design and share new

smells from a personal database and create a personal smell

box/bottle.

(2) To support decision making: use smells for a quick

judgment in online shopping (like/dislike can be determined

easily); create smell profiles about holiday places and travel

destinations; smell match maker in dating websites for

allowing a better decision making about going on a date or

not with a person (smell enhanced profiles).

(3) To regulate mood actively/passively: smell to relive

good experiences whenever you want to get in a better

mood; to calm yourself down in stressful moments such as

in traffic jam or at work; as a reminder of past memories

you would have forgotten otherwise but that can cheer you

up when you feel depressed and life seems too difficult.

(4) To combine with other technologies and activities:

integrate smell into radio; combine smell with music such

as with ‘soundhound’ or ‘shazam’ apps; smell-enhanced

advertisement on TV (for food channels); enhance visits of

concerts, theater and performances with smell; allow underwater

smelling when diving.

(5) To combine with everyday objects: enhance wristband

with smell for keeping a preferred perfume lingering; have

special glasses to see and smell the beach; smell-enhanced

jewelry and clothes. One participant imagined her wedding

ring enhanced with the smell of that day.

(6) To make oneself and others aware about body smell: to

avoid embarrassing moments; provide invisible cues to a

person about her/his smell level; quick smell check after

sporting activities.

 

Comment 35

The first idea matches the experiences in Categories 1 and

2, where particular events/moments in life are associated

with a smell. The desire for capturing and sharing these

experiences enhanced with smell becomes prevalent and

suggests design implications for real-time smell-enhanced

technologies (e.g., mobile phone, photo or video camera).

The second idea can be linked to Category 5, allowing

people to identify and detect a smell. Moreover, smells are

seen as very powerful for supporting quick decision-making

(e.g., smell-enhanced website navigation and searching).

The third idea shows a direct link to Category 9 and the

potential of smell to change mood. Interestingly participants

whose story was in Category 1 or 2 were wishing for the

possibility to capture pleasant smells, for instance from

their childhood, and released to them in the present. This

desire for smell-enhanced technologies or products is also

apparent in the fourth and fifth ideas, where technology,

objects, or even activities can be enhanced through smells

from the past, or actual smells sourced through nature (e.g.,

diving in the ocean). Finally, the sixth idea is linked to

Category 8, aiming to avoid embarrassing moments in

social interactions.

Participants also expressed concerns about future smell

technologies. They were concerned about the possible

misuse of smell when sharing it through the Internet or

mobile phones (e.g., teasing people with smells, how to

trust a smell message), and about the potential manipulation

through smell (e.g., TV ads, online shopping). Some

participants were also afraid to get sick, catch an allergy, or

become addicted if they are exposed to chemical

stimulations from technology. Finally, one participant

raised the question of copyright and ownership of smells

(e.g., ‘can I share others’ smells?’).

 

Comment 36

Ideas for technology from HCI researchers

Below we outline the ideas that emerged from the two-hour

brainstorming session prompted by 36 smell stories (one

from each sub-category). Four groups of design ideas

emerged from this session and are described below:

(1) Smell-enhanced performance regulator: a technology

stimulating smell in order to structure the day, taking

activities and moods into account, and combining different

smells to avoid habituation (training and evaluation phase

needed). Smell as a reminder to take a break or as

motivation to keep going a bit longer to meet the deadline

[inspired by #526 ‘White Lilac Sheets’, Category 9].

(2) Autonomous smell agent: a technology spreading

ambient cues (e.g., a robot) to guide someone to a certain

place, to build up expectations, and motivate action. Smell

trails in the environment can also make hidden information

accessible, for instance, before entering a room (e.g., smell

warning: tense working atmosphere) [inspired by #801

‘Don’t forget to check your gas stove before you leave the

house’, Category 5].

(3) Reminder alert with smell: a technology to remind us

about important events, birthdays, and appointments.

Although we have reminders on mobile phones and

computers, they are often ignored, snoozed or in the worst

case forgotten about. A smell can provide a pleasant

reminder to say ‘it is time to call your mom’ by presenting

the smell of your favorite dish your mom makes for you.

On the other hand, if more critical, bad smells can be very

powerful as a reminder and are not easily ignored [inspired

by #530 ‘Don’t want to smell that twice’, Category 7].

(4) Smell-enhanced storytelling: a technology that

stimulates storytelling around a digitized version of an

incense stick. A stick was imagined with different layers,

representing smells related to a loved person who passed

away. When friends or family members come together, for

instance at an anniversary year, they can add new smells to

be shared in the group and thus trigger new stories about

the dead person. It is as if looking through a photo album,

telling the stories from the past, and using the smells as

anchor points for keeping the memory alive [inspired by

#672 ‘The Scent of Moving On’, Category 4].

We saw that the smell stories, even if they only provided

limited information (story, story title, context, gender, and

age), triggered vivid discussions, created empathy, and

stimulated the sharing of personal smell experiences. The

four ideas described above provide a starting point for

exploring smell in HCI. The categorization along with

additional background information on smell qualities and

experienced emotions (see supplementary material) can

inspire further explorations of smell technologies.

 

Comment 37

DISCUSSION

Our findings about experiences with smell in combination

with the ideas for technologies just presented show several

design opportunities for smell. Below we do not provide

solutions for smell-enhanced technology designs, rather we

illustrate where our findings might be relevant to stimulate

novel designs in existing areas of interest within HCI. We

see three anchor points for smell-enhanced technology.

First, the smell stories in Categories 1 and 2 suggest design

opportunities for remembering and recalling the past. Our

findings might enrich ongoing research on the design for

personal memories. Apart from enhancing research

supporting the capturing and sharing of personal

experiences (e.g., in family relationships [18]) through

smell, our findings support research to support people who

are living with memory loss (e.g., patients with dementia

[32]), where smell can play an important role in

remembering the past. An increasing body of research also

explores the potential of digital technologies to support our

memory in everyday tasks (e.g., reminder systems), to

recall past events and experiences (e.g., life-logging tools),

to design end-of-life technologies allowing reminiscence of

passed away people [21], and to record and reproduce

smells [35]. All this research shows the potential of smell to

enrich experiences, for instance by enhancing personal

memories such as photos or videos with smell. Based on

their study of a smell-based photo-tagging system, Brewster

et al. [5] stated that participants asked for personal smells to

be added. The information on how to classify such smells

was still missing; the present analysis allows us to relate

smell qualities to particular types of experiences.

When designing with smell, as for any memory-based

technology, access to such memories has to be considered

carefully to preserve their uniqueness. One participant

wrote: “I could see it being desirable in that it would allow

me to experience the scent whenever I want, but it’s kind of

a two edged sword in that experiencing that scent time and

time again will make it common place” [#513]. The power

of smell might not persist if always available, thus

participants suggested to either restrict the access and

retrieval of smells to special times (e.g., at ‘grandmas

birthday’) or to link them to a certain social setting (e.g.,

smelling only in company with ‘your sister’). This way the

uniqueness of the smell can be preserved.

Second, the stories in Categories 3 to 8 as well as 10 draw

the attention away from past memories and suggest design

opportunities for the present moment. Designing for in-situ

stimulation, the ability to capture and share smells in the

moment, and the capability to mask and neutralize bad

smells creates a vast space for smell interaction design. One

suggestion made by participants was the combination of

smell and social media, such as “An app that would allow

me to store smells, send smells, or attach smells to a picture

that I could post on social media or Instagram or

something”. This supports existing research on the delivery

of smells through the Internet [24]. We draw attention to

three additional design directions concerned with (1) first

time experiences with smell, (2) the power of smell to build

up expectations, and (3) the potential of designing for bad

smells. User experience designers put a lot of effort into

designing ‘out-of-the-box’ and first time experiences to

create positive experiences [13]. Our categorization not

only provides designers with rich descriptions of such first

time experiences, but also describes the related qualities of

smells in combination with descriptors of the experienced

emotions. This can be used to stimulate positive smellenhanced

experiences with technology, build up

expectations, and create anticipation as studied within

experience research [33]. Typically this anticipation stage is

influenced by a variety of aspects (e.g., advertisements,

product descriptions, accounts from existing customers).

Smell stories in Category 10 provide evidence for the

power of smell to build up expectations, create surprise by

exceeding anticipated experiences, and enhance momentary

experiences through capturing and sharing pleasant smells.

Categories 6 to 8 contain stories about bad smells, which

are wished to be neutralized or masked to change the

experience from something negative to something positive.

While the idea of outbalancing smells seems to be

desirable, the design brainstorming session stimulated a

discussion on the usage of bad smell in design, particularly

as part of the design idea (3) Reminder alert with smell.

Designing for bad smells might not seem appropriate at

first, but through intensity manipulation it can open up an

interesting space for design. Similar to a snooze function,

which slowly increases volume, smell stimulations could be

added to certain events (e.g., reminder for mother’s

birthday). Starting with a pleasant smell, it could turn

slowly into something unpleasant if you did not act.

Category 8 also contained stories recounting social

experiences with smell, where the smell of a person or of

other people caused embarrassment or discomfort. Despite

the importance and frequency of social contact in everyday

life, few such stories were shared. They might not seem

meaningful enough to be memorable or to be shared. Yet,

this set of stories holds potential for personal and social

smell-enhanced awareness systems, as well as for wearable

technologies, and smart fabrics. Technology could, as stated

by a participant, “…make the people in those settings feel

more comfortable if I interact with them… My holding my

nose could be insulting and impede communication.”

Third, the smell stories in Category 9 suggest design

opportunities reaching out to the future through positive

stimulation, with potential relevance for wellbeing and

behavior change research in HCI. The stories shared in this

category were about the power of smell to regulate mood,

change attitudes, and behavior. Designing for smell could

be combined with behavior change research in HCI (e.g.,

tools to support healthy nutrition and diet), and thought of

in relation to positive psychology and research on

wellbeing. Seligman and Csikszentmihalyi [28] suggested

that happiness can be learned and cultivated and that

positive psychology can help change how a person feels.

They point to the power of positive emotions for our health,

happiness and wholeness. We would suggest that our

findings add an understanding of the positive emotional

impact of smells that might be a valuable research strategy

in wellbeing research (e.g., for regulating mood).

 

Comment 38

Smell can have a regulating impact on a person’s mood and can, as in

one case explicitly reported, be used to regulate mood

(‘White Lilac Sheets’). The participant wrote, “I guess the

experience could have been enhanced by some kind of

mood moderator. Something that would have sensed my

sadness and filled the room or house with comforting

scents” [#526]. The participant pointed out that technology

would not change the situation to something more positive,

as it just was not a happy time at all, but that it could

support the sad moments in this transitional period of life.

Limitations

We would like to acknowledge three limitations of this

work. First, by using Amazon Mechanical Turk for

recruiting and asking participants to describe personal

relevant experiences, we were limited to the US and do not

know to what extent the smell stories are representative of

more general experiences with smell. We are aware about

cultural and geographical differences (as described by Seo

et al. [29]), which require further studies with a more

diverse group of participants. Second, collecting narratives

by means of an online questionnaire has an influence on

how people narrate their experience and deprives us of the

advantages of an interview situation where we can engage

in a dialogue with the participant to explore the meaning

behind the shared experience in more depth as described by

Bruner [6]. We tried to collect information beyond the

initial trigger of the shared smell stories in order to allow

the establishment of meaningful categorizations and the

creation of a basic understanding of experiences with smell

in HCI.

 

Comment 39

Third, our approach provides an overview on the

emerging field of smell-enhanced technologies. Future

studies will, we hope, lead to in-depth research into

experiences with smell inspired by our identified categories.

 

Comment 40

CONCLUSIONS

Despite interactive technologies increasingly disappearing

into our environment (in ubiquitous and mobile computing)

and becoming essential in everyday life, the senses used to

interact with technology are still limited.

Comment 41

We have discussed the opportunities for smell in HCI based on an

analysis of 439 smell stories. We identified 10 primary

categories for stories about experiences with smell, which

help discuss the potential implications for technology.

Implications were drawn from feedback from our

participants envisaging desired connections between their

own personal experience and future smell technology. The

implications for designing for smell were further enriched

through ideas from an initial brainstorming session with

HCI researchers. Our findings provide guidance for smell

enhanced technology design, not only giving a

categorization of the role of smell in personal experiences,

but also extracting the qualities of smell across the smell

stories and the experienced emotions. We argue that this

research enriches existing technology driven research on

smell in HCI and provides a fruitful starting point when

designing for experiences with smell.

 

Concluding Comment

ACKNOWLEDGMENTS

This work is supported by the Marie Curie IEF Action of

the European Union (FP7-PEOPLE-2010-IEF) and the

Swiss National Science Foundation (PBBSP1 144196). We

thank our participants and especially Annika Haas for her

valuable support in designing the supplementary material.

REFERENCES

1. Alaoui-Ismaïli, O., Robin, O., Rada, H., Dittmar, A.,

Vernet-Maury, E. Basic Emotions Evoked by Odorants:

Comparison Between Autonomic Responses and Self-

Evaluation. Physiology & Beh. 62, 4 (1997), 713–720.

2. Bakalar, N. Sensory science: Partners in flavour. Nature

486, 7403 (2012), S4–S5.

3. Baumeister, R.F., Newman, L.S. How Stories Make

Sense of Personal Experiences: Motives that Shape

Autobiographical Narratives. Personality and Social

Psychology Bulletin 20, 6 (1994), 676–690.

4. Bodnar, A., Corbett, R., Nekrasovski, D. AROMA:

ambient awareness through olfaction in a messaging

application. Proc. ICMI, (2004), 183–190.

5. Brewster, S., McGookin, D., Miller, C. Olfoto:

designing a smell-based interaction. Proc. CHI (2006),

653–662.

6. Bruner, J. Life as narrative. Social Research: An

International Quarterly 71, 3 (2004), 691–710.

7. Chrea, C., Grandjean, D., Delplanque, S., Cayeux, I., Le

Calvé, B., Aymard, L., Velazco, MI., Sander, D.,

Scherer, KR. Mapping the Semantic Space for the

Subjective Experience of Emotional Responses to

Odors. Chem. Senses 34, 1 (2009), 49–62.

8. Ehrlichman, H., Bastone, L. The use of odour in the

study of emotion. In S. Van and G.H. Dodd, (Eds.),

Fragrance: The psychology and biology of perfume.

Elsevier, (1992), 143–159.

9. Emsenhuber, B. Scent Marketing: Making Olfactory

Advertising Pervasive. In J. Müller, F. Alt and D.

Michelis (Eds.), Pervasive Advertising. Springer,

(2011), 343–360.

10. Erickson, T. Design as storytelling. Interactions 3, 4

(1996), 30–35.

11. Faste, H., Rachmel, N., Essary, R., Sheehan, E.

Brainstorm, Chainstorm, Cheatstorm, Tweetstorm: new

ideation strategies for distributed HCI design. Proc. CHI

(2013), 1343–1352.

12. Fleiss, J.L., Levin, B., Paik, M.C. Statistical Methods

for Rates and Proportions. John Wiley & Sons, (2013).

13. Ghinea, G., Ademoye, O. The sweet smell of success:

Enhancing multimedia applications with olfaction.

TOMCCAP 8, 1 (2012), 2.

14. Hassenzahl, M. Experience Design: Technology for All

the Right Reasons. Syn. Lect. on HCI 3, 1 (2010), 1–95.

15. Herz, R.S., Engen, T. Odor memory: review and

analysis. Psy. Bulletin & Review. 3, 3 (1996), 300–313.

16. Jonsson, F., Verhagen, H. Senses working overtime: on

sensuous experiences and public computer game play.

Proc. ACE (2011), 56:1–56:8.

17. Kaye, J. “Jofish.” Making Scents: aromatic output for

HCI. Interactions 11, 1 (2004), 48–61.

18. Kazakos, K., Howard, S., Vetere, F. Revisiting the

relationship between reunion and technology-mediated

separation in periodically transitioning families. Proc.

CSCW (2013), 1157–1168.

19. Loumakis, F., Stumpf, S., Grayson, D. This image

smells good: effects of image information scent in

search engine results pages. CIKM (2011), 475–484.

20. Mason, W., Suri, S. Conducting behavioral research on

Amazon’s Mechanical Turk. Behavior Research

Methods 44, 1 (2012), 1–23.

21. Massimi, M., Moncur, W., Odom, W., Banks, R., Kirk,

D. Memento mori: technology design for the end of life.

Proc. CHI EA (2012), 2759–2762.

22. Matsukura, H., Yoneda, T., Ishida, H. Smelling Screen:

Development and Evaluation of an Olfactory Display

System for Presenting a Virtual Odor Source. IEEE

TVCG 19, 4 (2013), 606–615.

23. Petitmengin, C. Describing one’s subjective experience

in the second person: An interview method for the

science of consciousness. Phenomenology and the

Cognitive Sciences 5, 3-4 (2006), 229–269.

24. Ranasinghe, N., Karunanayaka, K., Cheok, A.D.,

Fernando, O.N.N., Nii, H., Gopalakrishnakone, P.

Digital taste and smell communication. Proc. BodyNets,

(2011), 78–84.

25. Saldana, J. The Coding Manual for Qualitative

Researchers. SAGE (2012).

26. Scherer, K.R. Appraisal considered as a process of

multilevel sequential checking. Appraisal processes in

emotion: Theory, methods, research 92, (2001), 120.

27. Scherer, K.R. What are emotions? And how can they be

measured? Social Science Inf. 44, 4 (2005), 695–729.

28. Seligman, M.E.P., Csikszentmihalyi, M. Positive

psychology: An intro. Am. Psy. 55, 1(2000), 5–14.

29. Seo, H.-S., Guarneros, M., Hudson, R., et al. Attitudes

toward Olfaction: A Cross-regional Study. Chem.

Senses 36, 2 (2011), 177–187.

30. Suddendorf, T., Corballis, M.C. The evolution of

foresight: What is mental time travel, and is it unique to

humans? Beh. & Brain Sciences 30, 3 (2007), 299–313.

31. Tuch, A.N., Trusell, R., Hornbæk, K. Analyzing users’

narratives to understand experience with interactive

products. Proc. CHI (2013), 2079–2088.

32. Wallace, J., Wright, P.C., McCarthy, J., Green, D.P.,

Thomas, J., Olivier, P. A design-led inquiry into

personhood in dementia. Proc. CHI (2013), 2617–2626.

33. Willander, J., Larsson, M. Smell your way back to

childhood: Autobiographical odor memory. Psy.

Bulletin & Review 13, 2 (2006), 240–244.

34. Wright, P., McCarthy, J. Experience-Centered Design:

Designers, Users, and Communities in Dialogue. Syn.

Lect. on HCI 3, 1 (2010), 1–123.

35. Wyszynski, B., Yamanaka, T., Nakamoto, T. Recording

and reproducing citrus flavors using odor recorder.

Sensors&Actuators B: Chemical 106, 1(2005), 388–393.