Festschrift Published Papers

 

John Long Comments on Festschrift Published Papers

 

In my own contribution to the Festschrift (Some Celebratory HCI Reflections on a Celebratory HCI Festschrift), I celebrated ‘the Festschrift papers themselves (both accepted and rejected), their authors and their reviewers’. I also went on to write: ‘My natural instinct is to peer review the papers. Space and my honoured status forbid such a review. However, I hope to do this elsewhere (I owe it to the authors and myself)’. The elsewhere has arrived and is now here.

 

However, on reflection, to peer review the Festschrift papers now seems inappropriate. They were peer reviewed before publication and are unlikely to be re-published, at least in their present form. Nevertheless, I would still like to respond to the papers (‘what I owe myself’) and to contribute to the ideas, expressed in them (‘what I owe the authors’). My ‘response’ and ‘contribution’, then take the form of a commentary – a set of comments on the ideas put forward by the authors. The comments are wide-ranging from simple clarifications to complex suggestions as to how the ideas might be developed further. The comments are intended to be constructive, even when critical. They are my way of expressing my thanks to the authors for their contributions to the Festschrift. I hope they find my comments both interesting and useful.

 

 

 

 

 

 

 

 

Longing for service: Bringing the UCL Conception towards services research

Peter J. Wild

Institute for Manufacturing, University of Cambridge, 17 Charles Babbage Road, UK

 

John Long's Comment 1 on this Paper

Abstract

There has been an increase in the relevance of and interest in services and services research. There is a acknowledgement that the emerging field of services science will need to draw on multiple disciplines and practices. There is a growing body of work from Human–Computer Interaction (HCI) researchers and practitioners that consider services, but there has been limited interaction between service researchers and HCI. We argue that HCI can provide two major elements of interest to service science: (1) the user centred mindset and techniques; and (2) concepts and frameworks applicable to understanding the nature of services. This second option is of major concern in this paper, where we consider Long’s work (undertaken with John Dowell) on a Conception for HCI. The conception stands as an important antecedent to our own work on a framework that: (a) relates the various strands of servicer research; and (b) can be used to provide high-level integrative models of service systems. Core concepts of the UCL Conception such as domain, task, and structures and behaviours partially help to relate systematically different streams of services research, and provide richer descriptions of them. However, if the UCL Conception is moved towards services additional issues and challenges arise. For example, the kinds of domain changes that are made in services differ; services exist in a wider environment; and that effectiveness judgements are dependent on values. We explore these issues and provide reflections on the status of HCI and Service Science.

1. Introduction

As well as becoming an ever more important part of local and global economies, services and service design are emerging, crossing, and in some cases redefining disciplinary boundaries. Papers have emerged in HCI venues that have explicitly examined services (e.g. Chen et al., 2009; Cyr et al., 2007; Magoulas and Chen, 2006; van Dijk et al., 2007). Service has emerged as a frequent metaphor for a range of computing applications, web based, pervasive and ubiquitous: here researchers and practitioners often talk of services instead of applications. This is in addition to a service metaphor in Service-Oriented Architectures (Luthria and Rabhi, 2009; Papazoglou and van den Heuvel, 2007), and the Software as a Service concept. The user, value, and worth centred ethos of HCI (e.g. Cockton, 2006; McCarthy and Wright, 2004), is making its way into service design approaches (e.g. Cottam and Leadbeater, 2004; Jones and Samalionis, 2008; Parker and Heapy, 2006; Reason et al., 2009).

Definitions of services stress the intangible, activity, and participatory nature of services (e.g. Hill, 1977; Lovelock, 1983; Lovelock and Gummesson, 2004; Rathmell, 1974; Shostack, 1977; Vargo and Lusch, 2004a). Hill defined services as ”some change is brought about in the condition of some person or good, with the agreement of the person concerned or economic unit owning the good (1977, p. 318).” This definition suggest that services are activities upon objects and artefacts, both natural (people, pets, gardens) and designed (cars, houses, computers) as well as concrete (e.g. bodies, equipment) and abstract (e.g. education, publishing, therapy).

Comment 2

Hill is also keen to stress the role of exchange, and to distinguish between activities that can and cannot be solely performed by oneself, noting that ”if an individual grows his own vegetables or repairs his own car, he is engaged in the production of goods and services. On the other hand, if he runs a mile to keep fit, he is not engaged because he can neither buy nor sell the fitness he acquires, nor pay someone else to keep fit for him (1977, p. 317).” Hence services are potentially transferable activities performed by self or other to achieve a range of benefits (e.g. save money, sense of accomplishment). Some can be legally enforced onto an economic unit (e.g. tax, insurance, MOT), therefore implying a forced transfer.

Recently, the monikers service science and service systems have emerged from initiatives to support an interdisciplinary dialogue on services (IfM and IBM, 2008). Service systems have been defined as ”dynamic configurations of people, technologies, organisations and shared information that create and deliver value to customers, providers and other stakeholders (IfM and IBM, 2008, p. 1),” with service science being ”the study of service systems and of the co-creation of value within complex constellations of integrated resources (Vargo et al., 2008, p. 145).” There is much in common, both conceptually and empirically between HCI and service science.

Comment 3

Including the goal to create robust and repeatable activities/experiences that are objectively and/or experientially successful; continuing issues in the speed of change in the phenomena they are studying; and the theory–practice gap. However, despite potential opportunities and overlaps, this is not a rebranding of HCI by another name.

Hence, as an emerging area, service science could benefit from HCI’s experience, specifically: (1) the user centred mindset and techniques; and (2) concepts and frameworks for understanding the nature of services.

Comment 4

 

It is the second area that is the major concern of this paper, although we return to the issue of user centred mindset in the paper’s conclusions. HCI has both produced and adopted rich theoretical tooling in its efforts to understand Interaction with and through IT artefacts. Whilst seemingly diverse with ontological and epistemological differences they share a common concern to represent the structure of individual and collective activities in a manner that informs the design of new IT artefacts and activities. This key role of activity representations in HCI is often background in favour of a view centred on the technology being developed. However, it is the activity and latterly the experience of that activity that are being supported/enabled by technology that is one of HCI’s key methodological outputs.

One of this paper’s concerns is with the UCL Conception (Dowell and Long, 1989) one of the conceptual frameworks put forwards for HCI. The conception offers a set of abstractions for HCI, and has guided several streams of work within UCL and elsewhere, Diaper noted that it is ”perhaps the most sophisticated framework for the discipline of HCI (2004, p. 15)”, with its emphasis on effectiveness/ performance being a key part of Diaper’s reasoning behind this assertion. Several of the concepts from Dowell and Long’s work have informed our own framework developed to relate different strands of service research together (Wild et al., 2009a,b) and thus Dowell and Long’s work acts as an important antecedent to our work in services.

Comment 5

 

1.1. Paper overview

With this context in mind, Section 2 is concerned with providing an outline of relevant aspects of services research by covering service definitions. In addition, the section covers our understanding of the UCL Conception (Section 2.2) and examines core concepts from the UCL Conception to different strands of services research (Section 2.2.1). Section 3 first covers the Activity Based Framework for Services (ABFS), our own framework, of which the UCL Conception is an important antecedent. We then consider a number of issues that prevented us from applying Dowell and Long’s concepts as-is to represent and relate strands of services research and model service systems. Finally, we provide a number of illustrative examples of the use of the ABFS for modelling service systems (Section 3.2). Section 4 summarises and concludes the paper, discussed whether services are within the remit of HCI; and if so what they may face when interacting with Service Marketing and Service Operations, two areas that have had a focus on services and have varying claims to user representation and/or involvement.

2. Relevant literature

2.1. Services: a necessarily short and biased overview

It is difficult to trace the growth in importance of services because of differences in the ways that they are defined and reported over time and between countries (Hill, 1977, 1999). Economic downturns aside, a figure that is often cited is that services account for 70–80% of Western economic activity (IfM and IBM, 2008; Parker and Heapy, 2006). During recent years, a number of monikers have been put forwards for a shift to service as the focus of economic and intellectual activity.1 Focus on listing these terms alone will distract the reader and this overview concerns service definitions. The aim is to provide an overview of the different disciplinary perspectives on services.

To even the casual observer the term services embraces a number of different forms and contexts; from intangible services undertaken on abstract objects (such as information and knowledge); via services on people (such as medicine and education); through to maintenance procedures on hardware. In addition, many contexts include all these types of services. Large scale availability contracts for complex products can involve information gathering, forecasting, education and training, the supply of additional tools (e.g. IT artefacts and Support Equipment) in addition to the maintenance of the actual product (Goedkoop et al., 1999; Terry et al., 2007).

Work within Economics and Services Marketing has attempted to provide generic characterisations that show the commonality between these different types of services. The earliest work on service definition was in Economics. Hill (1999) provides a good review of thought on services in Economics, he covers the travails that Economists such as Smith, Say, Senior, Mill, Marshall and Hicks went through in trying to define services. This early work characterised services as different to material goods, and involving different forms of production and delivery. In addition, ownership rights could be established over goods and because of their material nature they can be stored and inventoried, as well as having their life extended through maintenance or remanufacture. In contrast, services were deemed as intangible, variable in quality, and could not be owned or stored. During the 1970s and 1980s, Services Marketing emerged as a discipline in its own right to study flows of services between producers and consumers, working from the view that products and services were different enough to warrant an approach different to mainstream marketing. Two literature reviews (Fisk et al., 1993; Zeithaml et al., 1985) helped to solidify four characteristics as the core distinctions between products and services, namely Intangibility, Heterogeneity, Inseparability, and Perishability (IHIP). Vargo and Lusch summarised these four features as ”Intangibility—lacking the palpable or tactile quality of goods. Heterogeneity—the relative inability to standardize the output of services in comparison to goods. Inseparability of production and consumption—the simultaneous nature of service production and consumption compared with the sequential nature of production, purchase, and consumption that characterizes physical products. Perishability—the relative inability to inventory services as compared to goods (Vargo and Lusch, 2004b, p. 326)”. Lovelock and Gummesson (2004) characterised the IHIP qualities as forming a ‘textbook consensus’ on how services Marketing represented itself to its own students, and to other disciplines. The IHIP characteristics partially enabled services marketing to ‘break away’ from mainstream product-oriented marketing and fuelled a number of research streams, including representation and evaluation of services. Several papers (Lovelock and Gummesson, 2004; Vargo and Lusch, 2004b; Wyckham et al., 1975) question the IHIP characteristics as a foundation for Services Marketing. Some researchers suggest refinements (Hill, 1999; Wild et al., 2007). Others suggest alternative paradigms/logics such as Nonownership (Lovelock and Gummesson, 2004) or the Service-Dominant logic (Vargo and Lusch, 2004a).

We return to the Service-Dominant logic later, but one of the most useful refinements of the core IHIP ideas come from Lovelock (1983) and Hill (1999). Hill (1999) argued for a retention of a distinction between services and goods. However, the dyad needed refining into a triad, with Intangible goods (e.g. books, music compositions, films, processes, plans, blueprints and computer programs) being added. Hill recognised that intangible goods need a manifestation mechanism. Traditionally this has been via physical media, but increasingly the medium is virtual (albeit one which is still reliant on computer memory structures). Lovelock (1983) provided a number of useful characterisations of services, one of which was that they can be tangible or intangible, we gain a fourfold division, tangible and intangible goods; and tangible and intangible service activities (Wild et al., 2009a).

Comment 6

A marketable offering will in general provide a range of such tangible and intangible elements (Shostack, 1977), it is also possible to refine each of the four entities and the relationships between them, as well as relate them to other product and service attributes (Wild et al., 2007).

Outside of economics and marketing, there has been strong and growing interest in services with research undertaken in disciplines such as engineering, manufacturing, computing, and design. A number of approaches have emerged that explicitly tackle the design of services, or the co-design of product(s) and services. Prominent approaches include Product-Service systems (PSS, Goedkoop et al., 1999; Tukker, 2004) and Functional Products (FP, Alonso-Rasgado et al., 2004). PSS has often been associated with the sustainability agenda, with a key idea being the substitution of a general service function (e.g. transportation) for a specific product (e.g. personally owned cars) to reduce the ecological impact of high-levels of under-utilised or inefficient products. Goedkoop et al. (1999) defined a PSS as ”a marketable set of products and services capable of jointly fulfilling a user’s need. The PS system is provided by either a single company or by an alliance of companies. It can enclose products (or just one) plus additional services. It can enclose a service plus an additional product. And product and service can be equally important for the function fulfilment (1999, p. 18).” PSS can be seen as designing the PSS Service-In (Wild et al., 2009b), in contrast the Functional Product (FP, Alonso-Rasgado et al., 2004) and Industrial Product-Service Systems (IPS2, Aurich et al., 2007) approaches work from the Product-Out (Wild et al., 2009b), offering services that can feasibly be offered around a product, or product family. The latter approaches tend to be explicitly motivated to seek additional profit and revenue opportunities from service (Wild et al., 2009b) along with a general shift from ‘product plus parts’ to providing overall product availability (Terry et al., 2007). The design foci for PSS/FP/IPSS can cover the product, its support artefacts, related activities and necessary social and organisational structures (Goedkoop et al., 1999; Wild et al., 2009b). There can be an over emphasis on technical issues in proposed PSS/FP/IPS2 design approaches put forwards (see Roy and Shehab, 2009), however because of the influence of marketing there can be elements of customer representation and interaction in the proposed design processes (e.g. Alonso-Rasgado et al., 2004). Ironically, despite the professed interest in the environmental issues work in PSS has provided little in depth theorisation about ecological and environmental factors (c.f. Costanza et al., 1997; Hawken et al., 1999).

In computing, a service metaphor – rather than mathematical or physical ones (i.e. functions or modules) – has driven developments in Service-Oriented Architectures (Luthria and Rabhi, 2009; Papazoglou and van den Heuvel, 2007). The risk is that SOA researchers see such architectures as being solely concerned with application-to-application interaction (Kounkou et al., 2008), ignoring the fact that these applications are carrying out activities for people. Some work links business process models and SOAs; but service as an analogy for software modularity has been be argued by Kounkou et al. (2008) to miss deeper user centred abstractions based on the needs and values of various stakeholders. Despite a variety of perspectives within the process modelling communities (Melão and Pidd, 2000) there is sparse evidence that a user-centred perspective is being taken in such process modelling efforts. Work in progress is moving towards remedying by integrating HCI knowledge into the SOA development lifecycle practices (Kounkou et al., 2008); or supporting composition of services by non technical users (Namoune et al., 2009). In addition to work in SOA there is concept of Software as a Service (SaaS) (Bennett et al., 2000), whereby an IT artefact is used, but transfer of ownership does not take place (Bennett et al., 2000). These concepts have interacted with the SOA and Cloud Computing communities who have largely focussed on exploring architectures. Again we see a lack of input from HCI knowledge and practice and a corresponding lukewarm reception (Pring and Lo, 2009).

There are a number of service design approaches from the design community (e.g. Cottam and Leadbeater, 2004; Jégou and Manzini, 2008; Nelson, 2002; Parker and Heapy, 2006). In many ways design practitioners are the vanguard of the interaction between HCI concepts and approaches and the design of services (see Jones and Samalionis, 2008; Parker and Heapy, 2006; Reason et al., 2009). Parker and Heapy assert that the prevailing mindset for services is that they are ”seen as a commodity, rather than something deeper, a form of human interaction (p. 8)”. They utilise a number of techniques from HCI, but there is little linkage of their work to explanatory accounts of design processes. Without such explanations it is unclear how much service design success is due to the craft skill of the designers involved and how much is down to the methods and ethos employed. Nelson (2002) a designer and systems theorist discussed various service metaphors (Lip, Room, Social/Public Military/Protective) along with their strengths and weaknesses. Nelson’s concern is to use design processes to enable an approach that combines the best of these service metaphors whilst avoiding their downsides. The goal of such ”full service is adequate essential and significant to the well-being of the clients and stakeholders” (Nelson, 2002, p. 46).

Comment 7

 

Such full service involves: a relationship of maturity and complexity; conspiracy of empathy and creative struggle; a contract between equals where all parties have a voice; provides for the common good; and evokes the uncommon good (Nelson, 2002, p. 416). Nelson has offered little in the way of methodological support, but related work by Cockton in HCI could assist the exploration of service values (e.g. Cockton, 2006).

Nelson’s consideration of values driving design practice and those that should be enabled by services leads us to consider a position has emerged in Service Marketing, the Service-Dominant Logic (SDL Vargo and Lusch, 2004a). Here all marketable offerings, whether classed as goods or services, are considered to provide an element of service. Building on Hill’s (1977) definition presented previously, service is defined as ”the application of specialized competences (knowledge and skills), through deeds, processes, and performances for the benefit of another entity or the entity itself (Vargo and Lusch, 2004a).

Comment 8

Products – whether physical or software – along with services, exist to provide service. When we buy a product such as car or bike, we gain both the product and the benefits of skills of those who produce and supply it to us. When we use a service such as a bus, we gain the benefit from the journey, and the skills and capabilities of the bus company, but also temporary use/benefit of their products. The distinctions between products and services – it is argued – become irrelevant (Vargo and Lusch, 2004b), they are both approaches to providing something the recipient cannot or will not do themselves. Stauss (2005) is one of the most strident critics of this position noting that the fact that physical goods and services both bring about value does not necessarily imply that both are also produced in the same way or they bring about the same kind of value in the same way.

A key part of the SDL is with value-in-exchange rather than value-in-use.

Comment 9

Essentially value-in-exchange views the value or benefit of a product or services as not being realised until the product is used or services are used/enacted. In contrast Value-in-Exchange is the value actually received from use or a product or receipt of a service. Whilst the two concepts have been around since Aristotle, it is claimed the predominant mindset in society has been valuein-exchange (Ramirez, 1999; Vargo and Lusch, 2004a). However, beyond the distinction between value-exchange and value-in-use there remains considerable ambiguity within Vargo and Lusch’s literature on the very meaning of value. Their original paper (Vargo and Lusch, 2004a) uses the term value over a 100 times without definition.2 Since Vargo and Lusch’s earlier presentations a 10th Foundation Principle for the SDL has been added that states ”value is always uniquely and phenomenological determined by the beneficiary (Vargo and Lusch, 2008).” This appeal to the subjective and intersubjective nature of value is predated by work in Economics such as that of the ‘Austrian’ school.3 Finally, the SDL builds on two types of resources. Operand resources: are physical resources upon which an operation or act is performed to produce an effect. Operant resources: are employed to act upon operand resources, and concern issues such as knowledge and skills, they are ”likely to be dynamic and infinite and not static … they enable humans to multiply the value of natural resources and to create additional operant resources (Vargo and Lusch, 2004a, p. 3)”.

HCI may be a discipline well placed to explore the implications of this conceptualisation of product and service use. Its focus on user participation in design processes could be extended to provide analysis and guidance throughout the life of artefacts.

Comment 10

 

There is relevant work on issues such as aesthetics (Hassenzahl et al., 2000; Lindgaard and Whitfield, 2004), experience (McCarthy and Wright, 2004; Sengers, 2003), emotion (Harper et al., 2008), and value(s) (Cockton, 2007; Harper et al., 2008) to draw upon in turning the SDL into a methodologically tractable approach.

Comment 11

 

However, HCI has had to date limited interest in post-delivery usage. Data can be collected as a basis for the redesign of the next version, but the founding methodological principle of early and continuous focus on users and their tasks fades once an artefact is delivered (Wild and Macredie, 2000). To assume that an IT artefact assessed as effective at launch will remain so throughout its lifetime seems naive. Therefore, whilst in principle HCI is pursuing Value-in-Use its actual performance in evaluating it is below its potential and stated ethos.

Finally, our concern turns to Service representation. Shostack (1984) developed and introduced Service Blueprinting, which has become the primary representation technique for services. It originally included: the temporal order of customer and service provider actions; the timings on these actions; tangibles in support of the activities, and the line of the visibility (i.e. actions that the Service recipients can and cannot see in their exchanges with a service provider). Later publications have refined the work (e.g. Bitner et al., 2007; Fließ and Kleinaltenkamp, 2004), including a spiral lifecycle model. Service Blueprinting is described in a numerous services marketing textbooks; and has been elaborated in papers by Fließ and Kleinaltenkamp (2004) and Bitner et al. (2007). There has been an increase in the number of lines of interaction. Fließ and Kleinaltenkamp (2004) list five: (1) interaction: separates customer and supplier interaction; (2) visibility: what customers see; (3) internal interaction: front and back office capabilities; (4) order penetration: activities that are independent and dependent on customers; and (5) implementation: separates planning, management, control, and support activities. Of all the work in Services Marketing Service Blueprinting has been one of the approaches used most ‘outside’ of the discipline, for example, in research and practice in Functional Products (Alonso-Rasgado et al., 2004), Design (Parker and Heapy, 2006), and Product-Service Systems (Morelli, 2006). There is however, no known conceptual or empirical comparison of Service Blueprinting with other methods for mapping processes or analysis of tasks (e.g. CTT, IDEF, BMPN, UML). There are no known studies on the efficacy of Service Blueprinting or its perceived or actual usability by end users of a service. Neither is it clear what depth of user representation and involvement is needed in the approach. Is it simply an attempt for the service designers to represent the service from what they think is the user’s perspective, or does Service Blueprinting demonstrate a deeper philosophical commitment (see Bekker and Long, 2000) to user involvement and representation?

We caution against viewing services research as coherent, despite the large amounts of rhetoric, discussion, support from industrial sources, and recent research funding in the area. Our own modest efforts in this area have shown relationships between activity concepts embodied in approaches such as tasks analysis, process modelling, and the UCL Conception, and a range of research strands in services (Wild et al., 2009a,b), but this remains a long way from a unified and accepted paradigm. What this work recognised was that across the varieties of services research in existence – some of which we have covered here – there are a variety of recurring concerns, but they had not been brought together within one framework. These concerns cover: value and values; the relationship between domains and activities; the relationship between products and domains, and service activities and domains; the relationship between service provider and service recipient and the kind of overt and covert resources that are used in service performance. In turn, concepts within service approaches relate to activity oriented approaches such as Task Analysis, Process and Domain modelling. Dowell and Long’s work was an important antecedent our work, and this paper provide an opportunity to reflect on both its influence and why it could not be used as-is to represent and relate strands of services research and model service systems.

2.2. The UCL Conception

One of Long’s many contributions to HCI is his work with John Dowell on what we label the UCL Conception (see Dowell and Long, 1989, 1998). The UCL (University College London) Conception’s utility has been demonstrated in a number of contexts: it has been used to compare task analysis; scope HCI education syllabi; to help understand change occurrences; modelling task planning, control, perception and execution; emergency management; air traffic control; and cooperative work.

The UCL Conception defines Interactive Work Systems (IWS) as cognitive systems whose scope encompasses two types of participant, people and IT artefacts, interacting to perform tasks in a domain. ‘Domains’ are composed of abstract or physical objects whose attributes may be mutable. Tasks are activities that are concerned with changing these domain object attributes. Organisations express their requirements for changes to a domain through the specification of goals. A ‘product’ goal is scoped towards a domain and is the intention to change several attributes and objects in the domain. A product goal breaks down into a number of ‘task’ goals that alter individual attributes. Different forms of task are recognised notably interactive, offline, automated (Dowell and Long, 1989; Lim and Long, 1994) and enabling (Whitefield et al., 1993). With the later being tasks that put the IWS into a state where it can be used (e.g. booting, opening applications). Each participant of an IWS (i.e. person or IT artefact) of an IWS has ‘structures’ and ‘behaviours’ that support task performance. ‘Structures’ provide capabilities in reference to a participant’s environment and can be physical or abstract. ‘Behaviour’ is the activation of structures to execute the changes in the IT artefact and domain. Structures are physical (e.g. electronic, neural, biomechanical and physiological) or abstract (e.g. software or cognitive representational schemes and processes). Similarly, behaviours may be physical, such as printing to paper or selecting a menu, or abstract, such as deciding which document to open, or problem solving. The many-to-one mapping between people’s structures and their behaviours allows the production of different behaviour from the same physical and psychological structures.

Comment 12

 

Dowell and Long (1989) view effectiveness (later termed performance) as a function of task quality (the quality of a ‘product’ created by an IWS) and resource costs (the costs to participants of establishing structures and producing behaviours). Desired effectiveness is set by specifying desired task quality and the desired resource costs. Similarly, actual effectiveness can be measured as a function of the actual task quality and the actual resource costs (Dowell and Long, 1998, p. 139). Task quality pertains to goals, that is, what do people and organisations want from a domain (e.g. speed, vs. quality). Structural resource costs relate to costs of setting up and executing structures and behaviours of both IWS elements to carry out tasks. Whilst behavioural resource costs are those incurred for actual task performance. There has been no attempt to produce – or link to – a taxonomy of development costs, beyond the distinction between structural and behavioural costs.

Comment 13

Dowell and Long view the domain as a distinct part of an IWS’s environment – it is the world in which tasks originate, are performed, and have their consequences. The domain characterisation is ‘oriented to objects’. Domains are composed of abstract and physical objects with attributes. These attributes have a state that may be able to change. Attributes can be physical or abstract, and may need to be inferred when the domain is studied. Objects can have both abstract and physical attributes. Printed texts, have abstract properties that support the communication of messages and physical properties that support the visual representation of information (Dowell and Long, 1989), there is therefore a coupling between the domain objects and entities with Structures and Behaviours capable of perceiving their affordances. Dowell and Long (1989) maintain that levels of complexity emerge amongst attributes at different levels of analysis. Attributes that emerge may subsume those that arise out of lower levels; thus, a printed text could be a letter, tax return or an instructional text. Dowell and Long (1989) stress that objects be described at an appropriate level, but give no indication of how to do this.

Comment 14

 

 

Their work has emphasised physical changes and cognitive processes (i.e. changes to information and knowledge), little is said about social or emotional changes.

Comment 15

 

2.2.1. Relating the UCL Conception to services research

We now consider how the entities within UCL Conception relate to varying strands of services research. Table 1 provides a comparison of core concepts of the UCL Conception with certain strands of services research across different disciplines.

Comment 16

Given these overlaps between concepts within services research and the conception at first sight, the concepts could provide framework to situate and relate the various disciplinary strands of service research. However, we argue that the Dowell and Long’s work cannot be used as-is to represent and relate strands of services research and model service systems. The reasons are varied and include: services exist in a wider environment (Section 3.1.1); effectiveness judgements are dependent on values (Section 3.1.2); service demands a richer notion of people (Section 3.1.3); different kinds of abstract objects need to be represented and reasoned about (Section 3.1.4); the relationship between a core and service system needs to be represented (Section 3.1.5); and going beyond engineering (Section 3.1.6). We ask the reader to bear with us whilst we provide an overview of the ABFS. Which then allows us to address and expand on these points.

3. The activity based framework for services (ABFS)

Working from the view that services are consistently defined as activities – rather than objects or artefacts – the concepts (e.g. Roles, Domains, Actants, Artefacts, Goals, Tasks) of the ABFS are drawn from activity modelling approaches, such as task analysis (Diaper, 2004), domain and process modelling (Dowell and Long, 1989, 1998; Melão and Pidd, 2000), and soft systems methodology (Checkland and Poulter, 2006). This synthesis produced a framework that can relate together the disparate streams of service research (Wild et al., 2009a) and help classify the design Foci (i.e. what is being designed) of service design approaches (Wild et al., 2009b).

Comment 17

 

Tasks: the majority of service definitions class services as tasks, but they are perhaps most prominent in the Service Blueprinting approach (Shostack, 1984). There are classifications of service tasks, focussing around organisational division (Shostack, 1977); economic relationships (Hill, 1977, 1999); the relationship to a product (Tukker, 2004) or a more ‘general’ consideration of their nature (e.g. Lovelock, 1983) Goals: are not a first class entity in most approaches to services, but can be tacit in Service Blueprinting and in discussions of value (Flint, 2006; Vargo and Lusch, 2004a) People: are considered as co-creators of value in the Service-Dominant Logic (Vargo and Lusch, 2004a); are implied by lines of visibility in Service Blueprinting; and within the IHIP qualities are implied by Inseparability, and their role in service paradigms such as Nonownership (Lovelock and Gummesson, 2004) IT artefacts: the Service-Dominant Logic argues that software, along with tangible products and services are a mechanism for delivering benefit to another party, this mirrors work in computing/information systems that promotes a view based on Software-as-a-Service (Bennett et al., 2000). Beyond such a high level and general statements there is little work that examines the overall role of IT artefacts in service within the services community Domain: alongside tasks, domains have a close conceptual correspondence to the IHIP debates and refinements. In Section 2.1, we noted that there is a distinction between tangible and intangible products and tangible and intangible activities. We argue that there is a correspondence between intangible and tangible activities and products, and concrete and abstract IWS tasks and domain objects Structures and behaviours: are covered in the SDL as the resources (operand and operant) different parties bring to service exchanges. Often the terms are used loosely in other services research, and more importantly in a non-systemic manner Effectiveness, is timeand cost-oriented in Service Blueprinting, and whilst work service quality evaluation has often focussed on perceived or experienced Reliability; Assurance; Tangibles; Empathy, and Responsiveness (Seth et al., 2005), with some later work focussing on personal values (Lages and Fernandes, 2005). In Services Marketing, generally revenue and profitability have been high-level effectiveness measures

Table 1: Comparing the concepts of the UCL Conception to services research strands.

The ABFS is represented schematically in Fig. 1 service activities are carried out within a service system. The system embraces the objects (both abstract and physical); the goals and values held by various individual and collective Actants. Activities are carried out by actants and artefacts (both IT and non IT) to affect the objects in a domain. Not illustrated in the representation is the potential for an overlap between the domain and actants – artefacts. This overlap captures potential recursive relationships between people and domains (e.g., self directed education), and the distinction between coherence and correspondence domains (Vicente, 1990), that is, domains that ‘exist’ virtually within an IT artefact (e.g. 3D graphics, the internet).

 

 

A service system can be considered to have a variety of success measures, depending on the value (i.e. benefit) sought, and this is evaluated when the quality goals are balanced against the resource costs. Resource costs include affective, socio-cultural costs, as well as the consideration of the physical environment discussed by Stahel (1986). A service system has an environment, which has sociocultural and physical dimensions. Borrowing from Dowell and Long (1998), we suggest that actants and artefacts have structures and behaviours. Long and Dowell’s concepts generalise and we assume a wider set of structures and behaviours, namely the physical and socio-cultural (Stahel, 1986). Thus, we assume physical (ecosupport system, toxicology system, flows-of-matter system), and socio-cultural (Elster, 2007; Hall, 1959) structures and behaviours alongside those of artefacts and individuals (i.e. the IWS). The costs of setting up and maintaining these structures and behaviours are evaluated in service success assessments. This in turn depends upon the set of values that can be identified as applicable to the service system.

Comment 18

A comparison of the ABFS concepts against specific streams of services research can be made, in Table 2, (adapted from Wild et al., 2009a) presents our assessment of the depth of consideration of varying approaches against the core concepts of the ABFS.

Comment 19

 

Several additions and considerations mark a departure from the Dowell and Long’s concepts, due to the need to engage with the services literature and with modelling service systems. Notable divergences and additions are: the inclusion of artefacts other than computers; the inclusion of a physical and socio-cultural environment; the inclusion of values; the reintroduction of affective and conative resource costs; an environment other than the domain, and the potential overlap between the domain and actants or tools.
The original purpose of the ABFS was to relate different strands of services research (see Wild et al., 2009a). However, recent work has started to examine the framework as a modelling approach. Specific interest is modelling service systems for complex engineered products, and the transitions to different configurations, possibly more ‘effective’ ones. The aim is to produce high-level and integrative models that illustrate: the systemic nature of service systems; the distinction between different kinds of domain; how different values affect the overall structure of a service system. An ABFS ‘model’ can be considered an elaborated representation of a Human Activity System (see Checkland and Poulter, 2006), and act as a general system model.

3.1. ‘Barriers’ to moving the UCL Conception towards services

Having provided an outline of the ABFS including details of the work we draw upon we examine the barriers within the UCL Conception to its use raw as a framework for relating wider strands of services research and providing models of service systems.

3.1.1. Service activities exist in a wider environment

The environment tends to become a catch-all in activity-based approaches, covering all the things that are not first class entities in their modelling worldview. Those with a systems orientation may try and draw a boundary around the core system of interest.

Comment 20

However, boundaries are rarely easily defined (Mingers, 2006), existing across both socio-cultural, physical and computational levels and often being observer dependent (Checkland and Poulter, 2006; Mingers, 2006). In the UCL Conception, the domain is the only ‘environment’ represented, thus, whilst it is ecological, it is weakly so.

Comment 21

Outside of the domain or IWS nothing is discussed about context. How the domain is distinguished from the general environment is related to goals – they refer to desired changes in part of the world but what processes demarcate the domain are not clear.

Comment 22

 

Table 2: Comparison of the ABFS concepts against specific streams of services research

Dowell and Long (1998 p. 130) admit that a domain cannot be completely formalised, but it is not clear whether the domain’s boundary with the rest of the world is open or closed or something in between.

Comment 23

 

We argue that to embrace services research it is important to represent not just the immediate environment of interest to services activities (i.e. the domain), but the wider environment within which activities are carried out. Here we find that HCI approaches can benefit from environmental concepts within services research strands such as the functional economy (e.g. Stahel, 1986). Within our own work, a secondary environment was synthesised by drawing on Stahel’s (1986) work on the Functional Economy, along with other work (Elster, 2007; Hall, 1959). This secondary environment is assumed composed of a socio-cultural system, along with a physical system.

Comment 24

 

The former is discussed and the intellectual frameworks we drew upon are discussed in the presentation of the ABFS. The latter breaks down into: the eco-support system for life on the planet (e.g. biodiversity); the toxicology system; and the flows-of-matter system, especially as they relate to recycle and remanufacture decisions. Both elements provide potential sources of quality and resource cost measures (e.g. social coherence and pollution) for judging the effectiveness of service systems.

3.1.2. Effectiveness judgements are dependent on values

Effectiveness is taken to refer to whether activities are achieving some higher level or long term goal (see Checkland and Poulter, 2006, pp. 42–44), and we assume the term was originally chosen to distinguish it from efficiency (using resources well); and efficacy (that the activity is working).

Comment 25

 

Values are the criteria with which judgements are made about other entities (Checkland and Poulter, 2006). There has been a steady increase in the use of values and related terms such as value, quality, choice and worth (e.g. Cockton, 2006; Karat and Karat, 2004; Light et al., 2005). Cockton (unpublished) notes that HCI’s original quest to be a scientifically and engineering oriented discipline led it to background values, and not examine its own values as a discipline.

In our context, our interest is a little more mundane with the impact that different value sets and value choices have on the configuration and judgement of service systems. Values affect how actants view the effectiveness of service system configurations that are acceptable and unacceptable (e.g. labour and time savings vs. social cohesion and fight reduction). For example, placing high value on one’s carbon footprint can lead to transport choices such as cycling and walking which may be in a trade-off situation with time use goals and values. Alternatively, certain values and their trade-offs may lead to alternative behaviours such as carbon offsetting, using renewable fuel and lift/car share schemes.

Values are key aspect of scoping the other elements of a service system. The form of the service system can reflect the values held by its actants. Whilst a high-level goal of restaurants is to provide food and generate profit; the values held by the owners, staff and patrons can drive radically different manifestations of eating location, menu, and experience. Comparing a high-class joint with a roadside catering outlet without reference to values would be meaningless. Yet their basic transformations and domain objects remain remarkably similar: the preparation and serving of foodstuffs.

Comment 26

Some individuals and groups see their values as true and objective, those who do not share your values are classed as having none (Beck and Cowan, 1996; Goodwin and Darley, 2008). Witness the view by many in the ‘environmental’ movement that shortterm non-resource renewing profit driven enterprises have no values. Such enterprises have values, but they are focussed on assessing effectiveness by profitability, capital liquidity, rather than sustainability. It becomes a misnomer to claim to be placing human values at the core of the HCI discipline (see Harper et al., 2008), as all our values are human. A human valuing technological progress and subscribing to technological determinism – whilst potentially passé – is still holding and enacting human values, to claim otherwise would be claiming a truly objective and non-human position on values. The issue is whether those values are reflected in the design process and in some way their maintenance, or desired change is supported by the entities (artefacts, processes, social structures) we design, and whether and how design methods and entities promote the reconciliation of differing valuing systems.

Comment 27

 

 

With respect to services one key trend in recent years has been the emergence of availability and capability contracts (Terry et al., 2007; Tukker, 2004). These go beyond outsourcing to contractual arrangements where two or more partners work together to deliver services. In many contexts, this brings commercial and noncommercial organisations together with a potential for clashes of values, the most obvious being when public sector services interact with commercially oriented organisations. Furthermore, these arrangements rely on the service recipient providing facilities back to its supplier, with both parties acting as supplier and recipient of services.

In other contexts, service design is tackling the design of public services, another form and context that can bring together different actants and values, from the efficiency driven targets beloved by bureaucrats and politicians to those concerned with retaining, or promoting broad and difficult goals such as community cohesion and sense of community participation (Parker and Heapy, 2006; Seddon, 2008). If HCI’s and service science’s frameworks cannot acknowledge the existence of values in relation to the effectiveness of activities and artefacts, they will be impoverished. Promoting values to a first class entity within HCI and services frameworks opens them up to a richer set of considerations about human action. Recognising that values exist in relation to goals, activities, desired and actual transformations adds more depth to the characterisation of values. They are not discussed context free, with the ”it all depends on” caveat, they are scoped in reference to actions within a domain and an environment.

Comment 28

3.1.3. Service demands a richer notion of people than the UCL Conception provides

Dowell and Long (1989) originally classed resource costs as being cognitive conative (motivational), and affective (emotion).

Comment 29

The later paper (Dowell and Long, 1998) dropped affective and conative resource costs. In our work, our ‘enrichment’ of the conception of people has been twofold. In the first case the recognition that values play an important role in shaping goals and effectiveness criteria and in scoping wanted and unwanted resource costs of a service system (see Section 3.1.2). The second is that the notion of resource costs and quality measures can embrace conative, affective, and socio-cultural issues.

Comment 30

 

The general argument for structures and behaviours can be expanded to cover emotional, sociocultural issues. Work by luminaries such as Teasdale and Barnard (1993), Elster (2007), Hall (1959), and Beck and Cowan (1996), Cowan and Todorovic (2000) suggests phenomena akin to structures and behaviours. However, the nature of these additional structures and behaviours both differs from, and interrelate with cognitive ones. They can probably be assumed to exist at different emergent levels of reality (Mingers, 2006) and vary in their objective, intersubjective and subjective qualities (Heylighen, 1997; Mingers, 2006). For most purposes activities can be seen to be simultaneously affective, cognitive and in some way socio-cultural. Whilst it makes the implementation of an engineering vision for HCI harder to achieve as prescribed by Long (Long and Dowell, 1989) it should lead to a broader understanding of how different kinds of structures and behaviours are involved in understanding the effectiveness of activities; for example, how a service deemed to achieve its goals efficiently and could fail experientially, socioculturally or incur physical environmental costs that cannot be sustained and vice versa.

3.1.4. Different kinds of domain objects need to be represented and reasoned about

The domain concept provides a useful abstraction for the modelling of service systems. What are the objects?, what is their nature?, are simple questions that force the analyst to grapple with what the service activities will be ‘doing’. Allowing us to consider similarities and differences between different service contexts Dowell and Long (1989) stressed that objects be described at an appropriate level, but give no indication of how to do this.

Comment 31

 

 

A legitimate goal of an activity could be to affect emotional, socio-cultural ‘objects’, for example entertainment, education and health services all alter personal or socio-cultural entities (Hill, 1977). In theory, the domain concept can generalise to cover alternative kinds of objects. Hence, there needs to be the recognition that ‘objects’ in the domain could have properties that are not just physical (e.g., material or energy) and information or knowledge. There is ambiguity about the status of social and affective issues in Long’s work. Green (1998) noted that Dowell and Long’s work in air traffic control covers social issues; but they are not considered as first class concepts within the UCL Conception. Nor are they explicitly discussed when discussing the determinism boundary (Dowell and Long, 1989). In turn affective and conative resources costs were covered in the first paper (Dowell and Long, 1989), but later dropped (see Dowell and Long, 1998).

Comment 32

 

There is a requirement to be able to model, and reason about, emergent properties within a domain that are motivational, emotional, and socio-cultural.

Comment 33

Hence, we suggest that activities can aim to alter emotional, socio-cultural, states along with the informational and physical objects. Despite views to the contrary, advice on how this could be achieved cannot be found in work on analysis patterns or domain modelling (e.g. Fowler, 1997; Sutcliffe, 2002), which share similar modelling concepts, but give little consideration on the different ‘kind’ of domain objects altered, and they should therefore be modelled.5 The development of a computer based representation of a domain, is the creation of a representing world, however these representing worlds are not grounded in a perspective that can handle different ‘kinds’ object with any theoretical depth. They provide no theory to allow us to distinguish between situations where, for example, fun is an evaluative criterion alongside others (e.g. work applications); where it is the goal of the activity (games and theme parks); or it is balanced against other factors such as knowledge gained (e.g. modern interactive museums).

3.1.5. Core system, service system

When modelling service domains, we need to represent the overlap between at least two different Human Activity Systems (HAS). Service systems are set up to support another human activity system, hence we posit a relationship between core-system and one or more service systems, and Fig. 2 illustrates the distinction. The darker boxes represent part of the Core human activity system, with the service system, represented by the lighter boxes. The actants, artefacts, and sometimes aspects of the domain become the domain of the support system. The support system will utilise its own tools and artefacts in support of the core system. It may however share artefacts, actants, and activities with the core system. Fleet planning and forecasting of assets are example of such shared activities.

In reality, the resulting model may be more complex than the structure represented in Fig. 2. Many contemporary service contracts are complex, in terms of both the core domain (e.g. healthcare, armed forces); but also the complexity of the contractual arrangements: involving multiple partners, long time scales and complex payment and reward mechanisms (Terry et al., 2007). There may also be a chain of service systems in a series of core/support relationships, the service support may draw on other service systems such as IT or payroll.

Comment 34

Fig. 2. Overlapping systems (core and service).

The distinction between core and support system could be modelled by including a rich set of ‘parallel’ enabling tasks (Whitefield et al., 1993), and expanding the range of (enabling) tools and domain objects. We argue that for most significant modelling exercises the use of the enabling task concept masks the importance of distinguishing a relationship between different systems undertaking different roles. The notion of a core and service domains allows the representation of issues such as clashes in values and visibility, in reference to different kinds of activity (e.g. physical maintenance, forecasting, and education), artefacts, and domain changes. However, modelling approaches are often used by different modellers in different ways (Melão and Pidd, 2000), the enabling tasks approach may also be suited to illustrating differences between self services and services provided by another party.

3.1.6. Going beyond engineering

Whilst the HCI 89 keynote (Long and Dowell, 1989) offers alternative perspectives such as HCI as craft and applied science; and acknowledge that HCI will provide knowledge embodied as principles, heuristics, and guidelines. Long’s work is probably most heavily associated with the promotion of an Engineering perspective on the development of IT artefacts (e.g. Dowell and Long, 1989, 1998; Lim and Long, 1994) and handling of human factors from an engineering perspective, such as specify and implement, and the development of engineering principles (Long and Dowell, 1989). It remains an open question whether ‘engineering’ principles can be widely developed. There are few in existence (see Cummaford, 2000; Johnson et al., 2000), and their development takes considerable resources.

Comment 35

 

HCI has expanded its focus to embrace social, hedonic, and experiential concerns (Hassenzahl et al., 2000; McCarthy and Wright, 2004) and focussed on the support of community and family activities (Harper et al., 2008) this engineering perspective has begun to look in some way problematic (e.g. Hassenzahl et al., 2001; Sengers, 2003). There is much to be said for an engineering perspective and none of what follows such be taken as a dismissal of the power of the engineering approaches. However, what we need to be aware of are situations where the overall success of service design and delivery will be dependent on the interplay of factors that are quantifiable, and repeatable, as well as personal, experiential and socio-cultural factors, that may be more tacit, subjective and variable.

Comment 36

 

A service or IT artefact that makes the required transformations effectively and efficiently can still be considered a failure if the customer, whether individual or organisational, has been subject to issues such as: over or under inclusion in decision making; reduced cohesiveness of social structures; reduced visibility; or recipients left feeling belittled, redundant, or deskilled by the service professionals. In addition, the failure to address such softer issues can lead to a breakdown in being able to obtain data about traditional quality measures and processes: data may not be recorded and passed on, equipment may be mishandled and shared assets may be withheld. Conversely, a service recipient can be made to feel welcomed, at ease, and involved, but without technical competence in executing the service, the service could still fail. For example well treated, but misdiagnosed patients, or service recipients being involved in decision-making processes that fail to fix faults in equipment.

Comment 37

 

When we move towards services, many services are carried out in distressing, and ultimately unwanted circumstances. A designer or researcher coming in and declaring either an old school (HCI as engineering of efficiency, errors and measurement of satisfaction) or new school (HCI as promotion of hedonic measures)6 approach should receive short shrift. Assessments about the effectiveness of such services need to balance this factor against other measures of effectiveness. Questions such as these cannot be handled within a single perspective dominated by engineering and work concerns (i.e. Dowell and Long, 1989, 1998; Long and Dowell, 1989),

Comment 38.

but equally someone viewing interaction as maximising hedonic issues would also fail. This is a subtly different proposition from preventing the decline of an emotional state – which could be a laudable goal of such service types (e.g. counselling services) – and the promotion of a hedonic experience. However, we maintain that much of the spirit of the concepts outlined in the Dowell and Long’s work generalise, particularity the explicit notion of structures and behaviours and the implicit notion of trade-offs in the design of systems.

Comment 39

3.2. Illustrative examples

Within this section, we provide illustrative examples. The first is a simple illustration taken from a novel by Gemmel; the second presents an initial ABFS for a Family’s transport choices, given the particular value set they wish to enact.

3.2.1. A literary example

An implicit issue that can be drawn out the UCL Conception characterisation of task qualities and resource costs, is that trade-offs can be made between different system configurations. The inclusion of a wider set of costs allows the representations of wider trade-offs, both as design options within a single IWS-domain coupling and in relation to different classes of application domains. In some situations, tasks/goals are harder to learn/achieve, but provide greater enjoyment (e.g. games); others are more efficient in terms of achieving the goal, but have a different ‘affective profile’. An IT artefact could be made more efficient, but at great cost in terms of the processing power and memory it needs, thus additional tasks are allocated to the user in the IWS.

A literary example of these trade-offs can be provided by Gemmel. ” ‘A wagon and a single driver would be more effective, surely?’ Observed Skilgannon . . . Landis smiled . . .. ‘In the main however they just bring food. You speak of effectiveness. Yes, a wagon would bring more supplies, more swiftly, with considerable economy of effort. It would not, though, encourage a sense of community, of mutual caring.’ (Gemmel, 2004, p. 52)”. Within this example, delivery of meals to the loggers in the story is done by the women of the local town, some married to the loggers, some not. Leaving aside the somewhat old-fashioned gender roles – this arrangement whilst less efficient than one man and a wagon – is continued because it promotes social cohesion, the presence of the women means less fights between loggers occur, and new relationships are formed. Thus, the domain can be seen to embrace both the physical changes of food preparation, but also affective and social states. Within this book, this form of catering arrangement leads to a key relationship between two characters, one that shapes the drive and motivation for one of those characters throughout the remainder of the story. These forms of decisions and trade-offs surround and permeate many activities both social and work based (see Wakkary, 2005; Wild et al., 2003). With a wider set of quality assessment criteria and resources to consider we can investigate the trade-offs that can be made when setting up a service system, rather than the narrower conception of IT effectiveness just physical and cognitive changes and costs.

Comment 40

3.2.2. Family travel services

Our next example uses the following scenario to drive ABFS models of services:

The Bloggs family live in Sandford, 8 or so miles west of the City that the parents work in. The parents work at a science park, that is well served with buses to the city centre, but there are no direct buses from their village to their workplace. They chose the village for the many benefits village life can bring, especially for their children (e.g., low crime, sense of community, proximity to the countryside, air quality, good local schools) even though it would entail extra travelling and decisions about travel planning. The youngest child attends the Sandford village junior school, however, the nearest high school is a village college about 5 miles away in Fordham, in the opposite direction from to the city they work in, although the college is served by a school bus. They consider themselves environmentally aware, community focussed and wish to support their children in their exploration of a range of hobbies and interests. However, as busy professionals they are aware that the greenest transport options – via bus services – conflict with the time constraints of their daily lives and parental responsibilities. Sandford includes some local amenities such post office/grocery, hairdressers, pub, several private businesses. Services such as general practitioner, dentists and vet are dispersed within local villages. The closest town has all three but is a different county, and health authority. The family choose a neighbouring village that has all three, and try and group routine visits for all the family. The nearest supermarkets are in several towns between 6 and 9 miles away. The nearest rail station to London via fast train is 6 miles or £20 round trip taxi journey. They live within 20 miles of two regional airports.

The core objects of this domain are people, things, locations, deadlines (both hard soft) and schedule. Transport artefacts include bikes, cars, buses, and delivery vans. Additional states to be changed or maintained include the emotional security of their children about travel, and the social fabric of the village (which is currently healthy with around 30 clubs societies, etc.). The actants include family members, friends of the family, and those providing transport related services. The generic goal for their transport services is to transfer family members and where appropriate friends, and things (possessions, new purchases) to different locations.

They have a number of high-level values and goals that they bring to their choice of transport and service choices:

• (A) Commitment to being carbon neutral and reducing extraneous journeys.
• (B) Supporting local businesses (biofuel company 10 miles away but this is the nearest local supplier).
• (C) Commitment to supporting their local and neighbouring village communities.
• (D) Commitment to ethical or state owned finance providers.
• (E) Reduce time costs of functional activities (travel to work).
• (F) Loyalty to a supermarket chain from the region they grew up in.

We examine, using ABFS constructs, the following transport service activities:

1. Use of Supermarket delivery service, with top up in the local store.
2. Eldest child catches school bus, rather than being dropped at school.
3. Youngest child walked to school by the eldest child.
4. Travel to work in car, rather than cycle (40 min on high speed road without cycle lanes) or bus (two journey 1.5 h travel time).
5. Use of biofuel (despite 8 mile diversion to pick up, and facilities to store spare), Carbon offsetting services.
6. Support village driving scheme (local scheme to offer lifts to people in the village without car).

These particular travel examples provide a broad demonstration that ABFS concepts are applicable to high-level models of Service activities.

Comment 41

They also demonstrate a close relationship between values held and high-level cross activity goals. Future work will need to clarify the nature of these. In previous work we suggested that such high-level goals could be represented within a heterarchical goal complex (Wild et al., 2004 see also Diaper, 2004). However, this could be a convenient representation for what are distinct cognitive representations (goals vs. values). At this level of representation, socio-cultural behaviours can be considered, but making the reasoning more formal will need greater theorisation. We outline some sources for this in the next section (see Table 3).

	Option 1	Option 2 & 3	Option 4	Option 5	Option 6
Domain	People, things, locations, deadlines (both hard soft) and schedule Maintain positive affective states for children, maintain/support village socio-cultural structures
Activities	Make list, make order, receive items	Walk to junior school, catch bus from there	Drive, navigate, park	Calculate mileage make payment Drive to supplier fill tank and barrel	Pick up person, plan route plan time
Goals	New possessions transported from store to home Parent	Both children safely transported to school	Parent A and B transported to and from work with possessions Parent A and B	CO2 for car offset	Person X, transported to location at time Y
Actants	Parent	ChildA, ChildB bus driver	Parent A and B	Parent A and B, Service provider, offset projects	Parent A and B, Service recipient, scheme organiser
Artefacts	Computer, website, delivery van	Timetable, bus stops bus, school, college, roads, etc.	Car, road system, car park	(1) Computer, website, bank account (2) Fuel tank, fuel pump, barrel	Car, road system, car park, booking forms telephone
Effectiveness	Shared shopping list Required items delivered on time	Child regularly at school/college on time	Journey regularly 15–20 min and no more	CO2 offset, but extraneous journeys are not encouraged Right fuel at the right time	Person X transported to location at time Y
S&B socio-cultural	Subsistence, association, protection Territoriality, Temporality	Learning, protective, association	Association, Subsistence (work)	Subsistence, Exploitation	Subsistence, Association, Interaction
Benefit	Reduce travel and shopping time	Reduce travel time for parents	Reduce travel time for parents	Reduction of guilt about greater car usage village life can entail Support economic communities that are beneficiaries of offsetting funds	Reduction of guilt about greater car usage village life can entail Support community, meet new people
Values	Support regional supermarket Support local store Reduce family CO2	Pride in children’s independence, maturity and behaviour Support local school bus service Encourage independence in children Children travel with trusted companions Reduce family CO2 Reduce personal CO2	Reduce temporal costs of functional activities	Commitment to being carbon neutral Support ‘local’ business	Support community
Costs	Delivery charge	Less time spent with children on journey	Car running and depreciation costs CO2 costs Need to build exercise into routine	Financial cost of offset fee Extra mileage to pick up fuel Registration cost for fuel supplier	Car depreciation costs CO2 costs
Trade-offs	Local store is pricey Lack of coordination could increase traffic to village for multiple deliveries Less chance to teach children about subsistence & economics Less chance for bargain hunting	50 miles a week in the car vs. more time to talk with childA	Adds to general traffic levels Cycle ride could provide exercise Risk of cancellation of bus services due to less customer	Higher rate of fuel filter changes needed Extra travelling to pick up Biofuel Biofuel less suitable for winter use	Combine lifts with other activities

Table 3: ABFS inspired representation of family travel services.

3.3. Towards formalisation of the additional elements

Within the preceding examples, our use of the additional concepts of the ABFS has been relatively informal despite our hints that relevant work could be used. The socio-cultural structures and behaviours were informally drawn from Hall’s work but without explanation, they can seem rather isolated. Table 4 presents additional elements to be used in future modelling efforts. They are put forwards as a checklist of options, rather than as a validated theoretical classification of resources. Several can co-occur, and others can be broken down in more detail. In the current state of evolution of ABFS the dimensions are put forwards as a checklist of qualities and/or resource costs. They should be considered as heuristics and despite their pedigree should not be considered as validated principles for the examination of affective and socio-cultural issues.

Type	Description
Conative	Physio sensory experiences Socio collective or shared experiences Psycho individual enjoyment and satisfaction Ideo intellectual and aesthetic experiences
Affective Evaluative emotions	Shame – negative belief about one’s own character Contempt and Hatred – negative belief’s about another character the former is that they are inferior the latter that they are evil Guilt – negative belief about one’s own actions Anger – negative belief about another’s actions towards oneself Cartesian Indignation – negative belief about another’s action towards others Pridefulness – positive belief about one’s own character Liking – positive belief about another’s character Pride – positive belief about one’s own actions Gratitude – positive belief about another’s actions towards oneself Admiration – positive belief about another’s actions towards a third party
State emotions	Envy – caused by the deserved good of someone else Aristotelian Indignation – caused by the undeserved good of someone else Resentment – caused by the reversal of a prestige hierarchy Sympathy – deserved good of someone else Pity – undeserved bad of someone else Malice – is caused by the undeserved bad of someone else Gloating – is caused by the deserved bad of someone else
Socio-cultural	Interaction – interactional Association – organisational Subsistence – economic Bisexuality – sexual Territoriality – territorial Temporality – temporal Learning – instructional Play – recreational Defence – protective Exploitation – exploitational

Table 4: A checklist of factors for conative, affective and socio-cultural qualities and resources

Our next concern is to present checklists of items for Conative, Affective and Socio-Cultural resources, drawing on work by Tiger (2000), Hassenzahl (2000), Elster (2007), and Hall (1959). To further scope conative goals, we use Tiger’s (2000) ‘imperfect categories’ to outline four major groups of motivators for products and services. For affective resources and states, we turn to Elster, a social scientist interested in the nature of explanation in the social sciences. He notes that affective states are to be distinguished from visceral reactions (e.g. pain, hunger) in that they have cognitive antecedent, are ‘states’ they represent something, are generally associated with physiological arousal and expression and are associated with action tendencies, even if those actions are not carried out. We use his classification (pp. 145–161) to scope future considerations of such resources that are risked.

Hall (1959) introduced ten dimensions of culture along with additional considerations such as the difference between formal, informal, and technical systems. In Hall’s text it is represented as a 10 by 10 grid (category plus the adjective form of the category) Each cell of the matrix could be divisible into the same ten dimensions. Thus the community defence interaction could contain factors such as ‘the temporal aspects of community defences’ or ‘economic aspects of community defences’ (Hall, 1959, p. 193). This kind of arrangement could be used to produce a huge array of socio-cultural issues, which could serve as the objects being changed by a service system, or resources to be assessed in effectiveness judgements. So for example community defences could be the domain whilst temporal and economics aspects could be evaluation factors.

4. Summary and conclusions

Earlier the paper made the observation that HCI could offer the emerging field of service science: (1) tools for enacting a user centred ethos; and (2) concepts and frameworks for handling the nature of services. In the context of a special issue on Long’s work the paper explored the influence, scope and utility of the UCL Conception in relation to our own work in relating strands of services research, and in modelling service systems. The conception offers a set of abstractions that can be built upon when moving HCI towards dealing with services, service systems, and service science. Set in its historical context, the conception offered a set of abstractions that for many brought clarity to the field, but when considering it in relation to services (and contemporary HCI concerns), often a case for a more general form of the concepts can be made. However, by generalising the concepts to embrace wider forms of resource costs and domain object, we increase the variability of the factors being examined and the need to make assessments of subjective and intersubjective factors. This increase in variability would take us further away from being able to specify then implement in the manner associated with Long’s vision for an engineering approach. Either we need to consider in more depth what it would mean to ‘engineer’ services, including experiences, or we need to work out the relationships between the different perspectives on development.

Comment 42

 

Long’s position on HCI as an engineering discipline did not preclude the development of heuristics (e.g., Smith et al., 1997). The ABFS as presented still embodies considerable craft understanding; the work presented in Section 3.3 could be further developed to provide more than a checklist of items, but this is a major challenge in itself.

Bødker (2006) was keen to portray HCI as being composed of first, second, and third waves. The trouble with Bødker’s view is that existing work within ‘1st wave theories’ is capable of embracing aesthetics, cognition, emotion, and culture within the same framework (Byrne et al., 2004; Lindgaard and Whitfield, 2004; Teasdale and Barnard, 1993), along with route maps for the generalisation of such a framework (Barnard et al., 2000). Similarly one piece of Long’s work (Long and Dowell, 1989) demonstrated the same concepts (e.g. work system, domain and discipline) can be viewed through different perspectives (craft, applied science and engineering). We have argued that his concepts of structures and behaviours generalise to different task qualities and resource costs, but the criteria associated with an engineering perspective may not hold for the new resource costs we have suggested in the ABFS. Indeed even colloquial use of the terms emotion engineering and social engineering send shudders down the spines of holders of certain value sets. Whilst work by Hassenzahl and colleagues demonstrates that robust statistics can be found about relationships between pragmatic and hedonic factors, these assessments are still subjective in nature, and less amenable to the engineering perspective prescribed by many for HCI (see also Hassenzahl et al., 2001; Sengers, 2003).

In the introduction, we mentioned that HCI had two elements to offer: (1) the user centred mindset and techniques; and (2) concepts and frameworks for understanding the nature of services. The major focus of this paper has been to explore one of these frameworks – the UCL Conception – as an important antecedent to our own work in the field. We now consider – in brief – this first issue. Benbasat and Zmud (2003) made the argument that the related field of Information Systems had started to dwell on peripheral issues that move the field away from its core focus on the development of IT artefacts. If Long and Dowell’s (1989) definition of HCI as ”the design of humans and computers interacting (p. 9),” and other such technology centred definitions of HCI remain in place, then services can be considered one of these peripheral issues, therefore interaction between HCI and service science moves HCI away from its core concern. HCI’s textbook consensus is still concerned with the user centred design of computer-based technology. Services are not technology, although they are frequently enabled, enacted by, and performed through IT artefacts across varying channels (van Dijk et al., 2007). If we hold that HCI is about the human-centric design of activities and experiences – whether involving IT artefacts or not – then the interaction between the disciplines can continue. Definitions of HCI tend to stress both aspects, but perhaps HCI’s biggest gift to other fields is the user centred focus that is generic enough to apply to activities and experiences as well as artefacts.

HCI could make both substantive and methodological contributions to the analysis and design of services. Long standing methodological principles such as: early and continual focus on people and their activities; multidisciplinary and integrated design teams; and iterative design processes (Gould and Lewis, 1985) are applicable to the design of services as well computational and non computational artefacts. HCI can offer approaches such as user participation, prototyping, conceptual design, and a range of approaches that provide sophisticated analysis of tasks/activities and their informational, cognitive, emotional, and social consequences. Characterisations of services overlap with participatory, experience and activity oriented approaches for analysing and designing computing applications and activities (e.g. Hassenzahl et al., 2000; McCarthy and Wright, 2004; Schrepp et al., 2006). Service design methodologies developed by designers borrow from, overlap with, or complement HCI. Parker and Heapy (2006) for example, recommend the use of prototypes, personas, and the measurement of service experience.

This of course begs the question whether HCI’s user centred focus is unique 7. If HCI does move into the Service Science arena alongside its existing interaction with and embracement of, disciplines such as Design Engineering and Sociology, HCI will come across Services Marketing and Services Operations, both of which could make a claim on user centeredness. In the case of Services Marketing, Fisk (2008) a renowned Services Marketer (who has worked with various disciplines e.g. Fisk et al., 2008), considers computer science in a pejorative and stereotyped fashion in its characterisation of people as users, and seemingly unaware of HCI, or its antecedents and relatives such as human factors, design, software engineering, and participatory design. In a similar vein Ng et al. (2009, p. 379) claims that Services Marketing is unique in the service science arena as being the only discipline that considers the customer as being within the service system. Seemingly unaware of HCI conceptions such as Long’s. So much for theory, what about practice? If Seigel and Dray’s observations is representative of Marketing, ”more time would be spent interviewing users about what they liked and disliked about the site than in observing and analyzing their task behaviour (Seigel and Dray, 2001, p. 20);” then there is room for co-existence, each approach focuses on different aspects, and HCI’s focus is on actual behaviours, not just opinions and perceptions. Other HCI practitioner papers support this view (e.g. Rohrer et al., 2008), and Cockton’s exploration of worth (Cockton, 2007) has drawn upon marketing techniques, as has van Dijk’s work on service channels (2008, 2007).

With regards to Service Operations, Wright and Mechling’s study (2002) study suggested that Service Operations practitioners saw three key issues to be: (1) determining how to utilise resources most effectively; (2) monitoring and measuring quality of services and (3) predicting future events, conditions, customer demand, which suggest many crossovers with HCI’s concerns and ethos. However, Seddon et al. (2009) characterise Service operations approach to services as dominated by a Lean Services agenda. This relies on a manufacturing analogy for services, centred on standardisation, control, and viewing all services operations as a form of demand management. They go onto note that a key part of the Lean Services approach is ”treating failure demand as though it is just more work to be done is to fail to see a powerful economic lever (p. 8)”. Service Operations’ user centred focus will generally apply too late in the service design and execution lifecycle to remedy the cause(s) of failure demand. HCI should be able to provide tractable design processes that prevent the bulk of failure demands, hence HCI could help bridge between Service Marketing and Service Operations. We take heart from Carroll’s assertion ”the continuing synthesis of disparate conceptions and approaches to science and practice in HCI has produced a dramatic example of how different epistemologies and paradigms can be reconciled and integrated (Carroll, 2009)”.

Acknowledgements

The writing of this paper has been supported by the EPSRC/BAE systems funded S4T (Solution Service Support: Strategy and Transformation) programme. The three reviewers provided great stimulus to clarify the concepts within this paper, I hope I have not put them off considering services themselves. Attendees at the CIRP IPS2 conference and the 1st and 2nd HCI and Services workshops at HCI 2008 and 2009 have provided much useful discussion and insight, alongside colleagues within the Institute for Manufacturing and Engineering Design Centre at Cambridge. The usual caveats about misunderstandings apply to this work.

References

Alonso-Rasgado, T., Thompson, G., Elfström, B.-O., 2004. The design of functional (total care) products. Journal of Engineering Design 15 (6), 515–540.

Aurich, J.C., Schweitzer, E., Fuchs, C., 2007. Life-Cycle Oriented Planning of Industrial Product-Service Systems. ICMR 2007, Leicester, De Montfort/Inderscience. pp. 270–274.

Barnard, P.J., May, J., Duke, D., Duce, D.A., 2000. Systems, interactions and macrotheory. ACM Transactions on Computer–Human Interaction 7, 222–262.

Beck, D., Cowan, C., 1996. Spiral Dynamics: Mastering Values, Leadership, and
Change. Blackwell Business, London.

Bekker, M., Long, J., 2000. User Involvement in the Design of Human-Computer
Interactions: Some Similarities and Differences between Design Approaches
HCI’2000 Sunderland, 4–8 September. Springer, pp. 135–148.

Benbasat, I., Zmud, R.W., 2003. The identity crisis within the is discipline: defining and communicating the discipline’s core properties. MIS Quarterly 27 (2), 183–
194.

Bennett, K., Layzell, P., Budgen, D., Brereton, P., Macaulay, L., Munro, M., 2000.
Service-based software: the future for flexible software. APSEC 2000, pp. 214–
221.

Bitner, M.J., Ostrom, A.L., Morgan, F.N., 2007. Service blueprinting: a practical tool
for service innovation. In: Innovation in Services Conference, Berkeley, April 26–
28.

Bødker, S., 2006. When second wave HCI meets third wave challenges.
NORDICHI’06, ACM, Oslo, pp. 1–8.

Byrne, R.W., Barnard, P.J., Davidson, I., Janik, V.M., McGrew, W.C., Miklósi, Á.,
Wiessner, P., 2004. Understanding culture across species. Trends in Cognitive
Sciences 8 (8), 341–346.

Carroll, J.M., 2009. Human Computer Interaction (HCI). <http://www.interactiondesign.org/encyclopedia/human_computer_interaction_hci.html>.

Checkland, P.B., Poulter, J., 2006. Learning for Action. Chichester Wiley.

Chen, J.V., Yen, D.C., Chen, K., 2009. The acceptance and diffusion of the innovative
smart phone use: A case study of a delivery service company in logistics.
Information & Management 46 (4), 241–248.

Cockton, G., 2006. Designing worth is worth designing. NORDICHI 2006. ACM, Oslo,
pp. 165–174.

Cockton, G., 2007. Creating value by design? Engineering Design Centre Seminar
Series, Cambridge.

Cockton, G. Virtues and Potentials Can Guide Design Choices of Qualities and
Values. University of Northumbria, Newcastle, unpublished manuscript.

Costanza, R., d’Arge, R., de Groot, R., Farber, S., Grasso, M., Hannon, B., Limburg, K., Naeem, S., O’Neill, R.V., Paruelo, J., Raskin, R.G., Sutton, P., van den Belt, M., 1997. The value of the world’s ecosystem services and natural capital. Nature 387
(6630), 253–260.

Cottam, H., Leadbeater, C., 2004. Health: Co-creating Services. Report, RED: Design
Council, London. Cowan, C.C., Todorovic, N., 2000. Spiral dynamics: the layers of human values in
strategy. Strategy and Leadership 28 (1), 4–12.

Cummaford, S., 2000. Validating effective design knowledge for re-use: HCI
engineering design principles, vol. 2. CHI ’00. ACM Press, The Hague, pp. 71–72.

Cyr, D., Hassanein, K., Head, M., Ivanov, A., 2007. The role of social presence in establishing loyalty in e-Service environments. Interacting with Computers 19
(1), 43–56.

Diaper, D., 2004. Understanding task analysis for human–computer interaction. In: Diaper, D., Stanton, N.A. (Eds.), The Handbook of Task Analysis for HCI. LEA, Mahwah, pp. 5–47.

Dowell, J., Long, J.B., 1989. Towards a conception for an engineering discipline of human factors. Ergonomics 32 (11), 1513–1535.

Dowell, J., Long, J.B., 1998. Conception of the cognitive engineering design problem. Ergonomics 41 (2), 126–139.

Dreyfuss, H., 1955/2004. Designing for People. Allworth Press.

Elster, J., 2007. Explaining Social Behavior. Cambridge University Press, Cambridge.

Fisk, R., 2008. Service Science, the Elephant and the Blind Men, Who’s Who?
University of Strathclyde, Glasgow.

Fisk, R.P., Brown, S.W., Bitner, M.J., 1993. Tracking the evolution of the services
marketing literature. Journal of Retailing 69 (1), 61–103.

Fisk, R.P., Grove, S.J., John, J., 2008. Interactive Services Marketing, 3rd ed. Houghton
Mifflin, Boston.

Fließ, S., Kleinaltenkamp, M., 2004. Blueprinting the service company: managing
service processes efficiently. Journal of Business Research 57 (4), 392–404.

Flint, D.J., 2006. Innovation, symbolic interaction and customer valuing: thoughts stemming from a service-dominant logic of marketing. Marketing Theory 6 (3),
349–362.

Fowler, M., 1997. Analysis Patterns, Reusable Object Models. Addison-Wesley,
Menlo Park.

Gemmel, D., 2004. The Swords of Night and Day. Corgi Books, London.

Goedkoop, M., van Halen, C., te Riele, H., Rommens, P., 1999. Product Service
Systems, Ecological and Economic Basics. Report, PRé Consultants, Amersfoort.

Goodwin, G.P., Darley, J.M., 2008. The psychology of meta-ethics: exploring
objectivism. Cognition 106 (3), 1339–1366.

Gould, J.D., Lewis, C., 1985. Designing for usability: key principles and what
designers think. Communications of the ACM 28 (3), 300–311.

Green, T.R.G., 1998. The conception of a conception. Ergonomics 41 (2), 143–146.

Hall, E.T., 1959. The Silent Language. Doubleday, New York.

Harper, R., Rodden, T., Rogers, Y., Sellen, A. (Eds.), 2008. Being Human: Human–
Computer Interaction in the Year 2020. Microsoft Research Ltd., Cambridge.

Hassenzahl, M., Platz, A., Burmester, M., Lehner, K., 2000. Hedonic and Ergonomic Quality Aspects Determine a Software’s Appeal Emotions and Values. CHI 2000,
The Hague, 1–6th April, pp. 201–208.

Hassenzahl, M., Beu, A., Burmester, M., 2001. Engineering joy. IEEE Software 18 (1),
70–76.

Hawken, P., Lovins, A., Lovins, L.H., 1999. Natural Capitalism: Creating the Next
Industrial Revolution. Rocky Mountain Institute, Snowmass, CO.

Heskett, J., 2009. Creating economic value by design. International Journal of Design
3 (1).

Heylighen, F., 1997. Objective, subjective and intersubjective selectors of
knowledge. Evolution and Cognition 3 (1), 63–67.

Hill, P., 1977. On goods and services. Review of Income & Wealth 23 (4), 315–338.

Hill, P., 1999. Tangibles, intangibles and services. Canadian Journal of Economics 32
(2), 426. IfM and IBM, 2008. Succeeding through Service Innovation. IfM, University of
Cambridge, Cambridge.

Jégou, F., Manzini, E. (Eds.), 2008. Collaborative Services: Social Innovation and
Design for Sustainability. POLI design, Milano.

Johnson, P., Johnson, H., Hamilton, F., 2000. Getting the knowledge into HCI. In:
Schraagen, J.M., Chipman, S.F., Shalin, V.L. (Eds.), Cognitive Task Analysis. LEA,
Mahwah, pp. 201–214.

Jones, M., Samalionis, F., 2008. From small ideas to radical service innovation.
Design Management Review, Winter, pp. 19–27.

Karat, J., Karat, C.-M., 2004. Experiences people value. In: Diaper, D., Stanton, N.A.
(Eds.), The Handbook of Task Analysis for HCI. Lawrence Erlbaum Associates,
Mahwah, NJ, pp. 585–602.

Kounkou, A., Cullinane, A., Maiden, N., 2008. Using HCI knowledge in service-centric
applications. In: Wild, P.J. (Ed.), HCI 2008 Workshop on HCI and Services.

Lages, L., Fernandes, J., 2005. The SERPVAL scale: a multi-item instrument for measuring service personal values. Journal of Business Research 58 (11), 1562–
1572.

Light, A., Wild, P.J., Dearden, A., Muller, M.J., 2005. Quality Value Choice: Exploring
Deeper Outcomes for HCI Products. CHI2005 workshop, Porland, Monday 26
April. ACM Press.

Lim, K.Y., Long, J.B., 1994. The MUSE Method for Usability Engineering. Cambridge
University Press, Cambridge.

Lindgaard, G., Whitfield, A., 2004. Integrating aesthetics within an evolutionary and
psychological framework. Theoretical Issues in Ergonomics Science 5 (1), 73–
90.

Long, J.B., Dowell, J., 1989. Conceptions of the Discipline of HCI. In: Sutcliffe, A.,
Macaulay, L. (Eds.), HCI’89, Nottingham, 5–8th September, CUP, pp. 9–32.

Lovelock, C.H., 1983. Classifying services to gain strategic marketing insights.
Journal of Marketing 7 (summer), 9–20.

Lovelock, C.H., Gummesson, E., 2004. Whither services marketing? Journal of
Service Research 7 (1), 20–41.

Luthria, H., Rabhi, F., 2009. Service oriented computing in practice. Journal of
Theoretical and Applied Electronic Commerce Research 4 (1), 39–56.

Magoulas, G.D., Chen, S., 2006. Human factors in personalised systems and services.
Interacting with Computers 18 (3), 327–330.

McCarthy, J., Wright, P., 2004. Technology as Experience. MIT Press, Bradford, MA.

Melão, N., Pidd, M., 2000. A conceptual framework for understanding business
processes and business process modelling. Information Systems Journal 10 (2), 105–129.

Mingers, J., 2006. Realising Systems Thinking. Springer, New York. Morelli, N., 2006. Developing new product service systems (PSS): methodologies
and operational tools. Journal of Cleaner Production 14 (17), 1495–1501.

Namoune, A., Nestler, T., Angeli, A.D., 2009. End user development of service-based applications. In: Wild, P.J. (Ed.), 2nd Workshop on HCI and Services at HCI 2009
Cambridge, 1st September.

Nelson, H.G., 2002. Systems science in service to humanity. Systems Research and
Behavioral Science 19 (5), 407–416.

Ng, I.C.L., Maull, R., Yip, N., 2009. Outcome-based contracts as a driver for systems
thinking and service-dominant logic in service science: evidence from the
defence industry. European Management Journal 27 (6), 377–387.

Papazoglou, M., van den Heuvel, W.-J., 2007. Service oriented architectures. The
VLDB Journal 16 (3), 389–415.

Parker, S., Heapy, J., 2006. The Journey to the Interface. Report DEMOS, London. Pring, B., Lo, T., 2009. Dataquest Insight: SaaS Adoption Trends in the US and UK
Report, Gartner.

Ramirez, R., 1999. Value co-production: intellectual origins and implications for
practice and research. Strategic Management Journal 20 (1), 49.

Rathmell, J.M., 1974. Marketing in the Services Sector. Winthrop Publishers,
Cambridge MA.

Reason, B., Downs, C., Lovlie, L., 2009. Service Thinking. <http://www.livework.co.
uk/articles/service-thinking>.

Rohrer, C., Au, I., Darnell, E., Dickenson, N., Evenson, S., Kaasgaard, K., 2008. Design,
Marketing, Strategy: Where does User Research Belong? CHI ’08, vol. 2. ACM,
Florence. pp. 2241–2244.

Roy, R., Shehab, E. (Eds.), 2009. Industrial Product Service Systems, IPS2 2009.
Cranfield University Press.

Schrepp, M., Held, T., Laugwitz, B., 2006. The influence of hedonic quality on the
attractiveness of user interfaces of business management software. Interacting
with Computers 18 (5), 1055–1069.

Seddon, J., 2008. Systems Thinking in the Public Sector. Triarchy Press, Axminster. Seddon, J., O’Donovan, B., Zokaei, K., 2009. Rethinking Lean Service. Vanguard
Consultancy, Buckingham.

Seigel, D.A., Dray, S.M., 2001. New kid on the block: marketing organizations and
interaction design. Interactions 8 (2), 19–24.

Sengers, P., 2003. The engineering of experience. In: Blythe, M.A., Overbeeke, K.,
Monk, A.F., Wright, P.C. (Eds.), Funology: From Usability to Enjoyment. Kluwer
Academic Publishers, Dordrecht.

Seth, N., Deshmukh, S.G., Vrat, P., 2005. Service quality models: a review.
International Journal of Quality & Reliability Management 22 (9), 913–949.

Shostack, G.L., 1977. Breaking free from product marketing. Journal of Marketing 41
(2), 73–80.

Shostack, L.G., 1984. Designing services that deliver. Harvard Review of Business 62
(1), 133–139.

Smith, W., Hill, B., Long, J.B., Whitefield, A., 1997. A design-oriented framework for
modelling the planning and control of multiple task work in secretarial office
administration. Behaviour and Information Technology 16 (3), 161–183.

Stahel, W.R., 1986. The functional economy: cultural & organizational change.
Science & Public Policy 13 (4), 121–130.

Stauss, B., 2005. A pyrrhic victory: the implications of an unlimited broadening of
the concept of services. Managing Service Quality 15 (3), 219–229.

Sutcliffe, A., 2002. The Domain Theory: Patterns for Knowledge and Software Reuse.
Erlbaum, Mahwah.

Teasdale, J.D., Barnard, P.J., 1993. Affect Cognition and Change: Re-modelling
Depressive Thought. Erlbaum Associates, Hove.

Terry, A., Jenkins, D., Khow, T., Summersgill, K., Bishop, P., Andrews, M., 2007.
Transforming Logistics Support for Fast Jets. Report, National Audit Office,
London.

Tiger, L., 2000. The Pursuit of Pleasure, 2nd ed. Transaction Press, New Brunswick. Tukker, A., 2004. Eight types of product-service system. Business Strategy and the
Environment 13 (4), 246–260.

van Dijk, G., 2008. HCI informing service design, and visa versa. In: Wild, P.J. (Ed.),
Workshop on HCI and the Analysis, Design, and Evaluation of Services, HCI
2008, Liverpool BCS.

van Dijk, G., Minocha, S., Laing, A., 2007. Consumers, channels and communication:
online and offline communication in service consumption. Interacting with
Computers 19 (1), 7–19.

Vargo, S.L., Lusch, R.F., 2004a. Evolving to a new dominant logic for marketing.
Journal of Marketing 68 (1), 1–17.

Vargo, S.L., Lusch, R.F., 2004b. The four service marketing myths. Journal of Service
Research 6 (4), 324–335.

Vargo, S., Lusch, R., 2008. Why ”service”? Journal of the Academy of Marketing
Science 36 (1), 25–38.

Vargo, S.L., Maglio, P.P., Akaka, M.A., 2008. On value and value co-creation: a service
systems and service logic perspective. European Management Journal 26 (3),
145–152.

Vicente, K.J., 1990. Coherenceand correspondence-driven work domains.
Behaviour and Information Technology 9 (6), 493–502.

Wakkary, R., 2005. Exploring the Everyday Day Designer. Understanding
Designers’05, Aix-en-Provence, 17–18 October 2005. Centre for Design
Computing and Cognition. pp. 277–282.

Whitefield, A., Esgate, A., Denley, I., Byerley, P., 1993. On distinguishing work tasks
and enabling tasks. Interacting with Computers 5 (3), 333–347.

Wild, P.J., Macredie, R.D., 2000. On change and tasks. In: McDonald, S., Waern, Y., Cockton, G. (Eds.), HCI’2000, University of Sunderland, 4–8 September. Springer,
pp. 45–59.

Wild, P.J., Johnson, P., Johnson, H., 2003. Understanding task grouping strategies. In: Palanque, P., Johnson, P., O’Neill, E. (Eds.), HCI 2003, Bath, 8th–12th September. Springer, pp. 3–20.

Wild, P.J., Johnson, P., Johnson, H., 2004. Towards a composite model for multitasking. In: Palanque, P., Salvik, P., Winckler, M. (Eds.), TAMODIA’04, Prague, November 15–16. ACM Press.

Wild, P.J., Jupp, J., Kerley, W., Eckert, C.M., Clarkson, P.J., 2007. Towards a Framework for Profiling of Products and Services. 5th ICMR, Inderscience, Leicester, pp. 285–290.

Wild, P.J., Clarkson, P.J., McFarlane, D., 2009a. A framework for cross disciplinary efforts in services research. In: Roy, R., Shehab, E. (Eds.), Industrial Product
Service Systems, Cranfield 1–2nd April 2009. University Press, Cranfield, pp.
145–152.

Wild, P.J., Pezzotta, G., Cavalieri, S., McFarlane, D.C., 2009b. Towards a Classification
of Service Design Foci, Activities, Phases, Perspectives and Participants.
MITIP’09 Bergamo.

Wright, C.M., Mechling, G., 2002. The importance of operations management
problems in service organizations. Omega 30 (2), 77–87.

Wyckham, R.G., Fitzroy, P.T., Mandry, G.D., 1975. Marketing of services. European
Journal of Marketing 9 (1), 59.

Zeithaml, V.A., Parasuraman, A., Berry, L.L., 1985. Problems and strategies in services
marketing. Journal of Marketing 49 (2), 33–46.

Diagnosing Co-ordination Problems in the Emergency Management Response to Disasters

Becky Hill

UCL Interaction Centre, MPEB 8th Floor, University College London, Gower Street, London WC1E 6BT, United Kingdom

John Long's Comment 1 on this paper

Abstract

In the United Kingdom, there is a system for the co-ordination of the emergency services in response to disasters – The Emergency Management Combined Response System (EMCRS). This is a general management framework with a complex three tier command and control system, set-up by the UK government in response to a need for better co-ordination between agencies, when they respond to disasters.

This research has developed models of the implementation of the EMCRS for specified disaster scenarios, that support diagnosis of co-ordination problems between agencies. Data for the modelling were acquired by means of training exercises. The co-ordination problems were identified through behaviour conflicts between the agencies. For example: the Fire Service behaviours of setting up a cordon around the disaster site conflict with the Ambulance Service behaviours of accessing the site for treatment of casualties. Model development was achieved through application of an existing framework.

The EMCRS models constitute substantive Human Computer Interaction design knowledge, that is, knowledge that is both explicit and supports design. One view of HCI (Long, 1996) is that of an engineering design discipline, whose research validates design knowledge, both substantive and methodological. Design knowledge supports design practice directly, as the diagnosis of design problems and indirectly, as the prescription of design solutions. An initial method for co-ordination design problem diagnosis by means of EMCRS models has been developed. This paper will describe the development of the EMCRS models and will apply the method and show the diagnosis from this application, of one co-ordination design problem.

Comment 2

1. Introduction

The aim of this paper is to show the development of models of the Emergency Management Combined Response System that can be used to diagnose inter-agency co-ordination problems. In the United Kingdom, there exists a system for the co-ordination of the emergency services in response to disasters, such as explosions, air crashes etc. – the Emergency Management Combined Response System (EMCRS) (Anon., 1994, 2001). This system manages, that is, plans and controls, agencies, such as Fire and Police, when they respond to disasters. The EMCRS was set up to support better co-ordination between agencies responding to disaster, in reaction to a succession of enquiries into disasters e.g. Hidden (1989) and Fennell (1988), which identified problems with co-ordination, both within and between the emergency services in their disaster response. Co-ordination in this context is defined as the ‘harmonious integration of the expertise of all the agencies involved with the object of effectively and efficiently bringing the incident to a successful conclusion’ (Emergency planning college document, 1995).

Comment 3

 

However, even after the introduction of the EMCRS there are still occasions when the emergency response to disasters has been identified as being un-co-ordinated. For example, on July 7th 2005, terrorist attacks in London left 52 dead and many more injured. The report into the terrorist attacks (Anon., 2006), found that there were issues with the emergency services response, mainly due to communication problems, which led to an un-coordinated response. For example, although it states in the London Emergency Procedure Manual that a major incident can be declared by any of the emergency services, the implication being that this will be done on behalf of all the services, on 7th July all three primary emergency services declared a major incident, independent of each other. It was not clear to the reviewers as to why this happened, and also why a declaration of a major incident by one emergency service had not automatically mobilised units from all three (Police, Fire and Ambulance) Services. As a result the review states: ”We recommend that the London Resilience Forum review the protocols for declaring a major incident to ensure that, as soon as one of the emergency services declares a major incident, the others also put major incident procedures in place. This could increase the speed with which the emergency services establish what has happened and begin to enact a co-ordinated and effective emergency response”.

There have been many methods, models and frameworks developed for analysis of Emergency Management. Specifically Rogalski and Samurcay (1993) have focused on communication between the services as a means of analysing distributed decision making. The analysis allows an understanding of why one group is better than the other. The group that has a better flow of communication and distribution of roles is more efficient. Samurcay and Rogalski (1991) have also developed a Method for Tactical Reasoning (MTR) and applied it in emergency management. This method describes a decomposition of the overall task (for the class of emergency situations) into specific tasks (involved in analysis and planning), as prescribed tasks in the sense developed by Leplat (1988). The MTR provides a model of the cognitive tasks, involved in emergency management. This research allows for an understanding of emergency management behaviours, but does not relate it directly to the design of the emergency management system. Kaempf et al. (1996) studied the decision making of experienced personnel in complex command and control environments, using the recognition-primed decision (RPD) model (Klein and Woods, 1993), which depicts how experienced people make decisions in natural settings. The results of the study suggested that decision makers use recognition processes and that situation awareness is of primary concern. However, it is difficult to generalise from this study to other command and control domains, as the domain studied was very procedural in nature, and thus, other command and control settings may place different requirements on the decision makers. Other work, such as Blandford and Wong (2004) and Blandford et al. (2002) has looked at the behaviours of individual services within emergency management, but does not relate these behaviours to the other services within emergency management, and does not develop systematic models that can be directly used for design of the emergency management system.

Frameworks and models are prevalent in the HCI literature (Long, 1987; Whitefield, 1990). However, most models of interaction are task based (Wright et al., 2000). Traditional task analysis methods such as GOMS (Card et al., 1983), Hierarchical Task Analysis (Shepherd, 1989), Task Knowledge Structures (Johnson, 1992), based on observable actions may not be appropriate for analysing complex work domains (Moray et al., 1992). These methods of task analysis do not account for the variability of behaviour that is observed in complex systems (Vicente, 1990). One approach to HCI that places an emphasis on the importance of the constraints in the environment (i.e. the task or the work domain) that are relevant to the operator is an ecological one (Vicente, 1990). The framework used in the current research (the HCI-PCMT framework) has an ecological perspective, in as much as it includes a domain external to the system of concern.

Comment 4

 

 

This approach is similar to the work of others, for example, the Means-Ends Abstraction Hierarchy of Rasmussen and Vicente (1989). However, although the Means-Ends Abstraction Hierarchy can function as a mechanism to cope with the complexity of the natural environment, unlike the HCI-PCMT framework, no distinction is made between the work domain and the interactive worksystem, which allows for an expression of the performance of the system. The Means-Ends Abstraction Hierarchy is a also a more general framework for analysing complex HCI systems, its purpose is not specifically for modelling planning and control in multiple task work situations, and therefore it does not have a planning and control architecture. Consistent with a cognitive engineering perspective (Norman, 1986), the HCI-PCMT framework aims to model the cognitive behaviours of a ‘joint cognitive system’ (Woods and Hollnagel, 1987), considered in relation to their task ‘world’. The HCI-PCMT framework was developed specifically for analysing planning and control for multiple task work systems, and has been shown in previous case-studies to diagnose planning and control design problems (Smith et al., 1997). Thus the HCI-PCMT framework has an, albeit minimalist architecture, to accommodate such planning and control systems. The HCI-PCMT framework was also developed to analyse ‘to-be computerised systems’, and the EMCRS analysed was not computerised.

1.1. Development of design-oriented frameworks and models for HCI

The research presented in this paper is intended to constitute HCI substantive1 design knowledge, in the form of models that support diagnosis of specific design problems, and reasoning about potential solutions to these problems. Dowell and Long (1989) propose the discipline of HCI as the application of HCI knowledge, to support design practices, intended to solve HCI design problems. They identify validated engineering principles as a type of knowledge that best supports HCI practice. These principles would therefore support the design of general solutions to general classes of HCI design problems. The development of such principles represents a long-term goal for an engineering design discipline of HCI.

Comment 5

 

 

Design-oriented frameworks are one form of HCI knowledge, which is both explicit and is intended to support design directly. Such frameworks provide the basis for modelling specific design problems. Their purpose is to enable designers to reason more effectively about potential design solutions. Frameworks lack the ‘guarantee’ of validated engineering principles. Instead, they support the practices of ‘specify-and-implement’. That is, practices where design proceeds through iterations of successive cycles of specification and implementation. Such frameworks support the designer in producing better specifications at an earlier stage of design, thus reducing costly iteration. These frameworks produce models of the systems under investigation, that support diagnosis of design problems, and reasoning about design solutions. For HCI knowledge to be design-oriented, it must be formulated in relation to an adequate expression of HCI design problems. In turn, an adequate expression of HCI design problems must be constructed in relation to a complete and coherent conception of the ontology of HCI; that is, a conception of those entities constituting the scope of application of the HCI discipline (Long, 1996). Dowell and Long (1989) have developed a conception of HCI as engineering (HCIE) in which they attempt to outline a general, complete and coherent ontology for HCI comprising: (i) an interactive worksystem – the tobe-designed system comprising users and computers, (ii) a domain of application – the work to be carried out, and (iii) performance – the effectiveness with which work is carried out. The framework used for the modelling described in this paper is thus based on the Dowell and Long (1989) conception.

As stated above, the research described in this paper developed such models for the EMCRS – a system that manages the response of the emergency services to disasters, that support diagnosis of EMCRS co-ordination design problems and reasoning about design solutions to these problems.

To show the development of such models this paper will first describe the EMCRS modelled. Then the background to the use of Design-oriented frameworks for such modelling will be presented. The actual framework used for the modelling, based on the Dowell and Long (1989, 1998) conception will then be described in detail in Section 3. In Section 4 the EMCRS data gathered for modelling are outlined. Section 5 describes model development through application of the framework presented in Section 3. Section 6 presents the method for co-ordination design problem diagnosis and applies the method to identify one behaviour conflict and diagnose the co-ordination design problem of cordon restrictions. The last section summarises the paper and identifies future work.

2. Domain of study

Emergency management is an example of a multi-user planning environment, which requires operators to deal with emergency situations. Controlling these situations requires the co-ordination of numerous agents, who share the various specific tasks, which fulfil the overall goal of making the situation stable. These tasks involve a number of people, often geographically distributed, working simultaneously (rather than sequentially), as a team towards the achievement of shared goals. The development of systems for emergency management, therefore, demands the analysis and modelling of co-operative work tasks, placing strong emphasis on the capture and representation of concurrent task activities, involving multiple agents.

The EMCRS has a command and control organisation with a three tier structure. The EMCRS is a general management framework, which has been agreed nationally which:

• Defines relationships between differing levels of management.
• Allows each agency to tailor its own response to plans to interface with the plans of others.
• Ensures all parties involved understand their relative roles in the combined response.
• Retains sufficient flexibility of option to suit local circumstances to enable the emergency services to interact effectively (Anon., 1994, 2001

The primary objectives of a disaster response, as declared by the Home Office are:

• To save life.
• To prevent escalation of the disaster.
• To relieve suffering.
• To protect property.
• To safeguard the environment.
• To facilitate criminal investigation and judicial, public, technical or other inquiries.
• To restore normality as soon as possible.

All the different agencies should use this structure to organise their own planning procedures, so that they interface effectively with each other. The three levels are operational, tactical and strategic (sometimes referred to as bronze, silver, gold). At each level, each of the agencies has its own commander for co-ordinating the response. At the strategic level, these commanders make up a senior co-ordinating group. The operational response is carried out by each agency, concentrating on their specific tasks within their areas of responsibility, e.g. the Fire Service fighting fires. The tactical response determines the priority in allocating resources. It also plans and co-ordinates the overall response, obtaining other resources as required, for exam-ple additional fire engines. The strategic co-ordinating group has to formulate the overall policy within which the response to a major incident will be made. At the strategic level, there is one person from each emergency service. Under the EMCRS, the management of the response to major emergencies will normally be undertaken at one or more of the three levels. The degree of management required will depend on the nature and scale of the emergency.

There needs to be co-ordination at all levels of the EMCRS, so that the disaster situation is brought under control, as quickly and efficiently as possible. There needs to be co-ordination at each level within the hierarchy and between the levels. One of the main mechanisms, by which the performance of any planning system is affected, is that of coordination. Hutchins (1990) has identified important features to be accounted for in a distributed task, which ensure effectiveness:

• Shared task knowledge – each person understands enough about each others’ work to co-ordinate effectively.
• Horizon of observation – which allows other team members to witness other performances.
• Multiple perspectives, which allow for activities to be observed from different points of view.

The behaviour of each member of the team is contingent on the behaviour of all the other members of the team. An action by one member will trigger an action (reaction) by another member, until the task is complete. Each member of the group has knowledge of a specific part of the distributed task, the whole group is undertaking. The co-ordination among the actions of the members of the team is not achieved by following a master procedure, instead it emerges from the interactions among the members of the team. The procedure is used as a guide to organising actions. Distribution of tasks leads to a need for co-ordination. Thus, in the case of EMCRS, co-ordination is required, because each emergency services agent only possesses a local view and incomplete information and, therefore, must co-ordinate with other agents to achieve globally coherent and efficient solutions. In emergency management, there is not only co-ordination between each agent within one group, but also co-ordination between each agency (which is made up of many single agents) on a horizontal level and vertical co-ordination between the different command levels. As stated earlier, co-ordination, or rather lack of co-ordination, has been identified as a major factor in the ineffective response of the emergency services to disasters.

The aim of this research was to attempt to diagnose these coordination problems with respect to the planning and control of the EMCRS. The emergency services response to disasters has different phases. First, there is the initial response, when the situation is usually fairly chaotic. Second, the response some time later (could be a few hours, maybe longer depending on the scale of the incident), when the situation is more stable. Last, the restoration of normality phase, when the actual incident has been brought under control, but the situation has not returned to normal. Within each of these phases, the emergency services will have different roles/tasks, that they need to carry out. During the initial response phase, the tasks being carried out by the emergency services will be their primary tasks in response to the situation, e.g. Fire Service fighting fires, Ambulance Service treating casualties. Collaboration, coordination and communication are thus, vital at the initial response stage (Anon., 2001). Co-ordination problems, occurring between services in the initial response phase, will, therefore, have more of a detrimental effect on EMCRS performance, than co-ordination problems occurring at other phases, when the tasks being carried out are not dealing with the initial effects of the situation. Data collected for use in the modelling of the EMCRS will thus need to include the initial phase as a priority. The EMCRS is thus a complex system interacting with a complex dynamic situation. There are multiple agencies, with multiple personnel, at multiple levels of command, carrying out concurrent task activities.

3. HCI planning and control for multiple task work framework

The aim of the present research was to develop models of the EMCRS that support the diagnosis of EMCRS co-ordination design problems and the reasoning about solutions to these problems. To develop such models a design-oriented framework was required, that supports modelling of the EMCRS – a distributed cognitive planning and control system, comprising more than one user, or groups of users, whose activities must be co-ordinated for effective performance. One such framework was developed for a class of HCI design problem, expressed as the planning and control of multiple task (HCI-PCMT) work in office administration a ‘to be computerised system’ (Smith et al., 1992, 1997). The office administration domains previously modelled by the HCI-PCMT framework were single user planning and control systems. Application of the HCI-PCMT framework to model the EMCRS would, thus, extend the scope of the framework to accommodate multiuser planning and control systems. The models produced would identify planning and control co-ordination design problems and thus diagnose ineffective performance.

The HCI-PCMT framework is based on a conception of HCI (Dowell and Long, 1989, 1998). The conception distinguishes the interactive worksystem, which comprises users and computers or, more generally devices/equipment, from its domain of application, constituting the work carried out by the worksystem. The effectiveness with which work is carried out, that is performance, is a function of the quality of the work (how well it is performed), and the resource costs to the worksystem (the effort, etc. of performing the work that well). Overall performance, thus, expresses whether goals have been achieved, and at what cost. A design problem is diagnosed, if actual performance (Pa) does not equal desired performance (Pd), where performance (P) is expressed as task quality (Tq), user costs (Uc) and computer (device) costs (Cc). A design solution is prescribed, if Pa is equal to Pd.

Comment 6

 

In the Dowell and Long conception, a domain of application (or work domain) is described in terms of objects, which may be abstract or physical. Objects are constituted of attributes, which have values. The attribute values of an object may be related to the attribute values of one or more other objects. An object, at any time, is determined by the values of its attributes. The worksystem performs work by changing the value of domain objects (i.e. by transforming their actual attribute values) to their desired values, as specified by the work goal. Attributes may be affordant or dispositional. Affordant attributes are transformed by the worksystem; their transformation constitutes the work performed. Dispositional attributes are relevant to the work (they need to be used by the worksystem); but are not changed by the worksystem.

The worksystem is conceptualised as a behavioural system comprising the interacting user behaviours (supported by user structures) and computer (device) behaviours (supported by device structures). Abstract structures comprise representations and processes. Abstract representation structures refer, for example to the worksystem’s knowledge, databases or information stores. Abstract process structures refer, for example, to the worksystem’s procedures, methods or heuristics. Abstract structures support worksystem abstract behaviours, when abstract process structures, such as procedures, act on abstract representation structures, such as a database. Similarly, worksystem physical structures support worksystem physical behaviours. The HCI-PCMT framework specifies worksystem structures for the planning and control of multiple task work. These structures are expressed at both abstract and physical levels of description. Physical structures embody abstract structures and physical behaviours embody abstract behaviours. At the abstract level, the framework describes the worksystem’s cognitive structures. These comprise four process structures (planning, controlling, perceiving and executing), and two representational structures (plans and knowledge-of-tasks). These structures support the planning and control behaviours of the worksystem and are distributed across the physical users and devices/equipment. The four processes support the behaviours of planning, control, perception and execution respectively. The physical structures support the physical behaviours, but are not differentiated further by the HCI-PCMT framework, as the framework’s concern is primarily with abstract behaviours associated with planning and control.

The rationale for what to some might appear a ‘minimalist’ architecture is threefold. First, the general architecture of representations and processes is commonly assumed by Cognitive Psychology models in the information processing tradition. Secondly, the architecture was adequate to support the construction of the initial HCI-PCMT framework for the domain of secretarial office administration. Thirdly, the architecture supported the construction of models, whose form and granularity were commensurate with solving user interface design problems. The full argument for this set of structures, can be found elsewhere (Smith et al., 1997); but can be summarised as follows:

Influenced by Newell and Simon (1972), much planning research in Cognitive Science and Artificial Intelligence has tended to view plans as complete and fully-elaborated behaviour sequences, which ensure task goal achievement. This view has been undermined by research into planning in HCI. The behaviours of users, who are part of worksystems, it has been argued, cannot be regarded entirely as the output of executable plans (e.g., Suchman, 1987; Larkin, 1989; Payne, 1991) – rather they are often, at least partly, direct responses to the task environment. Within this perspective, plans need not be complete and fully-elaborated, but rather they may be partial (in the sense that they may specify only some of the behaviours to be implemented) and/or general (in the sense that some behaviours may be specified only generally and not at a level that is executable). Such plans might be more generally viewed as ‘resources’ for guiding behaviour (Suchman, 1987). Furthermore, if a plan is regarded as a resource to guide behaviour it is no longer necessary that it be limited to specifying behaviours. Rather it might, instead, specify required states of the task or conditions of the environment. Plans, which serve as resources for guiding behaviour, rather than as specifications of complete and fully-elaborated behaviour sequences, cannot ensure that goals will be achieved. This research also undermines the assumption that perception precedes planning, which precedes execution. Ambros-Ingerson (1986) argued that all planning can precede execution only when:

1. The task environment is static – relevant changes in the task environment do not occur after the plan is complete.
2. The task environment is simple enough to be practically modelled – the consequence of behaviours can be predicted sufficiently well to generate a complete and fully-elaborated behaviour sequence.
3. The task environment is known – the planner’s knowledge of the task environment can be complete before planning commences.

Most task environments studied by HCI researchers do not embody these assumptions (Young and Simon, 1987). In direct contrast, they are usually dynamic, complex and partly unknown by the planner (e.g., Hollnagel et al., 1988). Execution behaviours in worksystem task environments are required to commence before plans are complete and fully-elaborated and therefore the perception, execution and planning behaviours must be temporally interleaved – having no necessarily fixed order in which to be performed.

When performing a task, a system has to exercise control; that is, it has to select the next behaviour to be carried out at each moment (e.g. Hayes-Roth, 1985). For a system, which constructs complete and fully-elaborated plans, controlling is a simple process of selecting behaviours, according to the plan and initiating their execution. However, for worksystems, which employ plans as resources to guide behaviour, some more complex control behaviour is required to select execution behaviours over time – since the selection is constrained by, rather than specified by, the plan. Furthermore, if a worksystem interleaves execution behaviours with planning and perception behaviours, controlled sequencing of these behaviours is also required.

Consistent with the preceding arguments, the PCMT framework describes the worksystems’ cognitive structures for planning and control as follows:

At the first (abstract) level of description, Plans are specifications of required transformations of domain objects and/or of required behaviours. They may be partial (in the sense that they may specify only some of the behaviours or transformations), and they may be general (in the sense that some behaviours or transformations may be specified only generally and not at a level that is directly executable). Planning behaviours, thus, specify the required domain object transformations and/or behaviours to support those transformations.

Perception and execution behaviours are, respectively, those whereby the worksystem acquires information about the domain objects and those whereby it carries out work, changing the value of the object attributes as desired. Information about domain objects from perception behaviours is expressed in the knowledgeof-tasks representation. Control behaviours entail deciding which behaviour to carry out next, both within and between tasks; but involve more than reading off the next behaviour from a complete and fully-elaborated plan.

The second level of description of planning and control structures is physical, wherein the framework describes the distribution of the abstract cognitive structures across the physically separate user and devices of particular worksystems. The framework, thus, allows the construction of alternative models of the distribution of cognitive structures across the user and devices, and so supports reasoning about allocation of function between users and devices, a major decision in design problem solutions. In the office administration domains, studied for the development of the framework, the physical worksystem was the person plus devices, but not a computer. These domains were: secretarial office administration; and medical reception. The more general notion of devices in the framework replaces the notion of a computer in the Dowell and Long (1989) conception. The use of devices (which include computers) allows the HCI framework to model work situations for which computerised support has yet to be developed. This would include the EMCRS, which at the time of analysis was not computer supported. The models of EMCRS produced by application of the framework would enable a designer to reason about design solutions including computerisation.

The outline of the HCI-PCMT framework, including its domain of application, its worksystem and its performance, is now complete. How the framework supports the analysis of multiple task work will now be outlined.

Multiple task work requires a user, as part of the interactive worksystem, to perform distinct, but overlapping tasks. Each task potentially competes for worksystem behaviours. Multiple task work represents an important concern for system designers, as performing overlapping tasks is likely to have an effect on work performance. The term ‘multiple task work’, as characterized by the framework, refers to situations, in which more than one task is carried out concurrently over relatively long and overlapping periods of time. Characterising multiple task work requires a single task to be defined. In the framework, a task is considered to be part of the work carried out in the domain of the worksystem. A task is thus conceptually distinct from the worksystem itself and its behaviours. In one of the systems previously analysed by the framework, medical reception, (see Hill et al., 1995) a task was expressed as the support of a medical case object (i.e. patients consulting with medical practitioners). The medical reception domain is an instance of multiple task work, since support is provided concurrently for multiple ongoing and temporally overlapping medical cases (i.e. for many patients together). In the EMCRS, there are multiple agencies, who need to work together towards the goal of stabilising the disaster situation. Each of the agencies involved within the EMCRS has its own set of tasks that it must carry out in order to achieve the overall goal of stabilising the disaster. These tasks are carried out independently from the other agencies; but the behaviours, associated with each of these tasks, need to be co-ordinated with the other agencies, to maximise the effectiveness of the overall EMCRS response. The work of the EMCRS can be described as the support of a disaster. Unlike, the previous systems analysed with the framework, there are obviously not multiple disasters, and thus a single task cannot be described as support for a single disaster. Rather, a single task would be each of the individual agency tasks in support of a disaster. Thus, the work of the EMCRS is multiple task, since support is provided concurrently for the multiple ongoing and temporally overlapping tasks carried out by the individual agencies in response to a disaster. This difference in the task description has implications for the application of the framework to the EMCRS, and will be one of the areas of extension for the framework, that modelling a multi-user planning system requires. These implications will be described in detail with respect to the framework axioms, (see Section 5). Extending the framework in this way, would enable application of the framework to other complex systems, where there are multiple users or groups of users carrying out independent, but concurrent tasks, that need to be co-ordinated for effective performance.

The EMCRS has more than one level of operation, in fact potentially three, (operational, tactical and strategic) depending on the characterisation of the disaster to which response is made. The HCI-PCMT framework has so far only been applied to systems with one level of operation. The HCI-PCMT framework will need to be extended to accommodate this difference.

The HCI-PCMT framework is expressed as a set of axioms,2 based on a partial and selective application of Dowell and Long’s (1989) conception for HCI. (The HCI-PCMT framework was developed for a ‘to-be computerised system’, and therefore some of the specifics of the conception which refer to computers are not applicable.) The purpose of the HCI-PCMT framework is to express design problems to aid a designer to reason about possible design solutions, in a specify-and-implement type of design practice. The axioms are described later in the paper with respect to EMCRS. A diagrammatic representation of the generic HCI-PCMT framework is shown in Fig. 1. This representation is used to apply the framework to the EMCRS.

Fig. 1. HCI-PCMT framework.

Thus, application of the framework allows for a description of the abstract and physical structures of the interactive worksystem and the abstract and physical objects of the domain of application (work). The framework defines the relationship between the abstract and physical structures of the worksystem, and the relationship between the abstract and physical objects of the domain. Performance is some function of the task quality, associated with the multiple task work carried out, and the resource costs, associated with worksystem structures and behaviours of planning and control. The framework will thus allow for a description of design problems, where actual performance does not equal desired performance.

4. EMCRS training – Exercise Scorpio

In order to develop models of the EMCRS by application of the HCI-PCMT framework, data regarding an instantiation of EMCRS were required. Ideally, such data would be from response to an actual disaster. However, these data are hard if not impossible to access, and also difficult to make direct observations from. There are various specialist training centres in the UK for the emergency services. However, there is only one centre where multidisciplinary training is provided – The Cabinet Office Emergency Planning College at Easingwold (formerly the Home Office Emergency Planning College). The aim of the college is to promote and sustain emergency preparedness within the United Kingdom through the concept of Integrated Emergency Management. Many different training exercises are run at the centre, that cover all aspects of emergency planning. The exercise where the data were gathered was the Emergency Services Seminar on Inter-Agency Response to Disaster. The aim of this seminar was to provide an opportunity for emergency services’ personnel, who may have a role to play in the dissemination of best practice at the operational, tactical and strategic command levels, to study problems which might arise from major civil emergencies with particular reference to the need for a co-ordinated response. The information regarding the exercise – Exercise Scorpio, made it clear that it was the most appropriate of all those run by the college, for gathering EMCRS data. The data were provided by two stagings of this exercise. The trainees were members of the emergency services and local authority emergency planning officers. There were 60 trainees taking part in each exercise. The emergency service personnel were brought together in multi-agency groups, called syndicates, to discuss Exercise Scorpio. These syndicates, were pre-selected, i.e. the members had applied to attend the seminar as a group. Syndicates comprised Police, Fire, Ambulance and Local Authority personnel from the same district, county or region. The exercise required the trainees to describe their response to Exercise Scorpio from initial response to the restoration of normality. There are various well defined phases in response to the disaster scenario, which involve different worksystem configurations and different worksystem behaviours. In the model, only one worksystem configuration and its behaviours are modelled – that of the initial response phase. This phase is considered critical, since co-ordination problems at this phase can have most serious effects on subsequent disaster stability.

4.1. Exercise Scorpio narrative

Newford is a market town in the county of Crownshire. The main town centre is built around the main A338 which bisects the town in a north/south direction. In the centre of the town a railway, carried on an embankment and bridge, runs directly across the town from east to west. From the railway bridge, in a northerly direction, the A338 is inclined and at the bottom of the incline is a canal with moorings for canal boats. This is a very busy holiday waterway and is a popular stopping point.

At approximately 0930 h on a weekday during school term time, a tanker train en-route from a refinery to an airport fuel depot is derailed whilst passing over a railway bridge.

The bridge which is a steel Victorian structure, carries the railway over the A338 which provides the main access into the town centre. It is market day and there are numerous market stalls set up on the side of the roadway on either side of the bridge. The market and town are a popular destination for locals and visitors from out of the town area, including foreign citizens on sightseeing tours through the area. There are a number of housing developments behind the shops in the High Street each side of the railway bridge and a nursery school with 60 toddlers is sited approximately 400 m away. A cottage Hospital with 30 beds is some 500 m to the south east of the bridge and a primary school with 200 pupils some 800 m to the south.

The train consists of a diesel electric locomotive and eight tank cars, each fully loaded with 100 tonnes of AVTUR Aviation Fuel (SIN 1863 with an emergency action code 3 (Y)E). During the derailment one of the tank cars is ruptured and aviation fuel flows down the sides of the embankment onto the roadway and into adjoining properties. Flammable vapours from the fuel have been ignited by an open gas burner from a catering caravan.

At the time the explosion occurred, a tourist bus was passing beneath the railway bridge. The bus is carrying 45 tourists of whom 25 are Japanese and German nationals.

The explosion has created severe structural damage to the railway bridge, premises adjacent to the bridge in an approximate 30 m radius and has created major leaks in two of the other tank cars. Structural damage of a moderate nature has occurred within an approximate 100 m radius. At least 50 people have been killed, including some of the foreign tourists. Many people have received burns from the flashover of the explosion and numerous people are trapped and injured in properties, beneath the bridge the structure and in vehicles on the roadway. A number have also been contaminated by aviation fuel. There are numerous fires in the area.

The leaking fuel has run down the road incline to the north of the bridge and is entering the canal and watercourses at the bottom of the incline. The river flows from east to west but there is no particular flow on the canal. The wind is north westerly Force 2–3. There are several barges used for residential purposes moored on the canal.

Early information from witnesses suggests youths have been seen running from the section of rail track where the derailment took place and that vandalism may be responsible for the derailment.

No recording device could be used for gathering data as each syndicate was located separately. Therefore, during the syndicate discussions data were collected through note taking by the researcher. These notes were an aide memoire for the researcher. All the syndicates were then brought together and asked to present their responses to the group as a whole. General information from these presentations was recorded in bullet-point form on a printable white-board. These records were printed for the researcher.

For each of the emergency services, the trainees were asked to describe: what their roles would be for the scenario; what their response would be to the scenario; what problems they would face; what problems the other services would face; and what potential conflicts there would be with the other services. The raw data recorded required explanation, as many aspects of the Emergency Service personnel responses were couched in language specific to the Emergency Services and would therefore be unclear or uninformative for a naïve reader. The researcher thus ‘expanded’ the data to express as much information as possible. This expansion was based on: the discussion within the syndicates; from further discussion within the seminar between the Emergency Services trainees and their trainers; from Emergency Services planning documentation and from information given by the Home Office Emergency Planning Research Group. Data were collected at two different stagings of Exercise Scorpio.

5. EMCRS model development

The EMCRS models were constructed by application of the HCIPCMT framework axioms and representation to the data collected from two stagings of Exercise Scorpio. The application of the HCIPCMT framework axioms to EMCRS will extend the framework to accommodate the EMCRS. Section 5.1 will present the HCI-PCMT framework axioms applied to EMCRS data. The HCI-PCMT framework representation was applied to the two datasets to produce two different EMCRS models. These models were then combined to produce EMCRS Model 1 which is presented and described in Section 5.2.

To reiterate to aid understanding for the reader, the HCI-PCMT framework makes a distinction between the interactive worksystem (users and computers/devices) and its domain of application or work domain.

5.1. HCI-PCMT axioms for EMCRS

5.1.1. Axiom 2.1 HCI-PCMT EMCRS design problems

HCI-PCMT EMCRS design problems and their possible solutions, generated by specify-and-implement design practice, entail the specification of the implementable3 planning (structures and) behaviours and control (structures and) behaviours of the emergency service personnel and emergency service devices of the worksystem, such that when they interact with the perception (structures and) behaviours and execution (structures and) behaviours of the emergency service personnel and emergency service devices, they provide support for a disaster, such that the actual level of performance falls within some desired level of performance.

5.1.2. Axiom 2.2 EMCRS domain: multiple task work

The domain is described at both an abstract and a physical level. At the highest level of description (Abstract level 2) is the Disaster object. This Disaster object is defined at Abstract level 2, the highest level. The attributes and values for the Disaster object were conceptualised as stability and normality, with values along a continuum. Thus, the overall task of the EMCRS is to transform the Disaster object to a desired level of stability and normality. The performance of the EMCRS worksystem is expressed by the transformation of the Disaster object’s attribute values. Each task carried out by the worksystem transforms the attribute values of the disaster object. These attribute values change by manipulation of the values of the attributes of the sub-objects of the domain. These sub-objects’ attribute value changes are affected by the sub-tasks of the EMCRS, which are the individual agency tasks (also the multiple tasks in EMCRS). Each of these sub-tasks will have associated domain sub-object transformations – sub-transformations. The required transformation of the Disaster object can be divided into a number of sub-transformations, concerning particular sub-objects and their attributes. The attribute normality was conceptualised from the notion that at the beginning of the disaster scenario the ‘disaster’ is chaos, and the work of the EMCRS is ultimately to restore normality, i.e. to bring the disaster under control. Thus, the more desirable the level of normality the better the performance of the EMCRS. The second attribute – stability was conceptualised through an understanding of the expected overall performance of the EMCRS worksystem – that of stabilising the disaster (preventing further loss of life and containing fires and other hazards). Both of these attributes’ values are changed by the transformation of the sub-object attribute values.

The other abstract objects, which are sub-objects of the disaster object, were conceptualised from: the primary objectives of the EMCRS (see Section 2); the primary roles/tasks for the emergency services (Anon., 1994); information from the exercise narrative; and the data. The sub-objects of the domain: the Lives sub-object; Disaster Character sub-object; Disaster Scene sub-object; Property sub-object, Environment sub-object and Emergency Service sub-object are at a lower abstract level of description – Abstract level 1. At the physical level of description, the abstract sub-objects of the domain are realised as physical objects. Abstract level 1 objects have realisation attributes whose values specify the physical objects of their realisation. These three levels are the same as those for the office administration domains. Vertical relationships exist between the values of attributes at different levels of description. The realisation relationship between Abstract level 1 objects and Physical level objects is a many-to-one relationship. The value of physical object attributes determine through emergence the values of Abstract level 1 attributes. In turn, the values of attributes of the Disaster object (Abstract level 2) are determined by an emergence relationship from the values of attributes at Abstract level 1 – the sub-object attribute values. Horizontal relationships exist. There is a relationship between the values of the attributes at the same level of description. For example, at Abstract Level 1, the Disaster Scene sub-object attribute scene containment value contained will require the Lives sub-object emergency services personnel safety to have a value of equipped. A task is the required state transformation of the Disaster object, including all lower level transformations of the associated sub-objects. As there is only a single Disaster object, at the highest level of description, multiple task work is that domain work in which, at the highest level of description, the Disaster objects attributes are undergoing independent, but temporally overlapping transformations. This HCI-PCMT extension is to accommodate a multi-user planning system. (The EMCRS domain thus, contrasts, with the office administration domain where there are multiple objects at the highest level of description.)

5.1.3. Axiom 2.3 EMCRS worksystem: planning and control behaviours and structures

Four types of abstract behaviour are generic to the worksystem and undifferentiated between the users and devices. These behaviours are planning, control, perception and execution. The four types of abstract behaviour are supported by abstract structures, also undifferentiated between users and devices. These abstract structures comprise four types of process, corresponding to the four types of behaviour. That is, a planning process, a controlling process, a perceiving process, and an executing process. There are two types of representations: a plan representation and a knowledge-of-tasks representation. The EMCRS abstract worksystem is defined in terms of these abstract cognitive structures. The physical worksystem structures were identified from analysis of the data, from information about EMCRS structures, and information about the required personnel and resources for particular worksystem behaviours (specified in the roles/tasks of the different services). Thus, from information on the EMCRS, we have commanders at the tactical and operational levels of control. From information on roles/tasks for the emergency services, and from the data, one role of the Police Service is the preservation of the scene for evidence and enquiries. Preserving the scene will require Police personnel to manage the scene. All the physical structures required for model diagnosis of design problems from the identified conflicts are represented in the model. These abstract and physical structures are shown in Fig. 2.

Fig. 2. EMCRS physical and abstract structures.

The behaviours associated with these structures are as follows: Perception behaviours are those, whereby the worksystem acquires information about Property, Lives and other domain objects, such as their risk status, and records these values. The states of the domain objects form the contents of the Knowledge-of-tasks representation. Perception behaviours update the contents of the Knowledge-of-tasks representation, based on the reading of the domain, for example that there are properties at risk. Execution behaviours are those which carry out the work of the worksystem by transforming the values of the domain object attributes values directly; for example, treating injured survivors at the scene transforms the Lives sub-object attribute survivor status from untreated to treated. These execution behaviours will in turn transform the Disaster object to a more desired level of stability along its continuum. Planning behaviours are those that specify what and/or how tasks will be accomplished in terms of required object state transformations and/or required worksystem behaviours, for example, that the site should be declared a crime scene, which requires Police personnel to manage it. Control behaviours select which behaviours should be carried out next, based on the contents of the plan and Knowledge-oftasks representation. A plan representation structure embodies the plans used in the combined response. These structures are distributed across the different levels of the worksystem, i.e. strategic, tactical and operational.

5.1.4. Axiom 2.4 EMCRS performance

Performance is some function of: the task quality associated with the multiple task work (for example, lives saved, fires contained) and the resource costs associated with the worksystem planning and control behaviours (for example, plans correct, firemen in place). (As above resource costs are not differentiated between the user and devices, also resource costs are specific to the planning and control worksystem structures and behaviours.)

All the HCI-PCMT axioms have now been re-expressed for the EMCRS. These axioms and the HCI-PCMT framework representation were applied to the data from the training exercises to produce EMCRS models. Details of the how the models were constructed will not be presented here as the remit of the current paper is to show how the models can be used for design problem diagnosis.4 Model 1 is the combined model from both datasets.

5.2. EMCRS Model 1

Fig. 2 shows EMCRS Model 1, the model constructed by application of the HCI-PCMT framework to the combined datasets 1 and 2. EMCRS Model 1 shows all the abstract and physical structures of the worksystem identified from both datasets, on the left handside of the diagram. At the abstract level, the three tier structure of the EMCRS, operational, tactical and strategic is depicted (but interactions within and between levels are not shown). These abstract structures are distributed across the physical worksystem structures. This distribution is not shown here, due to the limitations of the representation. The abstract structures representation is the same as that shown in Fig. 1. The physical level shows all those structures that have been identified as necessary to inform the abstract structures of the worksystem for the conflicts identified within both datasets. On the right-hand side, the domain objects, attributes and values are shown, from both datasets, both abstract and physical. The physical level objects attributes and values are not shown in full, as the representation shown here cannot support this. (For further information see Hill, 2005.) The links, between the abstract sub-objects and the physical objects, shown with a dotted line define the abstract to physical realisation relationship. The realisation relationship has a one-to-many mapping. The full lines between the Disaster object and the other sub-objects define a part-of relationship. The attributes with a star (*) are dispositional, that is they need to be perceived by the worksystem; but are not changed by it.

Although a strategic level of command is represented in Model 1, this intent is for completeness of the EMCRS representation. The structures of the strategic level are not referred to in the model descriptions. The reason is because, although a strategic level of command was set up in the exercise and so is, thus, included in the representation, this command level is not activated in the initial response phase to the exercise. The data for the modelling are only from the initial response phase, and therefore do not refer to the strategic command level. It is often the case that a strategic level of command is not activated in major incidents, until later in the response, or sometimes not at all, if it is decided that it is not required.

The EMCRS Model 1 is intended to be a model for diagnosing planning and control co-ordination design problems of the EMCRS, and offering up potential solutions to these design problems. A method was developed to aid in the use of the model for diagnosis. Thus, once constructed, Model 1 was used to diagnose EMCRS coordination design problems, through application of this method. A planning and control co-ordination problem is diagnosed if actual performance does not equal desired performance, where performance is expressed as a function of task quality and resource costs to the system, i.e. performance expresses whether goals have been achieved and at what cost. The method therefore guides the identification from the model of the planning and control behaviours of the interactive worksystem associated with a given task and the corresponding desired domain object transformations, such that the performance of the system for this particular task can be expressed. Being able to identify where (at a planning and control level) within the system a co-ordination design problem is occurring enables better reasoning about potential design solutions to these problems. The next Section describes the method for co-ordination design problem diagnosis.

6. Method for co-ordination design problem diagnosis

Application of the method and the diagnosis of one co-ordination design problem will now be described (see Table 1).

6.1. Method stage 1 – identifying potential conflicts

The training scenario questions included one on inter-agency conflicts. From the data recorded in responses to this question, five sets of conflicts were identified. This section will describe one of these conflicts as identified in the data.

Conflict 2 Cordon restrictions – due to information in the Exercise Scorpio narrative, regarding structural damage to buildings and a number of fires at the scene, the Fire Service set up an inner cordon to contain the scene. The Fire Service are responsible for the safety of all personnel within the cordon. Access is restricted to those with regulation safety equipment. The Ambulance Service need access to locate casualties and either treat them at the scene, or transport them to hospital. The Ambulance Service do not have regulation safety equipment and are not allowed access to the casualties. The Fire Service task of containing the scene conflicts with the Ambulance Service task of locating and treating casualties.

Having identified the behaviour conflict, stage one of the method is now complete. The next stages of the diagnosis method are now applied to the identified conflict, to diagnose co-ordination design problems. Co-ordination design problems are diagnosed, if actual performance does not equal desired performance.

6.2. Method stage 2

The behaviour conflict arose as follows: The Fire Service operational commander (physical structure) carries out perception behaviours that update his knowledge-of-tasks with the information that there are structurally damaged buildings, fires and leaking hazardous fuels. He then carries out control behaviours that direct him to consult the major incident plan. The plan specifies that the Fire Service is responsible for setting up an inner safety cordon, when there are hazards and dangers at the scene, and maintaining the safety of all those within the scene. Based on this plan, and the knowledge-of-tasks (about the fires etc.), the operational commander carries out control behaviours that direct him to consult the operational plan for setting up of a cordon. The operational plan gives guidance for cordon set-up and regulations. The operational commander then carries out control behaviours that direct him to carry out planning behaviours, based on the operational plan and the knowledge-of-tasks. The planning behaviour specifies how the inner cordon should be set-up and what the regulations are for entering it. The operational commander then carries out control behaviours that direct him to carry out an execution behaviour of setting up the cordon. This execution behaviour is carried out by the operational personnel (firemen) setting up the cordon and maintaining specified safety regulations. This physical object manipulation transforms the abstract Disaster Scene sub-object’s attribute scene containment from un-contained to contained. Thus, transforming the Disaster object attribute of stability to a more desired level along its continuum. (At the same time, other operational firemen and their fire equipment are controlling the fire and stabilising buildings, transforming the attributes of the Property and Disaster Character sub-objects, which are again transforming the Disaster objects stability attribute towards its desired level.)

Method Stages	Action	Example for Clarification
1	From data, identify tasks carried out by each agency in response to the scenario, where there are potential conflicts	Set-up of inner cordon by the Fire Service; access to casualties for triage without regulation safety equipment by the Ambulance Service
2	Use Model 1 to describe the behaviours associated with each task and the corresponding desired domain sub-object transformations	Desired domain sub-object transformations are those transformations that would be carried out, if an agency’s behaviours are not hindered. For the above example, one desired domain sub-object transformation for the Ambulance Service would be: Lives sub-object attribute survivor triage status from untriaged to triaged
3	Identify behaviour conflicts i.e. which domain sub-object transformations will hinder other domain sub-object transformations	From the above example the Fire Service behaviours of transforming the Disaster Scene sub-object attribute scene containment from un-contained to contained have hindered the Ambulance Servic behaviours and corresponding domain sub-object transformations
4	Use Model 1 to identify whether other domain sub-object transformations that will be hindered as a ‘knock on effect’ from the initial conflict behaviour	For example, the Ambulance Service not being able to transform the Lives sub-object attribute survivor triage status from untriaged to triaged will mean that the Lives sub-object attribute survivor treatment status cannot be transformed from not treated to treated. Also, as the Ambulance Service cannot access the casualties, the Fire Service will have to move the casualties to the edge of the cordon to enable triage to take place. In so doing, the Fire Service will reduce their fire fighting and property protection behaviours, as personnel will need to be taken away from these tasks to carry out rescue behaviours and will therefore not be able to transform the Disaster Character sub-object attribute fire status from uncontrolled to controlled, and the Property sub-object attributes of buildings/vehicles status from at risk to not at risk
5	Identify the performance effect of the hindered domain sub-object transformations by referring to the overall common objectives and priorities of the EMCRS (i.e. to save life, to prevent escalation of the disaster, etc.). The primary priority for all services is to save life. Therefore, hindering any domain sub-object transformation that reduces life saving by the EMCRS will have the greatest impact on performance	In the current example, hindering triage and subsequent treatment transformations by the Ambulance Service, of the Lives sub-object will greatly affect the performance of the EMCRS with respect to the primary priority of saving life. Reducing the fire fighting and property protection behaviours by the Fire Service will have an effect on the secondary priority of preventing escalation of the disaster. Thus, Model 1 gives a performance expression of actual performance being less than planned/ desired performance, as a performance deficit, is shown for both agencies

Table 1 Method for co-ordination design problem diagnosis.

The operational commander then carries out control behaviours that direct him to inform the tactical incident officer of the inner cordon set-up. (Some kind of internal communication behaviour is carried out to inform the incident officer about the cordon set-up.) The tactical incident officer (and his communication equipment) then carries out perception behaviours, which update his knowledge-of-tasks about the inner cordon set-up. The tactical incident officer then carries out control behaviours that direct him to consult his plan to assess the resources required for the setup. He then carries out planning behaviours to specify the resources required for this task.

At the same time, the operational Ambulance Senior officer (tactical level), (with his communication equipment) is carrying out perception behaviours that update his knowledge-of-tasks with the information that there are a number of casualties at the scene. He then carries out control behaviours that direct him to consult his major incident plan. According to the plan, casualties must be triaged and then either treated at the scene and/or transported to hospital. He then carries out control behaviours that direct him to carry out planning behaviours to specify in the operational plan what personnel are required to triage, treat and/or transport the casualties. Based on this plan and the knowledge-of-tasks, he then carries out control behaviours to direct the execution behaviours of triaging, treating and/or transporting casualties. These execution behaviours are carried out by the Ambulance operational personnel for triage and treatment with their Ambulances for transport. It is the manipulation of the physical casualties, which transforms the abstract Lives sub-object attributes survivor triage status from untriaged to triaged, survivor treatment status from not treated to treated and survivor transport status from not transported to transported. In turn these sub-object transformations, change the Disaster object’s desired level of stability.

6.3. Method stage 3

However, the Ambulance operational senior officer (tactical level, and his communication equipment) has not carried out perception behaviours that update his knowledge-of-tasks, that the scene is now contained, and regulation safety equipment is required to enter it. Therefore, when the Ambulance personnel attempt to carry out their execution behaviours, they do not fulfil the proper safety requirements, which would allow them to enter the inner cordon. Therefore, the execution behaviours of triaging, treating and/or transporting casualties cannot be carried out. Thus, they cannot transform the abstract Lives sub-object attributes and so cannot transform the Disaster object to a more desired level of stability. Thus, a behaviour conflict has been identified. Identifying behaviour conflicts enables co-ordination design problem diagnosis.

6.4. Method stage 4

The primary objective of EMCRS is to save life. In order to try to increase the desired level of stability of the Disaster object, the Lives object attribute values must be changed. Therefore, the Fire Service must carry out rescue execution behaviours to move the survivors to the edge of the inner cordon, so that the Ambulance Service can carry out their execution behaviours and, thus, increase the stability of the disaster object. However, the Fire Service carrying out rescue execution behaviours will decrease the resources available for performing the execution behaviours of controlling the hazard, thus decreasing the effectiveness of the response to the secondary objective of preventing escalation of disaster. The outcome is that the performance of the EMCRS is reduced even further.

6.5. Method stage 5

The Fire Service behaviours of containing the scene have conflicted with the Ambulance Service behaviours of triaging, treating and/or transporting casualties, and result in a behaviour conflict. This behaviour conflict results in a co-ordination design problem, which may relate to reduced overall EMCRS performance, either through hindered goal achievement (e.g. lives not saved) or through unacceptable system resource costs (e.g. excessive firefighter workload). The model diagnoses an EMCRS co-ordination design problem as actual overall performance being less than desired performance with respect to the EMCRS common objectives. This co-ordination design problem relates to the common EMCRS objectives, i.e. to save life (casualties not rescued); to prevent escalation of disaster; (fire not contained). Model 1 describes a performance deficit, related to hindered goal achievement and unacceptable resource costs for the Ambulance and Fire Services. Containment of the scene with safety requirements by the Fire Service reduces casualty triage and treatment (life saving) by the Ambulance Service. The Ambulance Service having to wait to treat casualties will increase their treatment workload. The Fire Service scene entrance safety requirements excluding the Ambulance Service, reduces Fire Service fire containment, as they have to rescue casualties and move them to the edge of the scene. The Fire Service workload will thus increase.

A design problem diagnosis by the EMCRS model has been described. To authenticate this design problem diagnosis, a potential design solution to this problem will now be suggested. The execution behaviours of the Fire Service (setting up an inner cordon and maintaining specified safety regulations) have been identified as part of the cause of the identified design problem. How these execution behaviours came about can be identified by the model. The behaviours, that affected the Fire Service execution behaviours of inner cordon set-up, were the planning and control behaviours of the operational commander. Planning behaviour, as specified in the model, is based on the plans and the information in the knowledge-of-tasks representation (in this case, for example, structural damage and many fires at the scene). The plan which the operational commander consulted, should have specified that set-up of the inner cordon should not be progressed without the knowledge of the Ambulance Service (Chief and Assistant Chief Fire Officers’ Association, 1994). The Fire Service operational commander, thus, had an inappropriate representation of the plan, which has caused inappropriate planning behaviours, that have contributed to the design problem. Now that it has been identified, where in the planning system the problem is occurring, reasoning about potential solutions to this design problem, are possible. A potential solution is to enable adequate training of the procedures specified in the operational plans, such that inappropriate planning behaviours are not carried out, and such that actual performance equals desired performance, for inner cordon set-up by the Fire Service, i.e. the set-up is as desired and with acceptable resource costs.

The Ambulance service execution behaviours of accessing casualties for triage and treatment are identified as part of the cause of this co-ordination design problem. These execution behaviours are affected by the planning and control behaviours of the Ambulance operational officer. Planning behaviour as specified in the model, is based on the plans and the information in the knowledge-of-tasks representation (in this case, for example, casualties and an inner cordon). The plan that the Ambulance Service officer consulted, should have specified that liaison should take place with the Fire Service about what safety equipment is required for entering the cordon (The National Health Service Ambulance Service, 1994). The Ambulance Service operational officer had an inappropriate representation of the plan, which has caused inappropriate planning behaviours, which have contributed to the design problem. A potential solution is to enable adequate training of the procedures specified in the operational plans, such that inappropriate planning behaviours are not carried out, and such that actual performance equals desired performance, for Ambulance Service triage and treatment of casualties, i.e. triage and treatment are as desired with acceptable resource costs.

7. Summary and future work

This paper has presented research, constituting HCI substantive design knowledge, in the form of models that support diagnosis of design problems and reasoning about solutions to these problems.

Comment 7

 

This paper has described the development of diagnostic models of the EMCRS, through application of the HCI-PCMT framework to data from EMCRS training exercises. The HCI-PCMT framework was based on the Dowell and Long (1989, 1998) conception of HCI. The scope of the HCI-PCMT framework needed to be extended in order to be applied to the EMCRS. The HCI-PCMT framework, as shown Fig. 1, has a representation and a set of axioms for producing diagnostic design models. The framework axioms for EMCRS have been presented in Section 5. The EMCRS axioms address some of the differences in nature of the EMCRS and the other domains previously modelled by the HCI-PCMT framework. The EMCRS axioms are considered to be extensions to the HCI-PCMT framework. The most important EMCRS axiom relates to how to describe a task and its sub-tasks. These descriptions have important implications for the characterisation of multiple task work and subsequent identification of behaviour conflicts exhibited by the data. Thus, the EMCRS multiple tasks are the individual agency tasks, which are identified as sub-tasks of the domain. These sub-tasks carry out sub-object transformations in the domain, which affect disaster object transformations. Behaviour conflicts are identified, when sub-objects are not transformed as desired. Behaviour conflicts are diagnosed as co-ordination design problems by the EMCRS model. Without the EMCRS axiom extensions co-ordination design problem diagnosis would prove difficult, if not impossible.

A diagnosis method for identifying co-ordination design problems using the EMCRS Model 1 has been presented and then applied to identify one co-ordination design problem – cordon restrictions. However, this method is at an early stage of development and must be further developed for operationalising and testing. Nevertheless, it has been applied in the current research, albeit by the method developer, and thus, has been shown to facilitate in EMCRS model diagnoses. Future research would develop this method for operationalisation, testing, generalisation and so validation. Within the original research five co-ordination design problems have been diagnosed using this method. It is realised that without validation with further examples there is a question about the generalisability of these design problems. However, these diagnosed design problems were discussed with the Emergency Planning Research Group at the Home Office, who agreed with their existence and importance. The EMCRS data gathered for this research were from training exercises. It is understood that data from a real disaster would give the EMCRS model greater ecological validity, than using data from training exercises. People with access to such data (for example, senior emergency service training and research personnel) should however, be able to apply the current framework to such data, and produce models that would support co-ordination design problem diagnosis. However, due to the complex nature of the EMCRS, data at any lower level of description, than presented in the training exercises, would have been very difficult to analyse effectively. Also, the EMCRS is a management system for co-ordination of the planning and control in response to disasters. Thus, the EMCRS is specified at a high level, with respect to the operational, tactical and strategic levels of command. The EMCRS does not specify within a service how each individual person should co-ordinate with respect to their individual agency roles. The training exercises, where the EMCRS data were gathered, were set-up specifically to train the emergency service officers in how to use the EMCRS for the management of response with respect to operational, tactical and strategic levels of command. Thus, the trainees were all emergency service officers, and not operational level personnel. The exercise data were thus viewed as appropriate for modelling the EMCRS, for planning and control of the different command levels for disaster response. The aim of the research was to model the EMCRS to diagnose design problems, but more specifically planning and control co-ordination design problems. Thus, again data at this level of detail were considered appropriate for meeting the current aims. Future work would however, involve validation of the EMCRS model by application to data from an actual disaster.

Acknowledgements

The research presented in this paper was carried out within a PhD supervised by Professor John Long, and second supervised for the last year by Professor Ann Blandford. This work was part funded by the Home Office Emergency Planning Research group under the EPSRC CASE scheme. I would like to thank Chris Johnson and the two anonymous reviewers for their constructive comments in the revision of this paper. Thanks also go to the special issue editors for pointing me in the right direction on a number of issues.

References

Ambros-Ingerson, J.A., 1986. Relationships between planning and execution. Quarterly Newsletter of the Society for the Study of Artificial Intelligence and Simulation of Behaviour 57, 11–14.

Anon., 1994. Dealing with Disaster, second ed. Home Office Publication, London (HMSO).

Anon., 2001. Dealing with Disaster, third ed. Cabinet Office Publication, Liverpool, Brodie.

Report of 7 July Review Committee (2006) Greater London Authority. <www.london.gov.uk/assembly/reports/7july/report.pdf>.

Blandford, A., Wong, W., 2004. Situation awareness in emergency medical dispatch.

International Journal of Human-Computer Studies 61 (4), 421–452.

Blandford, A.E., Wong, B.L.W., Connell, I., Green, T.R.G., 2002. Multiple viewpoints on computer supported team work: a case study on ambulance dispatch. In: Faulkner, X.,

Finlay, J., Détienne, F. (Eds.), People and Computers XVI: Proceedings of HCI’02. Springer, pp. 139–156.

Card, S.K., Moran, T., Newell, A., 1983. The Psychology of Human Computer Interaction. Lawrence Erlbaum, Hillsdale, NJ.

Chief and Assistant Chief Fire Officers’ Association, 1994. Fire Service Major Incident Emergency Procedures Manual. CACFOA Services Ltd, Tamworth.

Dowell, J., Long, J., 1989. Towards a conception for an engineering discipline of human factors. Ergonomics 32 (11), 1513–1536.

Dowell, J., Long, J., 1998. Conception of the cognitive engineering design problem. Ergonomics 41 (2), 126–139.

Emergency planning college document, 1995. Management Levels in Response to a Major Incident. The Principles of Command and Control.

Fennell, D., 1988. Investigation into the King’s Cross Underground Fire. Department of Transport, London (HMSO).

Hayes-Roth, B., 1985. A blackboard architecture for control. Artificial Intelligence 26, 251–321.

Hidden, A., 1989. Investigation into the Clapham Junction Railway Accident. Department of Transport, London (HMSO).

Hill, R., 2005. Diagnosing Co-ordination Problems by Modelling the Emergency Management Response to Disasters. Unpublished PhD thesis, University of London. Hill, B.,

Long, J.B., Smith, W., Whitefield, A.D., 1995. A model of medical reception— the planning and control of multiple task work. Applied Cognitive Psychology 9, 81–114.

Hollnagel, E., Mancini, G., Woods, D., 1988. Cognitive Engineering in Complex Dynamic Worlds. Academic Press, London. Hutchins, E., 1990. The technology of team navigation. In: Galegher, J., Kraut, R., Egido, C. (Eds.), Intellectual Teamwork. Lawrence Erlbaum, Hillsdale, NJ.

Johnson, P., 1992. Human-Computer Interaction: Psychology, Task Analysis and Software Engineering. McGraw-Hill, London. Kaempf, G.L., Klein, G., Thordsen,

M.J., Wolf, S., 1996. Decision making in complex naval command-and-control environments. Human Factors 38 (2), 220–231. Klein, G.A.

Woods, D.D., 1993. Conclusions: decision making in action. In: Klein, G., Orasanu, J., Calderwood, R., Zsambok, C. (Eds.), Decision-Making in Action: Models and Methods. Ablex 405-411, Norwood, NJ.

Larkin, J.H., 1989. Display-based problem solving. In: Klahr, D., Kotovsky, K. (Eds.), Complex Information Processing: the Impact of Herbert A. Simon. Lawrence Erlbaum, Hillsdale, NJ.

Leplat, J., 1988. Methodologie von Aufgabenanalyse und Aufgabengestaltung. Zeitsch-rift fur Arbeits-und Organisationpsychologie 1, 2–12.

Long, J., 1987. Cognitive ergonomics and human computer interaction. In: Warr, P. (Ed.), Psychology at Work. Penguin, Harmondsworth. Long, J., 1996. Specifying relations between research and the design of human computer interactions. International Journal of Human Computer Studies 44 (6), 875–920.

Moray, N., Sanderson, M., Vicente, K., 1992. Cognitive task analysis of a complex work domain: a case study. Reliability Engineering and System Safety 36, 207–216. Newell, A., Simon, H., 1972. Human Problem Solving. Prentice-Hall, Englewood

Cliffs, NJ. Norman, D.A., 1986. Cognitive engineering. In: Norman, D., Draper, S. (Eds.), User Centered System Design. Lawrence Erlbaum Associates, Hillsdale, NJ, pp. 31–62.

Payne, S.J., 1991. Display-based action at the user interface. International Journal of Man-Machine Studies 35, 275–289.

Rasmussen, J., Vicente, K., 1989. Coping with human errors through system design: implications for ecological interface design. International Journal of ManMachine Studies 31, 517–534.

Rogalski, J., Samurcay, R., 1993. Analysing communication in complex distributed decision-making. Ergonomics 36 (11), 1329–1343.

Samurcay, R., Rogalski, J., 1991. A Method for Tactical Reasoning (MTR) in emergency management: analysis of individual acquisition and collective implementation. In: Rasmussen, J., Brehmer, B., Leplat, J. (Eds.), Distributed Decision Making: Cognitive Models for Co-operative Work. John Wiley & Sons, Chichester, pp. 287–298.

Shepherd, A., 1989. Analysis and training in information technology tasks. In: Diaper, D. (Ed.), Task Analysis for Human Computer Interaction. Ellis-Horwood, Chichester.

Smith, M.W., Hill, B., Long, J.B., Whitefield, A.D., 1992. Modelling the relationship between planning, control, perception and execution behaviours in interactive worksystems. In: Diaper, D., Harrison, M., Monk, A. (Eds.), People and Computers VII; Proceedings of HCI’92. Cambridge University Press, Cambridge.

Smith, M.W., Hill, B., Long, J.B., Whitefield, A.D., 1997. A design-oriented framework for modelling the planning and control of multiple task work in Secretarial Office Administration. Behaviour and Information Technology 16 (3), 161–183.

Suchman, L.A., 1987. Plans and Situated Actions. Cambridge University Press, Cambridge.

The National Health Service Ambulance Service, 1994. Ambulance Service Operational Arrangements for Civil Emergencies. Internal Ambulance Service document.

Vicente, K.J., 1990. A few implications of an ecological approach to human factors. Human Factors Society Bulletin 33 (11), 1–4.

Whitefield, A.D., 1990. Human computer interaction models and their roles in the design of interactive systems. In: Falzon, P. (Ed.), Cognitive Ergonomics: Understanding, Learning and Designing Human Computer Interaction. Academic Press, London, pp. 7–25.

Woods, D.D., Hollnagel, E., 1987. Mapping cognitive demands in complex problemsolving worlds. International Journal of Man-Machine Studies (26), 257–275.

Wright, P., Fields, B., Harrison, M., 2000. Analysing human computer interaction as distributed cognition: the resources model. International Journal of Human Computer Interaction 15, 1–41.

Young, R., Simon, T., 1987. Planning in the context of human–computer interaction. In: Diaper, D., Winder, R. (Eds.), People and Computers III; Proceedings of HCI’87. Cambridge University Press, Cambridge, pp. 363–370.

Human–computer interaction: A stable discipline, a nascent science, and the growth of the long tail

Alan Dix

Lancaster University, Computing Departments, InfoLab21, South Drive, Lancaster LA1 4WA, United Kingdom

John Long Comment 1 on this paper

Abstract

This paper represents a personal view of the state of HCI as a design discipline and as a scientific discipline, and how this is changing in the face of new technological and social situations.

Comment 2

Going back 20 years a frequent topic of discussion was whether HCI was a ‘discipline’.

Comment 3

 

It is unclear whether this was ever a fruitful topic, but academic disciplines are effectively about academic communities and there is ample evidence of the long-term stability of the international HCI/CHI community.

Comment 4

 

However, as in computer ‘science’, the central scientific core of HCI is perhaps still unclear; for example, a strength of HCI is the closeness between theory and practice, but the corresponding danger is that the two are often confused. The paper focuses particularly on the challenge of methodological thinking in HCI, especially as the technological and social context of HCI rapidly changes. This is set alongside two other challenges: the development of reliable knowledge in HCI and the clear understanding of interlinked human roles within the discipline. As a case study of the need for methodological thinking, the paper considers the use of single person studies in research and design. These are likely to be particularly valuable as we move from a small number of applications used by many people to a ‘long tail’ where large numbers of applications are used by small numbers of people. This change calls for different practical design strategies; focusing on the peak experience of a few rather than acceptable performance for many. Moving back to the broader picture, as we see more diversity both in terms of types of systems and kinds of concerns, this may also be an opportunity to reflect on what is core across these; potential fragmentation becoming a locus to understand more clearly what defines HCI, not just for the things we see now, but for the future that we cannot see.

1. Overview

This paper has its roots in the Inaugural Lecture of SIGCHI Ireland (Dix, 2008), where it seemed a suitable occasion for a sort of ‘state of the nation’, giving a personal view of where HCI stands as a discipline and how it can develop and grow. This then seemed a suitable topic to build upon for this John Long Festschrift special edition of Interacting with Computers, as Long has himself written with such sharp insight on the directions of the field and raised questions that have prompted many others to look at the discipline of HCI as a whole, not merely their own work within it. The SIGCHI Ireland talk also drew upon the discussions at the UCLIC/Equator two-day workshop on The Future of HCI in the UK in 2007 (see also Blandford, 2007), especially the discussion on roles and genres in HCI.

The basic argument of this paper is that while HCI has matured as a community and also as a practice, it is still developing, or needs to develop its own roots as an academic discipline.

Comment 5

I focus especially on methodological thinking, not in the sense of attempting to establish a single methodological framework, which seems doomed to failure, but in encouraging an ongoing methodological critique of the methods we borrow from other disciplines as we adapt them to our own.

Comment 6

This discussion of methodology is set alongside two other, interconnected, challenges to the discipline, the need to establish reliable, validated knowledge (another theme of Long’s, although here used more broadly),

Comment 7

and the need to understand the way different roles within the HCI community fit together and should communicate their results in order to produce a stronger science as a whole.

Note that I will occasionally use the term ‘science’, but in a broader, perhaps more historically felicitous sense, than the ‘hard’ sciences. Much of HCI research (but not necessarily all, see Section 3) is oriented towards producing knowledge that is usable for design, hence that has, in a sense, ‘truth’ about the world. This knowledge may not be quantitative, or formal, but does need to be what I term synthetic (Dix, 2008); that is helping you to achieve some effect, or, in Roger’s (2004) terms, prescriptive, ”providing advice as to how to design or evaluate”. In the broad academic dichotomy this seems the domain of the Sciences as opposed to the Arts.

Fig. 1. Roadmap of the argument structure of the paper.

As this is a long paper, Fig. 1 shows a roadmap of the main argument structure. The next section provides a motivation for this work, noting the strengths of HCI as a growing discipline and the need to develop stronger independent disciplinary roots. Section 3 takes a short detour and discusses the issue of what HCI is about, its ultimate purpose and goal as a discipline (as opposed to the purpose of individual pieces of work within the discipline); this is largely to contextualise the work within that of others, most notably Long, who have focused strongly on this question. In contrast this paper is more about how the discipline is conducted, what it does. Section 4 picks up this theme proposing and discussing three challenges for the discipline: methodology, knowledge and roles. The need for clarity in understanding methodology is particularly important given the rapid changes in the technology and human practices that are the subject of HCI. Section 5 discusses these changes, probably familiar ground for many readers. This then leads, in Section 6, to a case study of methodological thinking in the work of Razak on single person studies. This technique is of particular use when applied to design for peak experience, best for some as opposed to good for all. This trend towards the long tail of more individual micro-applications and the democratisation of information technology seems to be common to many of the developments in the use of technology and presents its own challenges for traditional HCI practice.

The paper devotes a substantial part of Section 4 to discussing evaluation and the whole of Section 6 to the use of single person studies. In both cases these are used as examples of the importance of clear methodological thinking within HCI; although evaluation is itself a central topic in the discipline. It is hoped that the paper gives insight on both these topics; however, the main focus is on the broader issue of methodology alongside the other core challenges of knowledge and roles in HCI as an integrated community and a scientific discipline.

2. HCI discipline and science

2.1. The growth of a community

The roots of HCI go back at least 50 years, with Brian Shackel’s (1959) paper on ergonomics of displays. However, the real beginnings of HCI as an emerging discipline are more like 25 years ago, with the founding of early conferences: Interact, CHI, British HCI and Vienna HCI (now ceased). My own first international conference was Interact ’87 in Stuttgart at which Brian Shackel gave a plenary, a welcome, I believe, on behalf of IFIP TC.13.1 A key question he posed was whether HCI was a discipline, or merely a meeting between other disciplines. Now this seems rather like navel gazing, but, at the time when HCI was developing coherence, it was a significant question.

Looking back now we can easily say, ”of course it is an academic discipline”, because what is an academic discipline if it is not an academic community; after 25 years of IFIP TC.13, and numerous national societies: SIGCHI, Interaction (formerly known as British HCI Group), not least the recent SIGCHI Ireland – clearly there is a community!

Comment 8

But that is a little too glib. Science, using the word in the broadest sense, goes beyond community; to be an academic discipline also requires a coherent basis for knowledge. Mere acceptance of knowledge by a group is not sufficient; we need some assurance of the truth or validity of our knowledge.

Comment 9

 

When flying we are not happy to rely merely on the accepted opinion of aerospace engineers; we want to know that they have a basis for their designs beyond accepted practice.

2.2. Craft or science

This brings to mind the discussion in the late 1980s, initiated by Long and Dowell (1989), about whether HCI was a craft, engineering or science. Arguably craft is really more about individual experience, but craftsmanship is not what one would want in an aeroplane, nor in Internet banking. Whether we call it science, engineering or simply being academic, we need to be able to give away knowledge to others who should then be able to apply it with assurance. However un-politically correct it is to use this sort of positivist language – yes we do want truth and fact sometimes!

Comment 10

Note that in talking about the facts of HCI we need to distinguish between the domain studied, which is by its nature complex and nuanced, and our understanding of it, within which we seek clarity. Of course, we rarely, if ever, have complete knowledge – there is always an epistemological gap; and the level of confidence we accept varies from domain to domain. However, for many aspects of HCI research, while the subject matter is often culturally determined and rich, facts about these contexts are not matters of opinion. Similarly, in design itself, while the goals of design and the people for whom the product is being created are contextually and culturally situated, the success of methods in achieving appropriate designs is not.

Comment 11

So, are we getting there; are we developing this coherent basis for knowledge in HCI?

2.3. Second generation HCI

The demographic of the HCI community varies between countries, but certainly in the UK there are an increasing number of ‘second generation’ HCI people; that is people who have done PhDs, masters and maybe even undergraduate courses with a strong HCI element and have now become the teachers and senior researchers themselves. I would describe myself as originally a mathematician who moved into HCI, others have roots in psychology, computer science, sociology, but an increasing number are straight HCI people.
As a sign of community this is very powerful; no longing for a half-remembered academic homeland elsewhere, but an academic generation who own HCI as their home.
However, this has also given me concern for a number of years. As we gradually lose those strong connections with our old disciplinary roots, have we developed equally strong ones within HCI?

2.4. From community to discipline

Indeed, there are signs that we do not yet have strong enough methodological roots. One key example is the relationship between HCI research and practice. One of the great strengths of HCI is that the two are close. To some extent this is true of computing also, but even more so in HCI. There are few fields where the practitioners and researchers can so freely attend the same events, present work to one another and hold discussions. Again this is powerful for HCI as a community and good news for funders looking for industrial relevance; indeed, in many countries, it has been commercial pressure that has driven often reluctant computing departments to take HCI seriously. However, the danger of this is that it is easy to confuse the two. Nowhere is this more evident than in usability evaluation.

We all know that evaluation is the sine qua non of HCI; professionally, often the key role of the usability practitioner; and, academically, try getting a paper published without that evaluation box being ticked! The techniques and tools for evaluation are often the same for usability practice and HCI research, whether formal experiments, usability labs, ethnography, prototyping, or maybe even cultural probes or technology probes. However, whilst the techniques are similar, the goals are different. For the usability professional, the ultimate aim is to improve the product, whereas the goal of research is to gain new understanding.

Comment 12

In fact, even these goals are interlinked: research systems often need to be designed well enough for effective experimentation or deployment; and effective design will be based on a thorough understanding of the context and technology.

Comment 13

 

However, for the researcher this formative creation of an experimental prototype is NOT the research itself, but merely the preparation for the research; and for the practitioner the understanding they gain is primarily in order to design better systems now, not establish fundamental knowledge for 10 or 20 years’ time.

Comment 14

So here we have a great strength of our community, but one that needs a clear understanding of purpose in order to contribute to a well-founded academic discipline. Reading any conference proceedings or journal it is evident that this clarity of purpose is not yet there. We clearly have work still to do.

Comment 15

 

 

 

3. What HCI is about and how it goes about it

Long and Dowell (1989) seminal paper on whether HCI was craft, science or engineering, and moreover Long’s (1996) elaboration of the relationship between HCI research and design, were predominantly about what HCI is about as a discipline: the subject matter, what it produces; the latter paper in particular focused on framing the ”HCI general design problem”. Similarly, Diaper (1989) in his opening editorial for Interacting with Computers, and re-iterated more recently (Diaper and Sanger, 2006), suggests that the goals of the discipline of HCI are ”to develop or improve the safety, utility, effectiveness, efficiency and usability of systems that include computers”.

Of course, many would object to both Long’s and Diaper’s apparently Taylorist approaches to the definition of HCI’s purpose, certainly adding experience (McCarthy and Wright, 2004), or even human values (Harper et al., 2008).

Comment 16

 

However, it should be noted that Diaper and Sanger (2006) expressly state that for them ‘work’ is taken to encompass leisure and is the full activity of being human; that is, satisfaction is subsumed within effectiveness. Similarly, Long (1996) states that work includes ”office work, factory work and home work” (albeit followed by the rider ”any activity seeking effective performance”). However, this is all part of the discussion of what HCI is about.

 

Note that this ultimate ‘purpose’ of HCI is distinct from the goal of an individual piece of work discussed at the end of the last section. The goal of an evaluation of a prototype within HCI research should always be for gathering understanding not improving the prototype as an artefact. However, the larger purpose of obtaining that understanding may be to help others engaged in practical design to improve their own devices and systems or maybe purely for the sake of the understanding itself.

Comment 17

 

Although unavoidably based on the author’s personal prejudices about the ultimate purpose of HCI, this paper attempts, not to avoid this issue, but to address a slightly different one, namely, how HCI as a discipline goes about addressing its goals, concerns, purpose: that is how it goes about doing what it is about. Thus the breakdown in the next section is fundamentally about the community of HCI:

• (i) roles – how the community is constituted,
• (ii) methodology – how the members of the community act,
• (iii) knowledge – how the members of the community communicate.

Note that the ‘knowledge’ here has two purposes, (iii.a) communication within the community and (iii.b) communication to (or with) those outside the community. As I am focusing on the academic discipline of HCI, ‘outside’ here will include HCI practitioners (when acting in that role; a practitioner may also be an academic and vice versa). It is the latter, (iii.b), that Long (1996) is referring to when he describes the ‘discipline’ of HCI to be the ”use of HCI [knowledge]3 to support HCI practices…”. In contrast, the focus of this paper is more on the internal communications (iii.a) that together build a coherent and reliable body of knowledge, but of course the availability and reliability of that body of knowledge is exactly what is needed for its exploitation by those ‘outside’.

Comment 18

 

It may be evident that human interactions, including conferences and discussions, are not included explicitly in my breakdown, although they are of course often the place where ‘knowledge’ is first presented and also often the place where it is conceived. In fact the breakdown is not precious and it may be extended to include social and community events explicitly in this picture. Indeed, in reporting on the methods used in DEPtH, a project focused on understanding physicality in design, we remarked on the central importance of events as a source of knowledge and data as well as a place for community building (Dix et al., 2009). That is, the events were important in DEPtH as part of its methodology.

Comment 19

 

 

So, the focus on externalised knowledge as the locus of community interaction rather than face-to-face meetings may seem odd, but effectively it acknowledges the different ways in which knowledge is passed on. On the one hand there is a form of diffusion or contagion, where knowledge passes from person to person. This is not to suggest that knowledge is in lump-like memes (Dawkins, 1976), like passing on a library book; the process is much richer, more one of knowledge being formed and informed in the relations between people. We may discuss concepts with colleagues, at conferences, and in online forums, building up an understanding within the academic community. We may also discuss with practitioners, both learning about their concerns and experience and also passing on the research community’s accumulated and distributed understanding.

Comment 20

 

This human process is of the utmost importance. It can be seen as a form of long-term establishment of common ground. While the theory of common ground (Clark and Brennan, 1991; Clark, 1996) is normally applied to conversations or similar discourse, the same arguments for growing mutual understanding apply to communities albeit with slightly different ramifications and mechanisms. This human process is also a crucial part of education.

Comment 21

 

However, while the human process of diffusion and mutual understanding is important, it is not sufficient; one of the distinguishing features of an academic discipline is in its externalisation of knowledge. We communicate through papers, videos, and other artefacts such as software (although it rarely continues running for long). This is by nature de-contextualised and abstracted, inasmuch as any externalisation abstracts from its subject, but precisely because of this it is applicable and persistent. The published extant knowledge of the discipline is the defining boundary object4 of the community.

Now in fact this picture is itself idealised, and some might say naive, in that the interpretation of externalised knowledge is itself governed by understanding gleaned usually through diffusive processes. My own alma mater discipline, mathematics, is an arch example of this, where there is a vast, largely unwritten, understanding of processes and interpretation that accompanies the formal mathematical theorems and proofs. However, while it is certainly naïve to ignore the rich community that surrounds academic externalised ‘knowledge’, both practical and theoretical concerns suggest this externalisation as an ideal, or at least a touchstone.

As noted, this paper is not primarily focused on the purpose of HCI, what it is about. Still, the dichotomy between inside and outside, academic and practitioner deserves a few words.

Long’s own work situates HCI as an academic discipline (or in his words HCI research) in relation to HCI design as seen in the quote above or his phrase ”fit-for-design-purpose” (Long, 1996). Similarly Diaper, whilst critiquing the common use of ‘usability’ on its own as a synonym for HCI, still gives goals that are focused on design, albeit interpreted more broadly than pure usability: ”safety, effectiveness, efficiency, and usability”. Coming from a different, more qualitative, position within HCI, Rogers (2004) also, in her excellent review and critique of (social and psychological) theory in HCI, effectively situates HCI theory in relation to its utility in HCI practice.

In teaching I usually distinguish between HCI as an academic discipline and HCI as a design discipline. The latter concerns using skills, knowledge and processes in the production of devices, software and other artefacts (or more generally interventions (Dix et al., 2004)) that in some way influence human interactions with computers (or more generally technology). The former, HCI as an academic discipline, is the study of situations involving people and technology (note, the series name of the British HCI conference), the design practices involved in such situations, and tools and techniques that are or can be used in either. In order to clarify this distinction, the remainder of this section will use ”HCI Practice” to denote the set of situations and design practices and ”HCI Research” to denote the academic discipline.

Comment 22

 

This separation is relatively uncontentious and accords with all the positions described so far; in particular ”HCI Practice” here is close to or identical to Long’s ”HCI design”. What is critical is whether HCI Research is simply about HCI Practice, or whether HCI Research is for HCI Practice (and usually for its improvement). If the latter is the case, then HCI is purely a vocational discipline focused on its practical outcomes.5 The, now slightly dated, use of the term ‘usability’ for HCI or, more commonly now, ‘interaction design’, both orient us to regard HCI as just vocational. Now the word ‘just’ in no way minimises the importance of the vocational use of HCI, nor minimises its importance within HCI, but challenges whether it is the sole end of HCI.

Comment 23

 

The confusion already alluded to between the methods in HCI is partly fuelled by this apparent identification of HCI Research and HCI Practice. It would be consistent to maintain the distinction between HCI Practice itself (the practice) and HCI Research aboutbut-for-the-purposes-of HCI Practice. However, this muddling of the two in methodology does seem symptomatic of a general lack of distinction.

Comment 24

 

The roots of HCI undoubtedly began in the practice of creating systems and situations where people could effectively (and all the other adjectives) interact with technology. However, as HCI has grown, it has encompassed a whole study of human endeavour and activity, which appears to go beyond purely vocational aims. Probability was born out of gambling, but does not restrict itself to studies funded by bookmakers.

Whether or not one accepts the arguments for pure ‘curiositydriven’ research, and even if one takes a utilitarian approach, there are good reasons to believe that HCI Research for HCI Practice is often best served by focusing on HCI Research about HCI Practice.

Comment 25

 

An excessive utility focus tends to mean that research runs behind technology. Work on the newest thing is too late for it, and looking for the next big thing is almost bound to fail; the big win is in using the new, the old and ideas for the next as means of uncovering more general knowledge. It is this knowledge that will be of value for the next new big thing and the next and the next after that. For example, consider the explosion in Facebook research after it reached its first 100 million; this has applied some existing theory from outside HCI, but appears to be largely a matter of the field catching up on a phenomenon after it has happened, rather than informing the development and design. This clearly happens in any discipline when there are major changes, but it should be the exception not the rule.

Perhaps most worrying is that we can start to accept a technology-driven approach as normal. At CSCW 2002, Brand (1994) gave a keynote taking the idea of timescales of structures in his book” How Buildings Learn” and generalising this to look at different timescales in other kjnds of activity. He tentatively suggested that there was perhaps a similar set of timescales for research (longest timescale), development (faster) and production (now). However, one of the questioners suggested that in fact in CSCW (and read also HCI) research was if anything faster-moving than development, driven constantly by the latest technology; the audience seemed to agree without noticing the unintended irony of the situation following a talk that emphasised the importance of long-term thinking.

This all said, the rest of the paper stands neutral on this (important) issue of what HCI is ultimately about, and instead focuses more on the way in which we as a community orient ourselves to create reliable knowledge no matter what form it takes.

Comment 26

 

4. Three challenges

To recap, HCI research often has trouble distinguishing itself from HCI practice. Whether or not one believes that the purpose of HCI research is to serve HCI practice, clearly there is a difference between the two. However, this seems to stem from a lack of solid methodological roots as HCI has grown away from its parent disciplines, but not clearly established its own methodological heart.

Considering this, I propose three challenges, defined in the previous section, that we need to address in order to develop the academic discipline of HCI: challenges of methodology, of knowledge and of roles. As noted these are primarily challenges addressed to the conduct and process of the community in relation to building a systematic and reliable body of knowledge. I am not addressing the (more thorny) issues of what HCI is and what it is about. Even within this narrower remit, there are surely many other challenges, not least that of the inter-human relationships noted; however, three is enough to start with and is the classical number of points in any argument.

4.1. Methodology

New disciplinary roots require new methods. We have many methods in our HCI toolkit, so this does not seem to be a problem. However, these methods are often ‘borrowed’ from other disciplines. Within an established discipline one can use accepted methods without a great deal of thought as to whether they are appropriate because one is using them in the same way that others have before. But if we simply adopt these methods in a new context without considering the desiderata that made them appropriate in the original context, they may be misleading or lead us to conclusions that are downright wrong.

To adopt methods in a new discipline means we have to understand why the methods work; that is we have to think methodologically. Now here I am not using methodology in the way that has become common in computing and in HCI; that is simply to mean method! When I say we need to think methodologically I mean we need to think about our methods.

Comment 27

 

The frequent confusion between usability evaluation and evaluation in research is just such a failing–adopting methods without understanding methodologically why they are appropriate and what they are for. But to be fair this is because thinking methodologically is hard. Many established disciplines have been around hundreds of years and so have had time to develop or evolve appropriate methods, but if we look to newer disciplines or subdisciplines we often see methodological problems.

As an example, and at the risk of alienating a community … some years ago I was evaluating some work that was on the boundaries between HCI and distributed artificial intelligence (DAI). The work seemed to be fundamentally flawed, as the researcher had performed single runs of a stochastic simulation with different conditions. In an HCI setting this might be like running an experiment with a single user in each condition. Now because of the nature of simulations there are times when a single run is sufficient; when certain conditions hold, a long enough run will exhibit all possible behaviours and so one long run is effectively like doing lots of short ones. Unfortunately in this case, the runs were too short due to memory problems. I was worried that I was going to have to send the researcher back to do repeat runs of everything! Happily there was a second reviewer who came from the DAI community and so I was able to ask him about this . . . what was the accepted practice in the community. Again happily the other reviewer was not just from the community, but also had a deep methodological understanding of the history of the area. Early in the development of DAI a key figure had shown that single runs were acceptable so long as the relevant conditions applied … unfortunately the ‘single run’ part got remembered, but the conditions had been forgotten. The research we were evaluating was simply following accepted practice in its discipline . . . it is just that the accepted practice was flawed.

Now before you judge DAI too harshly, think how often you have read papers (or written them yourself) that quote ”you only need to test with five users” (Nielsen, 2000) or Miller’s (1956) 7 ± 2 without really checking that they are valid in the context? In the case of the figure of five users, this was developed based on a combination of a mathematical model and empirical results (Nielsen and Landauer, 1993). The figure of five users was calculated.

• (i) as the optimal cost/benefit point within an iterative development cycle, considerably more users are required for summative evaluation or where there is only the opportunity for a single formative evaluation stage;
• (ii) as an average over a number of projects and to be applied properly needs to be assessed on a system by system basis; and
• (iii) based on a number of assumptions, in particular, independence of faults, that are more reasonable for more fully developed systems than for early prototypes, where one fault may mask another.

Just as in the DAI example, Nielsen and Landaur’s original paper outlines many of these limitations, but the finding is typically used without similar care.

Similarly, Miller’s (1956) 7 ± 2 is about working memory, yet is frequently applied incorrectly when, in fact, other cognitive or visual processes are important, for example, the length of menus. In fact a visual menu does not require much working memory so long as it is organised clearly, and in laboratory experiments Larson and Czerwinski (1998) found that far broader menus were optimal. One can easily work out simple back-of-the-envelope models of menus to calculate the optimal trade-off between breadth and depth based on time to search the menu visually and time for the page to refresh (Dix, 2003). For the web, where visual search is typically much faster then refresh time, the optimal figure is typically 60-plus items per menu. This said, Miller’s 7 ± 2 may be useful when applied to the depth of the menu, if users try to keep track in their head of where they have been. There is some evidence that older users, often with poorer working memory, find deep menu structures particularly difficult (Rouet et al., 2003). For more examples of misuse, Eisenberg (2004) produced an excellent critique of 7 ± 2 from a designer’s viewpoint, although he does not appear to have realised that the poor uses of it are actually misuses.

As well as misusing past results, it is easy to go fundamentally wrong in the application of empirical methods whilst deriving new results. A vignette I have used several times previously (Dix, 2004,) illustrates the importance of methodological thinking and what can happen when this fails. It concerns a paper that was published at a major ACM sponsored conference in HCI a few years ago. To spare the authors, I will not name the paper. It may sound familiar; however, this is likely to be because it is typical of many papers that fail methodologically in the same way.

Fig. 2. Systems placed in a 2 x 2 matrix.

Note that the paper being described was not only in a conference with one of the most rigorous reviewing procedures, but also included among its authors a major figure in the field. The kind of methodological problems described below are not simply those of students or junior researchers, but common amongst the most senior figures in HCI.

The particular paper was in fact a solid empirical paper: experiment, design, and evaluation. It was considering collaborative support for a task; call it task X. The work began by considering three pieces of software; call them A, B and C:

• A domain-specific software, synchronous group interaction,
• B generic software, synchronous,
• C generic software, asynchronous.

The three systems were placed in a 2 x 2 matrix: domain-specific vs. generic on one axis and synchronous vs. asynchronous on the other (Fig. 2). Incidentally, these 2 x 2 matrices are ubiquitous in many areas and are indeed extremely powerful analytic tools (Dix, 2002, 2008).

The paper then went on to describe the experiment comparing these conditions. There were a reasonable number of subjects in each condition (not just 5!) and sensible quality measures were used for assessing the outcomes of the task. Furthermore the experiment revealed statistically significant results … well certainly p < 0.05. There were two main effects:

• (i) domain-specific software was better than generic software (Fig. 3a), and
• (ii) asynchronous was better than synchronous (Fig. 3b).

The paper then concluded that what was clearly required was the missing gap: domain-specific asynchronous software, and then went on to describe the design and evaluation of an application in this area.

This all sounds exemplary, so what’s wrong with it?

First of all the paper was a little strong in its suggestion that (i) and (ii) meant that domain-specific asynchronous software would be best of all. Interaction effects6 are very common in HCI, and there was no argument as to why we should not expect an interaction in this case. However, that said, certainly the results would suggest that it is a good case to investigate further.

However, the big problem is harder to see, indeed if you blinked at the wrong moment when reading the paper it was easy to miss entirely. The paper started off with three systems and quite properly analysed them along dimensions. However, it then went on to conduct the experiment as if there were two independent variables being manipulated, when in fact there was precisely one piece of software for each condition. This is analogous to having an experiment with precisely one user in each condition – clearly problematic, and yet it is common to use a single piece of software, just as in this case, and never realise there is a problem!

In case it is not obvious why this is so bad: these were three completely different pieces of software that happened to have the relevant properties. Suppose application B just happened to have been very poorly designed. Application B would perform worse than application A, giving rise to the apparent effect that domain-specific was better than generic (Fig. 3a). Application B would also perform worse than application C, giving rise to the apparent effect that asynchronous was better than synchronous (Fig. 3b). That is the effects may have been due to an entirely extraneous factor, and nothing to do with the actual properties being studied.

Fig. 3. Experimental results (schematic).

The problem here is that the paper (and many in HCI) has ‘borrowed’ controlled experimental methods from psychology, but these methods embody assumptions that often do not hold in HCI. In particular, controlled psychological experiments are designed so there is a single simple cause or manipulation between conditions. However, when used in HCI, as above, there are many uncontrolled causes. Often we want experiments that have some form of ecological validity, which makes this worse.

However, just imagine trying to run the above study as if it were really a heavily controlled psychological experiment: we take a single piece of generic synchronous software B, first tweak it so it becomes domain-specific (call it A) and then tweak it again to make it still generic but asynchronous (call it C). This sounds stronger, and sometimes can work. However, in this case (and many) you would need to take a piece of software that is well designed for a particular situation (task generic and synchronous) then change to ‘just’ make it domain-specific or ‘just’ make it asynchronous; and furthermore you would need to do this without changing anything else and have it equally ‘good’ in all other respects after the tweak . . . clearly not possible.

Comment 28

 

Does this mean no experiments are possible in HCI? Far from it; by understanding the assumptions underlying controlled experiments and the way in which HCI experiments do not meet these assumptions, we are in a position to alter the practice of the experiment and the methods of analysis in order to make more reliable interpretations of the outcomes.

In this case, we could collect additional qualitative data (video, logs, audio) as is common in HCI, but then use these in order to help interpret the quantitative measures. Based on our knowledge of human interaction and the data that was collected, the researchers might have been able to come to some judgement as to whether the effect seen was to do with the difference between synchronous and asynchronous interaction, or due simply to specific features of application B or C.

Quantitative end-to-end measures are good at telling you whether there is an effect, and how strong it is, but it is the qualitative data that helps you to understand why you are seeing the phenomenon. Furthermore, richer data can help reveal the mechanisms underlying the observed phenomena. By ‘mechanism’ I mean the details of how a person engages in some form of activity or task including, where appropriate, observable phenomena, social and cognitive aspects. That is, a detailed account not just of what happens end-to-end, but the steps, actions and thoughts that are necessary between. When you understand mechanism, it is easier to see whether an overall result is likely to generalise to new situation, and to address empirical or observational data in an analytic manner (Dix, 2008). Furthermore, if you understand mechanism then you may be able to add new measures or interventions to study finer aspects of the overall interaction.

The importance, and sometimes difficulty, of establishing mechanism is important in other fields also. Nutritional advice for many years quoted a recommended daily protein intake far higher than today. The reasons for this lay many years before when protein deficiency was first recognised during a famine in Africa. Once the medical team studying this realised the disease they were seeing was due to protein deficiency, they began to administer protein supplements until the signs of disease went away and concluded that this was the required amount of protein. What they did not realise was that the body also burns protein for energy if other sources (fats and sugars) are not available. The children involved in the programme were starving and the team were first providing sufficient calories in the protein supplement before the children could start to use it properly as protein. Like HCI research, nutritional research involves many complex interconnected factors and it is often hard to alter one without affecting others. This makes it even more important to understand the underlying physiological mechanisms, however difficult this may be, in order to prevent major mistakes.

While this section has focused particularly on evaluation and in particular empirical evaluation, it is here to serve as an example of the wider issue that we need to explicitly think about methodology in all its manifestations.

4.2. Knowledge

To be a strong discipline, we need ways of gathering sound knowledge, ways of knowing what is true, and ways of establishing validity. As part of The Future of HCI discussions in the UK in 2007, (Blandford, 2007) emphasises that ”HCI research delivers new insights … that are valid” and validation is critical also in Long (1996) discussion of the relation between HCI research and design.

As previously noted, evaluation is central within HCI, to the extent that if one wanted to point to a touchstone for what is accepted by the community to be the sign of valid research, surely evaluation would be it. The major exception is pure studies of existing work-practices in domains where technology is already present or expected to be introduced. Otherwise whatever kind of ‘thing’ you have produced as an outcome of your research, be it a concept, a method, a toolkit, or an application, what the reviewers of your paper want to see is some level of evaluation and typically evaluation with real users.

Comment 29

 

Of course seeking some form of validation of your work is critical – after all I have said that we are after truth not mere opinion. However, it is wrong to assume that evaluation is the only means to verify validity. In mathematics, you do not evaluate a theorem to see if it is true, you prove it, that is provide a justification of why it should be true. Mathematics is unusual in being able to put all of its trust in justification; the particular closed nature of mathematical argument makes this possible. In general, academic disciplines vary as to the relative importance of evaluation (Fig. 4); in particular, evaluation is more important where the phenomenon being studied is complex or hard to predict, or ability to reason may be limited. For example, in medicine one might establish, based on theory or prior art, that a particular family of compounds is likely to be effective in treating a condition (justification), but the complexity of the human body and pharmaceutical chemistry means you need laboratory studies and eventual clinical trials to find out which actually works (evaluation).

Fig. 4. Forms of validation.

Arguably if our work is only validated through evaluation it is pure invention, not academic research at all – after all we should have some reason for what we do, not just randomly trying what occurred to us in the bath one morning.

Comment 30

 

Evaluation is especially problematic for generative artefacts – that is things that in some way make other things or can be instantiated in different ways (Dix, 2008; Ellis and Dix, 2006). This includes theories, methods, guidelines, tools, architectures . . . indeed just about anything we produce as research outputs in HCI! The problem is that evaluation cannot exhaust all possible uses or instantiations of a generative artefact, so can never validate it fully. Indeed, as an easy to remember catch phrase:

the evaluation of generative artefacts is methodologically unsound (Ellis and Dix, 2006)

Even a single piece of software is a generative artefact as it is only in the specific moments of use that it becomes grounded. We cope with this in usability testing by trying to have sufficient users working on a sufficient range of tasks in order to sample the space of potential use. However, once we get to design notations or guidelines, sampling becomes all but impossible. To say we have reasonably covered the space we would need to get many different designers with many different briefs and then usability test each outcome with many different users … and that is just to answer the simple question ”does it work?”, let alone ”why?” and ”how can we improve it?”.

Note that this is not to suggest that empirical evaluation plays no part in validation of these complex generative artefacts such as methods; it is just that any empirical evaluation needs to be part of a theoretical argument or some other form of justification. As an example of this, Furniss et al. (2007) recent work studying usability evaluation methods is exactly adopting this approach, using a combination of observations of practitioners using different methods, but set within a theoretical framework including distributed cognition and resilience engineering theory. Related work on has shown how various usability evaluation methods have different scope as to what aspects of the design they are best suited (Blandford et al., 2008); that is each forming part of a larger argument or process.

Within HCI there are a gamut of techniques available for both justification and evaluation, including for justification:

• existing published results of experiments and analysis,
• one’s empirical data from previous experiments, studies, etc.,
• expert opinion (published or otherwise) and common sense,
• arguments based on the above. . . and for evaluation,
• fresh empirical evaluation, user studies, timing data, etc.,
• peer reviews of one’s work (do other people agree it is a good
  idea),
• comparison with previous work (do the parts that should behave the sameactually do so).

In any field, the powerful thing is how these work together to establish validity. Even in mathematics, the domain of pure justification, it is common to try out a potential theorem against example data either to look for counter examples (Popperian falsification (Popper, 1959) is evaluation based), or to suggest how a proof might proceed: anyone who has done geometry in school will have experienced this using sketched triangles and circles at the beginning of a proof. Here the evaluation is guiding the process of justification. This can be the case in HCI: as you notice patterns in empirical data you think ”of course, that must be because . . .”.

Equally important is that when one builds the justification of why something should work, the argument will not be watertight in the way that a mathematical argument can be. The data on which we build our justification has been obtained under particular circumstances that may be different from our own, we may be bringing things together in new ways and making uncertain extrapolations or deductions. Some parts of our argument may be strong and we would be very surprised if actual use showed otherwise, but some parts of the argument may involve more uncertain data, a greater degree of extrapolation or even pure guesswork. These weaker parts of the argument are the ideal candidates for focusing our efforts in evaluation. Why waste effort on the things we know anyway; instead use those precious empirical resources (our own time and that of our participants) to examine the things we understand least well.

This was precisely the approach taken by the designers of the Xerox Star. There were many design decisions, too many to test individually, let alone in combinations. Only when aspects of the design were problematic, or unclear, did they perform targeted user studies. One example of this was the direction of scroll buttons (see Fig. 5): should pressing the ‘up’ button make the text go up (moving the page), or the text go down (moving the view)? If there were only one interpretation it would not be a problem, but because there was not a clear justification this was one of the places where the Star team did empirical evaluation . . . it is a pity that the wrong answer was used in subsequent Lisa design and carried forward to this day, but that is a different story! (Johnson et al., 1989; Dix, 1998,)

Fig. 5. Xerox Star and modern (Mac OS X) scrollbars.

So ideally, for good science, we would focus our evaluation where our justification is weakest, thus obtaining maximum information from our work and pushing forward the field. Of course, we should certainly be aware, while we probe these areas of greatest uncertainty, that our assumptions may be wrong, that the obvious may in fact turn out to be false; but we should not primarily make the obvious our focus.

There is of course a place, albeit largely absent in HCI research, for reproducing previous studies as a basis for further work, especially when the earlier work was promising but inconclusive. In mathematics you will go back and recheck the proofs of earlier theorems on which your work depends. If you do not do this and subsequently a flaw is found in the older proof then your own work fails with it. If such a flaw were found in, say, Nielsen and Landauer (1993), would we have the means as a discipline to ‘fail’ all the succeeding work that relied on it?

There is a difference between reproducing previous studies for the purposes of verification and doing the obvious for the purposes of getting an ‘easy hit’. The overarching aim should always be systematically to increase the knowledge of the field.

Sadly when advising students I have to tell them that there is a conflict between this recommendation for good science, and what is best to get published. The easiest way to get a publication is to choose something that you have a pretty strong argument for and then run some sort of experiment on it. With such an experiment you know what to expect and so you can frame a clear experimental hypothesis, and are very likely to get a result that will be statistically significant. However, this gives least new knowledge. In contrast, experiments focused on the weak points in the justification will have unknown answers, may yield inconclusive results and are least likely to have statistical power – while they have the potential to add knowledge to the discipline they are risky for the individual.

Note that this risk is the opposite of ”nobody has done this, let us try it” experimentation. Instead it is the systematic exploration of gaps in knowledge set within a context that highlights valid possibilities.

As a discipline we should not find ourselves in a position where good science and publication are at odds. This is bad for new researchers entering the discipline and it is bad for the discipline itself. So when reviewing work we should seek

• (a) reasons why the issue/feature is as it is – that is, rational not just random ideas, systematic growth of the field; and equally important,
• (b) reasons why the issue/feature need not be as it is – that is, not obvious, adding information to the field.

Note that these two together ensure that collectively we systematically explore gaps in knowledge.

4.3. Roles

The discussion has moved to criteria for good work, but within HCI there are many genres of work, so we need different criteria of judgement depending on the genre. Again this can be a real problem during reviewing of papers. A couple of years ago as a meta-reviewer I had to explicitly say that I was entirely discounting one of the reviews, because the review was effectively criticising the genre of work within HCI, not assessing it with respect to criteria within the genre. Often it is not so clear and it is easy to let one’s own general opinions about the most appropriate approach (experimental, ethnographic, formal) colour the judgement of a particular piece of work. Blandford (2007) warns reviewers to judge research on its own merits, not ”would I have done it the same way”.

Now this is not to say that there is no place to critique and debate the validity of particular genres or approaches to work and assess their strengths and weaknesses when applied to particular problems. It is just that we should debate the validity of the genre as a whole within the discipline; and the validity of a given piece of work within its genre, so long as that genre is accepted as valid within the discipline and is applied within its understood bounds.

In the UK, the HCI community has noticed this is particularly problematic when it comes to reviews for projects and grants within HCI. It is very hard to get across-the-board support from reviewers to say a piece of work is of the best quality; someone will have something negative to say. If an HCI project is then viewed amongst those from different areas where the reviewing is more consistent (whether positive or negative), the best HCI projects will lose out compared to the best from those other areas. Now this is partly because the quality criteria within HCI are soft and less clear than in some areas, but partly because at least one of the reviewers may not like the general approach/genre.

There is certainly a need for discussion of the value of particular approaches and establishing new ones, but that should be a separate discussion. Furthermore, we need to think explicitly about these different approaches, techniques or genres and the criteria appropriate for each. This makes it easier to assign appropriate people to review work – and the recent CHI Conference subcommittees are an excellent move in this direction. Furthermore, if we are aware of these different criteria we can more easily say ”personally I don’t like this style of work, but within its genre it is strong”.

Just as there are different genres of work, there are different roles that we may take within HCI research.

Imagine a physics paper that started off with some experiments at CERN, then performed group-theoretic analysis of superstring theory, and finally applied the results to the design of a vacuum cleaner. This is clearly risible. But HCI papers are often like this, and furthermore expected to be: a little bit of theory, build a toy system, run some experiments, analyse the results, give implications for design.7 Now this can sometimes be done well, so it is not that we should never have work like this, but surely it should be more common to have different aspects of this work performed by those that do them best, rather than expecting every paper to have a bit of everything?

HCI as an academic discipline (and maybe science) will develop most strongly if we can understand how different parts fit together and allow people and teams to focus where their core strengths lie.

I can think of three broad roles (although I am sure there are more):

• generating ideas and theories,
• developing systems and designs,
• performing empirical studies.

Table 1 and Fig. 6 list some of the different criteria for each role; although in the process of delineating these criteria, empirical studies divide in two because the actual gathering of data may need different expertise from its analysis. This was certainly part of the origins of ethnography – report with as little interpretation as possible in order that someone else can interpret later.

	Some criteria
Ideas & theories	Clarity & Parsimony adequacy of explanation ability to feed into experiment, design, more theory
Systems and designs	Rationale, novelty (useful) critical appraisal (of novelty) availability for future research
Empirical studies – data gathering (experiment, study, ethnography)	Clarity of situation, provenance availability of data for further analysis
Empirical studies – data analysis (theoretical, inductive, statistical)	Soundness, lack of bias suitability for meta-analysis

Table 1 Roles and criteria.

Fig. 6. Roles in HCI research.

Comment 31

 

 

Also, the parlous state of statistics in Cairns (2007) is no doubt in part due to the ‘do it all’ methods. In medicine there are special medical statisticians who are not medics themselves but do the statistics, because the medics themselves do not expect to be able to do this.
Within each role there are criteria that are more to do with internal coherence of the work, but, at least as important, there need to be criteria about making sure the work contributes to the bigger picture of the discipline as a whole.

• If I have a new theory or framework; is it expressed clearly enough so that someone else can apply it to their new design, or analysis of experimental results?
• If I have constructed a new system that embodies some idea; is it available so that other researchers can deploy it for long-term study, or use it in an experiment?
• If I have gathered some ethnographic data; have I obtained sufficient consentsand described my data gathering techniques well enough so that the raw data can be made available for others to study in different ways?
• If I have performed some statistical analysis on an experiment; have I presented the results in a way that others can interpret and possibly perform meta-analysis?

The web was developed so that physicists could share data. We need to develop HCI so that we share data, systems and results equally easily, so that we can properly use each individual’s expertise and skills to build a coherent discipline that is greater than any of us.

Long (1991) emphasises the importance of a discipline having an accepted ontology, and in Long (1996), effectively develops a framework providing just such a high-level ontology. This is effectively about common language in order to communicate clearly. Sutcliffe (2000) proposals for reuse of HCI knowledge, and those engaged in patterns research (Tidwell, 2009, 2005; van Welie, 2009), instead seek to create common formats for sharing knowledge. In fact it is not essential that we all share a single language, for all our working and reporting, nor that we understand fully one another’s methods. Within our sub-areas of HCI we can use our own esoteric languages, but the core outputs of our work need to be communicated clearly to enable others to build on them.

Comment 32

 

It is not acceptable for an ethnographer to tell a technologist to go read Garfinkel, nor for me, with a PhD thesis originally about ”Formal Methods and Interactive Systems”, to tell an ethnographer to go read Gauss. Given the very broad nature of HCI, there may even be special roles for those who present the outcomes of one sub-area of HCI to others, a form of internal education within the discipline. Possibly we need suitable reward mechanisms such as special high profile venues for such communication works, in the way ACM Computer Surveys served computing as a whole.

5. The changing face of HCI

Of course HCI is changing as computer technology changes and these changes will require yet deeper considerations of the way we interact together as an academic community and discipline. These changes also demand that we question more profoundly accepted methodological practice. Most readers will be aware of the rapid rate of change in recent years, but this section briefly reviews these changes, before looking at a specific case study of the way some of these changes forced methodological reflection.

The birth of HCI as a discipline was around the same time as the introduction of the desktop computer, and it became hard to consider interfaces that were not WIMP-based GUIs. This was to some extent a breaking free from the computer in the machine room, but it only got as far as the desktop, and there it stayed for nearly 20 years. Indeed, Buxton (2001) wrote:

”In the early 1980s Xerox launched Star, the first commercial system with a Graphical User Interface (GUI) and the first to use the ”desktop” metaphor to organise a user’s interactions with the computer. Despite the perception of huge progress, from the perspective of design and usage models, there has been precious little progress in the intervening years. In the tradition of Rip van Winkle, a Macintosh user from 1984 who just awoke from a 17-year sleep would have no more trouble operating a ”modern” PC than operating a modern car” (Buxton, 2001).

However, over the last 5–10 years (with plenty of preliminary research work before), we have seen the role of the computer change dramatically in society.

With mobile and ubiquitous computing and tangible interfaces, the computer physically escapes the desktop into the outside world. This is not just the subject of research for the future, but day-to-day reality for all of us. I often ask people ”how many computers in your house” and most still say two or three or maybe (if they are very ‘nerdy’) more, but rarely do people remember their microwaves and HiFi, central heating and washing machines. Even our body load of computation is substantial. A few years ago I emptied my own pockets during a masters seminar and found four clear computers without anything particularly nerdy: (1) a mobile phone; (2) car keys, which include some form of coding processing for the remote locking; (3) a film camera, but with an LCD screen; and (4) the now ubiquitous chip in a credit card.

The Internet has also seen the computer escape the desktop virtually (Fig. 7). While the growth of corporate networking lead to the development of CSCW as an area, the Internet has been crucial in establishing collaboration that cuts across organisational boundaries and enters the home. It is also hard to believe that 10 years ago there was no Google. Equally it is easy to forget how un-ubiquitous universal Internet access is. In the 1998 business plan for aQtive, one of the dotcom companies I was involved with, we talked about designing products ready for the coming PopuNet Dix (1998) – the network for everyone, everywhere and everywhen. At the time this was just beginning to be a reality when at the office desk, but in the home a slow and expensive dial-up connection was the best one could hope for. The ‘everywhen’ was particularly critical – continuous not just continual, not just available (anytime), but always there; what is now termed ”always on”. For those with the iPhone in western cities this now appears to be the reality, but even within the UK, Europe or the US just move a little into rural areas, sail out to the islands, or climb amongst the mountains, and connectivity becomes more broken. Now move further afield to the developing world and we can see how easy it is to overestimate the universality of the Internet and correspondingly, albeit unwittingly, design to divide.

Amongst many things that have changed with the growth of the web is that much software that was a product has become a service. Although I still use an email client installed on my computer (because, whilst travelling a lot, I am not ‘always on’), many only use web-based email services. Now we also have online word-processing, spreadsheets and more. If you buy an expensive hair styling kit, then you will continue to use it even if you find flaws, but if you visit a hairdresser and do not like the service or style, then the next time you go to a different one. Shrink-wrapped products allow you one choice point maybe every few years, but services allow near continuous choice. From a business perspective for shrink-wrapped software you can ‘get away’ with bad usability and poor user experience, so long as you have good marketing; the customers have already paid their money by the time they find out it is rubbish! In a service-based world, usability and user-experience become key to success.

As we can see, these technological changes lead to changes in the environment within which HCI works, not just because we have different hardware to play with, but because recent technological change has had a major impact at a commercial and social level. Sometimes technology is servant to social change, perhaps ignored or resisted. However, at various points technological changes have made a radical difference to social order. For example, the invention and adoption of the stirrup not only revolutionised mounted warfare, but was also a driver for the whole feudal system (White, 1968). While wanting to avoid simplistic technological determinism, it is also clear that there are major societal, cultural and even cognitive changes that we need to recognise for their impact on research and practice in HCI as well as for their broader political and ethical import.

One of the aspects that is obvious is the increasing focus within HCI on user experience. The physical movement of computers out of the office into the home and into our hands as well as the domestication of the web means that the old utilitarian ‘efficiency and effectiveness’ now have to pay second fiddle to ‘satisfaction’. One of the amazing things about the numerous ethnographies of the home and of leisure (which are both methodologically harder than work ethnographies) is just how complex day-to-day life is (De Certeau, 1984). The industrial revolution, Taylorism, and the continuing need to deal with staff turnover, has led to a largely controlled and ‘normalised’ workplace with personal differences minimised. Of course workplace studies constantly show how much fluid working depends on various adaptations, but always set against and located within a framework of order. In contrast the home has never needed the same levels of uniformity except those established externally by work and society.

The domestication of technology is nowhere more apparent than in Web 2.0 with its focus on user-contributed context and social networking (O’Reilly, 2005). Again it is interesting to look back to the dotcom days, less than a decade ago. In 1999, when working on a new product/service, vfridge, we articulated the idea of the web sharer (Dix, 1999). At the time many were saying that there would be a shake-up in the web world with DIY home pages withering and all the traffic going to a tiny handful of sites (Yahoo!, AOL, Amazon) principally operating in a publication or broadcast manner, like the TV except with easier ways to buy things. Now this seems laughable, similar to the (misquoted) early predictions that five computers would be enough for the whole world,8 but at the time was becoming accepted wisdom. In contrast we sought to design products for the ‘web sharer’: ordinary people sharing with one another.

Everyone may be a web sharer—not a publisher of formal public ‘content’, but personal or semi-private sharing of informal ‘bits and pieces’ with family, friends, local community and virtual communities such as fan clubs.

This is not just a future for the cognoscenti, but for anyone who chats in the pub or wants to show granny in Scunthorpe the baby’s first photos.

Fig. 7. After 25 years chained to the desktop, the computer breaks free (images courtesy Matt Oppenheim).

the web sharer vision (Dix, 1999) There were probably others voicing similar ideas, but, like con tinental drift in the 1960s, this was against the prevailing wisdom of the time and so difficult to publish and hard now to trace. The crucial thing is that in 10 years what was completely counter-cultural has become passe. Given these rapid and substantial changes in the technical and social context of HCI, there is even greater need to re-examine methods and if necessary modify them or develop new methodology.

6. Case study: a single person study

Within this setting of changes in HCI we will focus on the PhD research of Fariza Razak on the use of single subjects in research and design. The purpose of this section is not to present the work in full, but rather to use it as a case study to illustrate the need for methodological thinking to address new kinds of issues. Because of this only sufficient details are presented for the purposes of illustrating the general issues; for more about Razak’s (2008) work see her thesis ”Single Person Study: Methodological Issues”. Interestingly, while studying just one user initially seems to stand against all accepted HCI practice, in fact we shall see that single user studies of different forms are common in many disciplines.

6.1. Background

Razak started out interested in mobile user experience and in particular looking towards mobile learning. As a preliminary exercise she conducted a small study asking a handful of people about their use of mobile technology. As anyone who has done this sort of study knows, it is very hard to get beyond the banal learning what you knew before you started. You ask questions and people tell you the answers you could have predicted, the difficulty is finding the questions to ask that are less obvious as questions and will lead to new knowledge really growing the field.

In the initial study one of the participants stood out as unusual. She said she rarely used her mobile phone and yet other answers seemed to suggest the opposite, for example referring to use of time organisation features that others hardly mentioned. Clearly, her initial answer was about the ‘normal’ use of the phone as voice communication, but it was more to her than that.

Because this subject was unusual and different I suggested Razak spent some time investigating her in more detail. Little did either of us know at that time that this would become the key focus of Razak’s PhD work.

Note that the subject was chosen because she was in one respect an extreme, an outlier, outside of the average. When analysing experimental results, outliers are often removed for statistical purposes, ignored as anomalous or extreme. Instead the outlier was chosen precisely because she was one. As an academic I always find extremes valuable. Partly because the abnormal, or extra-ordinary (strange how the words have different connotations) are just more interesting in themselves, but also because they cast light back onto the ordinary, showing us things that are often tacit and unnoticed. Djajadiningrat et al. (2000) also describe how extreme characters helped expose aspects of use, especially ‘undesirable’ emotions and character traits, that more ‘normal’ personas and scenarios may hide.

Estrangement, the ability to see the world from a different perspective, is of great value in uncovering the hidden dimensions of the quotidian and I challenge students to have deliberate bad ideas (Dix et al., 2006). Similarly, Merleau-Ponty et al. (1945) writes ”in order to see the world and grasp it as paradoxical, we must break with our familiar acceptance of it” and schraefel and Dix (2009) asks chemists to make cups of tea. Comedians are particularly good at this, seeing the oddness in the everyday, highlighting things that we recognise in ourselves that are slightly embarrassing or just strange when we look at them. Indeed I have often suggested that students look up humorous books about their domain of study as they may learn more from a comic’s eye than from many an academic study. The best ethnographers also seem to have this ability to see the normally overlooked details of a situation. Garfinkel (1967) used ‘breaching experiments’ with his students, getting them to break normal social conventions in order to see that they exist; like a car engine, one is unaware of the parts until they fail.

This simple decision to study a single unusual subject started Razak down the route of pursuing this single person study as the central focus of her work and her thesis became not one about mobile experience or mobile learning, but a methodological account of the issues surrounding the use of a single person for research and design.

6.2. The first text

One of the early steps along this path was a diary study of Razak’s subject. When we discussed the results of the study, the words of the very first entry leapt out of the page.

This first text read

Dear God Don’t need lots of frens! As long as real ones stay with me, so bless them all, especially the sweetest one reading this.

and the subject’s comment (her emphasis):

this SMS MADE MY DAY!

Research on SMS behaviour often discusses its use for intimate communications ”thinking of you”, ”love you” (Gamberini et al., 2004; Spagnolli and Gamberini, 2007). However, this message was something slightly, but significantly, different. The message here was in a way less personal; it was representative of a particular type of message: often small quotes of a devotional or otherwise encouraging nature, from friends, but not necessarily from her husband, or close family. John Rooksby (personal communication) described them as messages that need no reply. They are sent to encourage but not to establish communication in any interactive sense and certainly not to ‘communicate’ in an information sense – they are more gifts of thought.

Perhaps the closest thing in the physical world are the little cards or bookmarks that have poems, sayings or prayers written on them, often surrounded by flowers . . . a world away from the design studio with its austere black-robed occupants.

This single text message and the reaction it caused fundamentally changed our view of the use of the mobile phone.

But can studying a single user in this way contribute to theoretical HCI research or practical interaction design?

6.3. Research from single-user studies

At first the idea of studying a single user runs counter to common academic sense. Surely we need to study many users to be able to stand any chance of generalising results? However, it turns out that this is not so uncommon in other disciplines such as special education or studies of neurological deficit. For example, Battro (2001) studied the development of a child who had only a single brain hemisphere due to a congenital brain defect. Similarly, Damasio (1994) in building up his understanding of the role of emotion in human reason, draws heavily on documentary evidence of Phineas Gage, who in 1848 suffered a traumatic brain injury leaving him with full intellectual capabilities but severe emotional deficit. In psychology there have been well respected uses of single subject studies, and even in HCI ethnographies are typically of a single situation (even if it includes several people) and, as we have seen, experiments often use a single application or piece of software (even if they have many subjects).

Furthermore, the study of a single user brings particular benefits. As often found in ethnographic studies, rich empirical data reveals new issues . . . in this case the very first text! Furthermore, studying a single user in depth allows the researcher to build up a deep personal rapport with the subject and hence make sense of what would otherwise be irrelevant or meaningless aspects of the data.

In fact discovering novelty only needs one example, like a botanist discovering a new flower; a single specimen shows that the new species exists. Of course a different person at a different time in a different place would find a different flower. Studying a single person is not the way to find all the important issues, or establish how common a particular issue is, but the depth may be a good way to find new usage phenomena.

However, this still leaves us with the question: having found a new phenomenon, how common and critical is it? In other words, how do we generalise? In an empirical study if the sample of users is wide enough (not just psychology or CS students!), then we assume that if an issue is common in the sample it is common in the population. Of course, we can use new insights from any method, including those from studying a single user, to drive empirical work of this kind. However, in the case of the initial text message, extensive empirical studies were unnecessary for us to recognise that this was something that we would expect to see elsewhere, not for everyone (and maybe least for UK CS undergraduates, whom we might have studied in a larger scale survey), but at least for particular kinds of people and communities. That is we were able to generalise by reasoning based on the data we had seen, knowledge of other research work, our own personal experience and not least (albeit much undervalued in academia) common sense!

Generalisation through reasoning is again common in other areas, for example semiotics and mathematics, and is typically based in deduction or abduction rather than induction as used in reasoning from voluminous empirical data. However, possibly drawing on my mathematical roots, I would like to make a stronger claim:

generalisation never comes (solely) from data,

Instead, generalisation always comes through understanding. Even when we have copious data, the knowledge that we have chosen representative groups, the level of extrapolation we choose to make from the experimental tasks, or the belief in the methods are all matters of judgement. We generalise with our heads not our senses.

6.4. Designing for a single user

Akin to the research question of how we obtain knowledge from a single user is the practical one of whether we can use a single user in design. We all know that ‘five users is enough’. . . but one!

So as an experiment Razak attempted to design an application especially for that single user. Having got to know this individual intimately, what would be perfect for that single person? With a single user it is possible to spend sufficient time to collaboratively co-design in a way that is tuned for the specific lifestyle, abilities and personality of the user. Having done this one can then ask whether the application would work for others and maybe do more traditional user testing. Maybe this hyper-tuned application may form the start of a slightly more generalised application that is of more general appeal.

We do not expect such a perfectly tuned application to be liked by everyone, indeed often the opposite. In the case of Razak’s subject the application periodically texted uplifting messages, and one test user clearly found some of the messages simply annoying. However, a surprising number of other users did find it engaging.

Again, like single-user studies in other disciplines, one does not have to go far to find areas where taking 100, 20 or even 5 users would seem like overkill . . . indeed many designers find no users sufficient (although sometimes this is apparent).

In fact Nielsen and Landauer (1993) calculate the ”five users” figure based on a cost-benefit trade-off between the number of faults found with N users, the costs of performing the test with them, and the costs of a prototyping cycle. This calculation also took into account the fact that the number of new usability problems found with each additional user drops due to overlapping faults between users. If the costs of prototyping are high compared with the costs of usability tests, then it is worth doing more usability tests in each iteration cycle, if prototyping is cheaper or usability testing more expensive, then tighter cycles are optimal with fewer users tested per cycle. Nielsen and Landauer measured the actual prototyping costs in a number of projects compared with actual error rates and usability test costs and it was these empirical figures, derived using 1993 technology and applications, which gave rise to the now ubiquitous ”five is enough”.

Of course prototyping costs are now substantially smaller than they were in 1993, and if the costs of prototyping are low enough, then the optimal point may not be five but even a single user per cycle. This is precisely the approach taken in Marty and Twidale’s (2004, 2005a,b) ”extreme evaluation”, where short usability tests are carried out with a single person.

While extreme evaluation only evaluates with a single person per prototyping iteration, successive iterations will typically involve different people. Furthermore, the software being designed was not designed ‘for’ the single individual, just tested with one user. In contrast, Razak developed an application specifically optimised for just one user, although our expectation was that this would in fact lead to a concept that would have wider appeal. This form of single-user designing would not be good for all kinds of application of product, but is particularly useful when designing for peak experience.

6.4.1. Designing for peak experience

Imagine you have a group of children and want to give them lunch. In the UK you might well choose baked beans. Not the most exciting choice, but few children actively dislike baked beans; they are acceptable to everyone. However, give each of those children a euro (or maybe two) in a sweet shop . . . they will all come away with a different chocolate bar, the chocolate bar that is ‘OK’ for everyone gets chosen by none. Or imagine choosing a menu for a wedding dinner … maybe chicken with a bland sauce … something nearly everyone will eat, but few people would choose for themselves in a restaurant.

Much of traditional HCI design is like baked beans – a word processor installed for the whole company, a mail program used by every student, good enough for everyone. However, increasing personal choice, especially for web-based services, makes design more like the chocolate bar; different people make different choices, but what matters is that the product chosen is not ‘good enough’ for all of them, but best for some.

This designing to be best for some, the chocolate bar rather than the baked beans, results in a product for peak experience. Fig. 8 shows schematic profiles of three imaginary products. The horizontal axis represents different people/users and the vertical axis represents their level of satisfaction or experience. There is one ‘good enough’ product, which offers a consistent but mediocre experience. There are also two ‘peak’ products that offer high satisfaction for a few users and low satisfaction for others. Note with sufficient peak products the good-enough product will never be chosen.

Traditional user-centred interface design may use user profiles, or personae chosen to be representative of a group as a whole, with a focus on the typical or the average – good for all. We typically move from identified user needs to interaction solutions with an emphasis on method and processes that ensure usability.

In contrast, designing for peak experience may need a stronger focus on the individual user, possibly extreme personae, focusing on the specific and eclectic – best for some. Often the move is from concept to use with an emphasis on novel ideas and inspiration.

Some years ago I was on a panel at ECCE with Jon Sykes (2004) from the group at Glasgow Caledonian studying games and emotion. He was asked about the processes used by video games designers and many in the audience were shocked at the apparently ad hoc and non-user-centred way in which the designers have ideas, discuss them amongst themselves and only very late in the process submit them to user testing but this is exactly what one would expect in order to design for peak experience; a good enough video game will be bought by no-one.

Similarly, many of the most successful Web 2.0 sites such as Facebook and del.icio.us started out being for the designers and their friends; tuned to a small group or even a single individual. We would normally castigate designers who design for themselves, but somehow, in spite of that, or maybe because of that, they are successful.

Interestingly, even the computer language of choice of many of these sites, PHP, was originally developed by one person for his own home page (PHP, 2009).

Now this is not to say that there is no role for traditional HCI practice; there are many products that do need to be used by everyone (e.g. bank web sites) and even in Web 2.0 web sites, such as YouTube, there are some aspects where traditional usability breaks down and experience dominates, but other parts, such as the uploading of videos, where standard usability is crucial (Silva and Dix, 2007).

However, where individual choice and user experience dominate, we need to look increasingly at peak experience. Mash-ups, widgets and open-data allow large numbers of applications designed for smaller groups of intended users; indeed one of the defining features of Web 2.0 (O’Reilly, 2005) is this focus on the long tail of large numbers of web sites and web applications used by few people, as opposed to more traditional web applications aimed at mass use. We cannot expect that the vast army of mash-up builders will each employ a usability consultant; HCI for the long tail may need to consider how to build necessary aspects of usability into platforms or maybe even popularise HCI – the equivalent of ‘house makeover’ programmes on daytime TV.

7. Bringing it together

The single person study was introduced as a case study to show the importance of clear methodological reflection. At first it appears to flout the community conventions for effective HCI. However, by understanding methodology we were able to recognise similar methods in other disciplines and within HCI and also to see how it could be used effectively as part of research and design. Note that this is not to promote single-user studies above other techniques, but if it and other methods can be understood methodologically they can be applied where they are appropriate and give value depending on the context.

Note too that the adoption of the single user study was because we were addressing an issue at the changing boundaries of HCI. As a textbook writer I am always interested in what changes and what does not change between revisions. The things that have hardly changed in 15 years are likely to still be of value in another 15 years, but the things that changed in the last revision are likely to change again. We need to look particularly carefully at the new or changing things, so that we see the things of lasting value and are not simply swimming with fashion.

Fig. 8. Profiles of a ‘baked beans’ good-enough-for-all product, vs. two products offering peak of experience for some.

Just as taking the extreme user helped us to understand ‘normal’ use, so also as we look at new areas of technology, they help us understand afresh the old. Often the lens of unfamiliarity helps us explore the heart of things.

Others exploring the extremes of HCI have also needed to think seriously about methodology and how to adapt it to the circumstances of their work, for example, Button and Dourish (1996), with technomethodology; McCarthy and Wright (2004), in designing for experience and enchantment; and Gaver (2007), in ‘polyphonic assessment’ of designs for everyday life.

The danger of establishing new methodology and potentially new vocabulary and theory is that we further fragment the genres and roles within HCI. How do we avoid a community of interest becoming a cabal?

This takes us back to the three challenges at the heart of this paper. We do not need to establish a common ontology or model for all of HCI, a single method that we all use; we do not even need to understand the details of the language, methods and theories within all of our sub-communities. However, we do need to ensure that we understand the genres of work and the roles they play within a coherent discipline, we do need to ensure that the methods used within each role and genre are coherent, and above all we need to ensure that the results of each genre (not necessarily the full arguments and methods) are communicated in a way that is accessible to others in the wider community.

To be an academic discipline is about community, but not just any community, a community that establishes clear knowledge and together learns.

Acknowledgements

Many thanks to Aaron Quigley, Gavin Doherty and Liam Bannon for inviting me to share in the inaugural celebration of SIGCHI Ireland on which this paper was based; and to everyone who attended the presentation and chatted after and before that talk.

Special thanks also to Fariza Razak whose work I have referred to extensively, to Matt Oppenheim (alias hardware monkey) for his wonderful cartoons of the computer breaking free, and to Fiona Dix for proofreading. Thanks also to the critical insights of the anonymous reviewers of this paper and to the special issue editors.

References

Battro, A., 2001. Haifa Brain is Enough: The Story of Nico . Cambridge Studies in Cognitiveand Perceptual Development, vol. 5. Cambridge University Press.

Blandford, A., 2007. HCI research and quality: discussion document. In: UCLIC/

Equator Two-day Workshop on The Future of HCI in the UK: Research and Careers, 14–15th June, 2007. Loughborough University. <http:// www.uclic.ucl.ac.uk/projects/future-uk-hci/>.

Blandford, A., Hyde, J., Green, T., Connell, I., 2008. Scoping analytical usability evaluation methods: a case study. Human–Computer Interaction 23, 278–327. doi:10.1080/07370020802278254.

Brand, S., 1994. How Buildings Learn: What Happens After They’re Built. Viking Press.

Button, G., Dourish, P., 1996. Technomethodology: paradoxes and possibilities. In: Tauber, M. (Ed.), Proceedings of CHI ’96, The SIGCHI Conference on Human Factors inComputing Systems: Common Ground (Vancouver, British Columbia, Canada, April13–18 1996). ACM, New York, NY, pp. 19–26. <http://doi.acm.org/ 10.1145/238386.238394>.

Buxton, W., 2001. Less is more (more or less). In: Denning, P. (Ed.), The Invisible Future: The seamless integration of technology in everyday life. McGraw Hill, New York, pp. 145–179.

 

Cairns, P., 2007. HCI. not as it should be: inferential statistics in HCI research. In: Ball, L., Sasse, M., Sas, C., et al. (Eds.), Proc. of HCI 2007, vol. 1, BCS, pp. 195–201.

Clark, H., 1996. Using Language. Cambridge University Press, Cambridge. Clark, H.,

Brennan, S., 1991. Grounding in communication. In: Resnick, L., Levine, J., Teasley, S. (Eds.), Perspectives on Socially Shared Cognition. American Psychological Association, Washington, pp. 127–149.

Damasio, A., 1994. Descartes’ Error: Emotion Reason and the Human Brain. Putnam
Publishing. paperback: Penguin, 2005.

Dawkins, R., 1976. Memes: the new replicators. In: The Selfish Gene. Oxford
University Press, London (Chapter 11).

De Certeau, M., 1984. The Practice of Everyday Life (trans. S. Rendall, University
ofCalifornia Press, Berkeley, 1984). L’invention du Quotidien, vol. 1, Arts de
Faire’, 1980 (in French).

Diaper, D., 1989. The discipline of human–computer interaction. Interacting with
Computers 1 (1), 3–5.

Diaper, D., Sanger, C., 2006. Tasks for and tasks in human–computer interaction.
Interacting with Computers 18 (1), 117–138. doi:10.1016/j.intcom.2005.06.004.

Dix, A., 1998. Hands across the screen – why scrollbars are on the right and other stories. Interfaces 37 (Spring), 19–22. <http://www.hcibook.com/alan/papers/
scrollbar/>.

Dix, A., 1998. Sinister scrollbar in the Xerox Star xplained. Interfaces 38 (Summer),
11 (short update to the above article) <http://www.hcibook.com/alan/papers/
scrollbar/scrollbar2.html>.

Dix, A., 1998. PopuNET: Pervasive, Permanent Access to the Internet. eBulletin,
aQtive Ltd. <http://www.hiraeth.com/alan/ebulletin/PopuNET/PopuNET.html>.

Dix, A., 1999. The Web Sharer Vision. eBulletin, aQtive Ltd., November 1999.
<http://www.hiraeth.com/alan/ebulletin/websharer/>.

Dix, A., 2002. Teaching innovation. Excellence in Education and Training
Convention, 17th May 2002. Singapore Polytechnic. <http://
www.hcibook.com/alan/talks/singapore2002/>.

Dix, A., 2003. Upside down As and algorithms – computational formalisms and
theory. In: Carroll, J. (Ed.), HCI Models Theories and Frameworks: Toward a Mulitdisciplinary Science. Morgan Kaufmann, San Francisco, pp. 381–429 (Chapter 14) <http://www.hcibook.com/alan/papers/theory-formal-2003>.

Dix, A., 2004. Controversy and provocation. In: Proceedings of HCIE2004. The 7thEducators Workshop: Effective Teaching and Training in HCI, 1st and2nd April 2004. Preston, UK. ISBN 0-9541927-5-3 <http://www.hcibook.com/alan/ papers/HCIE2004/>.

Dix, A., 2004. European HCI Theory – a uniquely disparate perspective. In: European HCI Research Special Area CHI 2004, Vienna, Austria, 24–29 April 2004. <http:// www.hcibook.com/alan/papers/chi2004-euro-theory/>.

Dix, A., 2008. Human–Computer Interaction in the Early 21st Century: a Stable Discipline, a Nascent Science, and the Growth of the Long Tail. SIGCHI Ireland Inaugural Lecture, 2nd December 2008. Trinity College, Dublin. <http:// www.hcibook.com/alan/talks/Dublin-2008/>.

Dix, A., 2008. Theoretical analysis and theory creation. In: Cairns, P., Cox, A. (Eds.), Research Methods for Human–Computer Interaction. Cambridge University Press (Chapter 9).

Dix, A., Finlay, J., Abowd, G., Beale, R., 2004. Interaction design basics. In: Human– Computer Interaction, third ed. Prentice-Hall (Chapter 5).

Dix, A., Ormerod, T., Twidale, M., Sas, C., Gomes da Silva, P., McKnight, L., 2006. Why bad ideasare a good idea? In: Proceedings of HCIEd.2006-1 Inventivity, Ballina/ Killaloe, Ireland, 23–24 March 2006. <http://www.hcibook.com/alan/papers/ HCIed2006-badideas/>.

Dix A., Gill S., Ramduny-Ellis D., Hare J., 2009. Design and physicality – towards an understanding of Physicality in design and use. In: Designing for the 21st Century: Interdisciplinary Methods & Findings, Gower.

Djajadiningrat, J., Gaver, W., Frens, J., 2000. Interaction relabelling and extreme characters: methods for exploring aesthetic interactions. In: Boyarski, D., Kellogg, W. (Eds.), Proceedings of DIS2000 Designing Interactive Systems: Processes, Practices Methods and Techniques (New York 17–19 August 2000). ACM Press, New York, pp. 66–71.

Dourish, P., 2006. Implications for design. In: Grinter, R., Rodden, T., Aoki, P., Cutrell, E., Jeffries, R., Olson, G. (Eds.), Proceedings of CHI ’06, The SIGCHI Conference on Human Factors in Computing Systems (Montreal, Quebec, Canada, April 22–27, 2006). ACM Press, New York, pp. 541–550. doi:10.1145/1124772.1124855.
Eisenberg, B., 2004. Debunking Miller’s Magic 7. ClickZ, October 29, 2004. <http:// www.clickz.com/3427631>.

Ellis, G., Dix, A., 2006. An explorative analysis of user evaluation studies in information visualisation. In: Proceedings of the 2006 Conference on Beyond Time and Errors: Novel Evaluation Methods For information Visualization (Venice, Italy, May 23–28, 2006). BELIV ’06. ACM Press, New York, pp. 1–7. <http://www.hcibook.com/alan/papers/beliv06-evaluation/>.
Furniss, D., Blandford, A., Curzon, P., 2007. Usability evaluation methods in practice: understanding the context in which they are embedded. Proceedings of the 14th European Conference on Cognitive Ergonomics: Invent! Explore! (London, August 28–31, 2007), ECCE ’07, vol. 250. ACM Press, New York, pp. 253–256. http://doi.acm.org/10.1145/1362550.1362602.

Gamberini, L., Spagnolli, A., Pretto, P., 2004. Temporal structure of SMSmediatedconversation. In: Time Design Workshop, CHI2004, Wien, April 25 2004.

Garfinkel, H., 1967. Studies in Ethnomethodology. Prentice-Hall, Englewood Cliffs. Gaver, W., 2007. Cultural commentators: non-native interpretations as resources forpolyphonic assessment. International Journal of Human–Computer Studies
65, 292–305.

Harper, R., Rodden, T., Rogers, Y., Sellen, A., 2008. Being Human: Human–Computer
Interaction in the Year 2020. Microsoft Research, Cambridge. <http://
research.microsoft.com/en-us/um/cambridge/projects/hci2020/>.

Johnson, J., Roberts, T., Verplank, W., Smith, D., Irby, C., Beard, M., Mackey, K., 1989. The Xerox Star: a retrospective. Computer 22 (9), 11–26. 28–29. <http://
dx.doi.Org/10.1109/2.35211>.

Larson, K., Czerwinski, M., 1998. Web page design: implications of memory,
structureand scent for information retrieval. In: Proceedings of CHI ’98 Human
Factors in Computing Systems. ACM Press, pp. 25–32.

Long, J., 1991. Theory in human–computer interaction?, IEE Colloquium on Theory
in Human–Computer Interaction (HCI) (Digest No192) 17 Dec 1991. IEE, London, pp. 2/1–2/6. <http://ieeexplore.ieee.org/stamp/ stamp.jsp?arnumber=241136&isnumber=6182>.

Long, J., 1996. Specifying relations between research and the design of human– computer interactions. International Journal of Human–Computer Studies 44, 875–920.

Long, J., Dowell, J., 1989. Conceptions of the discipline of HCI: craft, applied science, andengineering. In: Sutcliffe, A., Macaulay, L. (Eds.), Proceedings of the Fifth
Conference of the British Computer Society, Human–Computer Interaction SpecialistGroup on People and Computers V (Univ. of Nottingham). Cambridge University Press, New York, pp. 9–32.

Marty, P., Twidale, M., 2004. Lost in gallery space: a conceptual framework for analyzingthe usability flaws of museum Web sites. First Monday 9, 9.

Marty, P., Twidale, M., 2005a. Extreme Discount Usability Engineering. Technical Report ISRN UIUCLIS–2005/1+CSCW.

Marty, P., Twidale, M., 2005b. Usability@90 mph: Presenting and Evaluating a New. High-Speed Method for Demonstrating User Testing in Front of an Audience. First Monday 10, 7.

McCarthy, J., Wright, P., 2004. Technology as Experience. MIT Press, Cambridge.

Merleau-Ponty, M., 1945. M. Phénomènologie de la Perception. Gallimard, Paris (quote in text from: M. Merleau-Ponty, Phenomenology of Perception, K. Paul
(trans.), Routledge, 1962, 2002).

Miller, G., 1956. The magical number seven, plus or minus two: some limits on our
capacity for processing information. The Psychological Review 63, 81–97.
<http://www.musanim.com/millerl956/>.

Nielsen, J., 2000. Why You Only Need to Test With 5 Users, Alertbox, March 19,
2000. <http://www.useit.com/alertbox/20000319.html>.

Nielsen, J., Landauer, T., 1993. A mathematical model of the finding of usability
problems. In: Proceedings of the INTERACT ’93 and CHI ’93 Conference on Human Factors in Computing Systems (Amsterdam, The Netherlands, April 24–29,1993). ACM Press, New York, pp. 206–213. doi:10.1145/169059. 169166 .

O’Reilly, T., 2005. What Is Web 2.0: Design Patterns and Business Models for the NextGeneration of Software, O’Reilly Media, 30th September 2005. <http:// www.oreillynet.eom/pub/a/oreilly/tim/news/2005/09/30/what-is-web20.html> (accessed 23.06.07).

PHP Manual, 2009. A History of PHP. <http://gtk.php.net/manuall/en/html/ intro.whatis.php.history.html> (accessed 01.05.09).

Popper, K., 1959. The Logic of Scientific Discovery. Basic Books, New York.

Razak, F., 2008. Single Person Study: Methodological Issues, PhD Thesis. ComputingDepartment, Lancaster University, UK. <http://www.hcibook.net/
people/Fariza/>.

Rogers, Y., 2004. New theoretical approaches for HCI. Annual Review of Information
Science and Technology 38.

Rouet, J.-F., Ros, C., Jégou, G., Metta, S., 2003. Locating relevant categories in web menus:effects of menu structure, aging and task complexity. In: Harris, D., Duffy, V., Smith, M., Stephandis, C., (Eds.), Human-centred Computing: Cognitive Social and Ergonomic Aspects, vol. 3 of Proc. of HCI Intnl Laurence Earlbaum, New Jersey, pp. 547–551.
schraefel, m.c.,

Dix, A., 2009. Within bounds and between domains: reflecting on makingtea within the context of design elicitation methods. International Journal of Human–Computer Studies 67 (4), 313–323. April.

Shackel, B., 1959. Ergonomics for a computer. Design 120, 36–39. Silva, P., Dix, A., 2007. Usability – not as we know it! In: Proceedings of BCS HCI 2007, People and Computers XXI, BCS eWiC. <http://www.hcibook.com/alan/
papers/HCI2007-YouTube/>.

Spagnolli, A., Gamberini, L., 2007. Interacting via SMS: practises of social closeness andreciprocation. British Journal of Social Psychology 22, 343–364. Star, S., 1989. The structure of ill-structured solutions: boundary objects and heterogeneous distributed problem solving. In: Gasser, L., Huhns, M. (Eds.), Distributed Artificial Intelligence, vol. II. Morgan Kaufmann, SF Mateo, pp. 37– 54.

Sutcliffe, A., 2000. On the effective use and reuse of HCI knowledge. ACM
Transaction on Computer–Human Interaction 7 (2), 197–221. http://
doi.acm.org/10.1145/353485.353488.

Sykes, J., 2004. Presentation at panel on ”Funology: A Science of Enjoyable Technology?” M. Blyth (chair), ECCE-12, (St. Williams College, Yor, 12–15 September 2004).

Tidwell, J., 2005. Designing Interfaces: Patterns for Effective Interaction Design. O’Reilly.

Tidwell, J., 2009. Common Ground: A Pattern Language for Human–Computer
Interfacedesign. <http://www.mit.edu/~jtidwell/interaction_patterns.html>
(accessed May 2009).

van Welie, M., 2009. Welie.com: Patterns in Interaction Design, dated 2008. <http://
www.welie.com/> (accessed May2009).

White, L., 1968. Medieval Technology and Social Change. Oxford University Press.

 

 

John M. Carroll
Center for Human–Computer Interaction and College of Information Sciences and Technology, The Pennsylvania State University, University Park, PA 16802, USA

John Long's Comment 1 on this Paper

Abstract

This essay is a personal reflection on John Long’s keynote address at the BCS People and Computers meeting in Nottingham in the summer of 1989. I try to locate the paper’s purpose and significance within the history of human-computer interaction (HCI), both prior to 1989 and subsequently, and particularly with respect to the abiding questions of what sort of enterprise HCI is, and of what sorts of knowledge it uses and produces.

Comment 2

 

1. Introduction: HCI’s first three decades

Comment 3

The general position, here, is that, in agreement with Carroll, the HCI community has grown; but, in disagreement with him, the HCI discipline has not made desired progress.

Human–computer interaction (HCI) is an area of research and practice that emerged in the early 1980s, initially as a specialty area in computer science. HCI has expanded rapidly and steadily for three decades, attracting professionals from many other disciplines and incorporating diverse concepts and approaches. To a considerable extent, HCI now aggregates a collection of semi-distinct fields of research and practice in human-centered informatics. However, the continuing synthesis of disparate conceptions and approaches to science and practice in HCI has produced a dramatic example of how different epistemologies and paradigms can be reconciled and integrated.

Comment 4

 

Until the late 1970s, the only humans who interacted with computers were information technology professionals and dedicated hobbyists. This changed disruptively with the emergence of personal computing around 1980. Personal computing, including both personal software (productivity applications, such as text editors and spreadsheets, and interactive computer games) and personal computer platforms (operating systems, programming languages, and hardware), made everyone in the developed world a potential computer user, and vividly highlighted the deficiencies of computers with respect to usability for those who wanted to use computers as tools.

The challenge of personal computing became manifest at an opportune time. The broad project of cognitive science, which incorporated cognitive psychology, artificial intelligence, linguistics, cognitive anthropology, and the philosophy of mind, had formed at the end of the 1970s. Part of the programme of cognitive science was to articulate systematic and scientifically-informed applications to be known as ”cognitive engineering”. Thus, at just the point when personal computing presented the practical need for HCI, cognitive science presented people, concepts, skills, and a vision for addressing such needs. HCI was one of the first examples of cognitive engineering.

Other historically fortuitous developments contributed to the establishment of HCI. Software engineering, mired in unmanageable software complexity in the 1970s, was starting to focus on nonfunctional requirements, including usability and maintainability, and on non-linear software development processes that relied heavily on testing. Computer graphics and information retrieval had emerged in the 1970s, and rapidly came to recognize that interactive systems were the key to progressing beyond early achievements. All these threads of development in computer science pointed to the same conclusion: The way forward for computing entailed understanding and better empowering users.

Finally human factors engineering, which had developed many techniques for empirical analysis of human–system interactions in so-called control domains such as aviation and manufacturing, came to see HCI as a valuable and challenging domain in which human operators regularly exerted greater problem-solving discretion. These forces of need and opportunity converged around 1980, focusing a huge burst of human energy, and creating a highly visible interdisciplinary project.

One of the most significant achievements of HCI is its evolving model of the integration of science and practice. Initially this model was articulated as a reciprocal relation between cognitive science and cognitive engineering. Later, it ambitiously incorporated a diverse science foundation, notably Activity Theory, distributed cognition, and ethnomethodology, and a culturally embedded conception of human activity, including the activities of design and technology development. Currently, the model is incorporating design practices and research across a broad spectrum. In these developments, HCI provides a blueprint for a mutual relation between science and practice that is unprecedented.

Early HCI sought to develop synergies between cognitive science and cognitive engineering. During the 1980s a rich reciprocal relationship developed. In areas like user modeling, HCI directly applied key cognitive science theories to the design of command languages and information visualizations. In other cases, HCI provided guidance to cognitive science through embodied concepts like direct manipulation and user interface metaphor. Mutual reciprocity between underlying science and application is rare, but not unprecedented (the discovery of the transistor effect in physics emerged from applied research). For HCI, this starting point was an intellectually sophisticated and ambitious foundation for more radical possibilities.

HCI research and application provided a strong force for theoretical integration within cognitive science. The very first HCI theories were far more ambitious integrations than had been attempted in the basic science. For example, the model human processor (Card et al., 1983) integrated aspects of perception, attention, short-term memory operations, planning, and motor behavior in a single model, at a time when most cognitive science models addressed only isolated laboratory phenomena. Ironically, early models were criticized within HCI as too limited with respect to understanding and creating applications. This self-criticism promoted increasingly comprehensive modeling that has jointly driven the basic science and its applications. But more importantly, these early successes, and their deconstruction, further fueled paradigmatic aspirations in HCI.

In the latter 1980s and early 1990s, HCI assimilated ideas from Activity Theory, distributed cognition, and ethnomethodology. This comprised a fundamental epistemological realignment. For example, the representational theory of mind, a cornerstone of cognitive science, is no longer axiomatic for HCI science. Information processing psychology and laboratory user studies, once the kernel of HCI research, became important, but niche areas. Field studies became typical, and eventually dominant as an empirical paradigm. Collaborative interactions, that is, groups of people working together through and around computer systems (in contrast to the early 1980s user-at-PC situation) have become the default unit of analysis. The contemporary theory-base of HCI draws broadly upon social, cognitive and computation science, and strongly emphasizes design research, pragmatics, and aesthetics. It is remarkable that such fundamental realignments were so easily assimilated by the HCI community.

Comment 5

Although HCI was always conceived of as a design science, this was construed at first as a boundary, with HCI providing guidance to system design and development. Throughout the 1990s, however, HCI directly assimilated, and eventually itself spawned, a series of design communities. This engagement with design communities coincided with substantial advances in user interface technologies that shifted much of the potential proprietary value of user interfaces into graphical design. Somewhat ironically, designers were welcomed into the HCI community just in time to help remake it as a design discipline. A large part of this transformation was the creation of design disciplines that did not exist before. For example, user experience design and interaction design were not imported into HCI, but rather were among the first exports from HCI to the design world. Design is currently the facet of HCI in most rapid flux. It seems likely that more new design proto-disciplines will emerge from HCI during the next decade.

Conceptions of how underlying science informs and is informed by the worlds of practice and activity have evolved continually in HCI since its inception. Throughout the history of HCI, paradigmchanging scientific and epistemological revisions were deliberately embraced by a field that was, by any measure, succeeding intellectually and practically. The result has been an increasingly fragmented and complex field that has continued to succeed even more. This example contradicts the Kuhnian view of how intellectual projects develop through paradigms that are eventually overthrown.

Comment 6

 

The continuing success of the HCI community in moving its meta-project forward thus has profound implications, not only for human-centered informatics, but for epistemology. (Other sketches of the history of HCI are Carroll, 1997; Myers, 1998; Grudin, 2005.). In this paper, I will elaborate the foregoing interpretation of the emergence of HCI, particularly with respect to conceptions of HCI as a discipline, and of what sorts of theory are possible and appropriate in HCI. My touchstone for this is the contribution of John Long to the development of HCI, especially during the latter 1980s, an extremely formative period. I will place these contributions into a context, reconstructed through the benefits of hindsight, and the latitudes of an essay format. In my view, there is not one true narrative for HCI (or for anything else of even reasonable complexity). Nevertheless, I think my view is grounded and valid, and more importantly, it leads to prospective interpretations of HCI, and of Long’s contributions to it. We do not want to, and, in any case, cannot relive the history of HCI, but we surely ought to try to learn what we can from it.

2. Long’s conception of an engineering discipline of HCI

I met Long in 1989. We met at a conference in Nottingham, at which we were both invited speakers. We were both wrestling, in our separate ways, with what some have called the mid-1980s ”theory crisis” in HCI. As briefly sketched above, HCI had been born at the beginning of the 1980s as a paradigm case of cognitive engineering. But what exactly was cognitive engineering? Most early discussions were vague. I first encountered the programmatic notion of cognitive engineering in discussions at the inaugural meting of the Cognitive Science Society in La Jolla, California, and subsequently in a talk by Norman (1982) at the early HCI conference held at the US Bureau of Standards in Gaithersburg, Maryland. The kernel of the idea was that domains strongly shape cognition, and that studying and supporting cognition in real and complex domains is salutary, if not essential, for developing a science of cognition and, of course, for applying it to real problems.

Comment 7

 

There is no doubt that this conception was transformative with respect to a range of cognitive science activities in the 1980s and subsequently, among them HCI. But likewise there is no doubt that this conception leaves out many critical details. For example, what does it mean to apply cognitive science? How exactly would that work? It would necessarily involve generalizing from principles and results that originated in narrow and contrived laboratory tasks. There would have to be some perilous inductive leaps, at the least. And even if some of the leaps worked out, would applying cognitive science in some number of cases be enough to warrant the claim that cognitive engineering had arrived?

Comment 8

HCI in the 1980s was strongly invested in the cognitive engineering vision, though for the most part the larger framework remained unarticulated. Card et al. (1983), and more pointedly Newell and Card (1985), made the most comprehensive early effort toward articulating a framework for science and engineering in HCI. Their work embraced a simplistic view of the relation between science and engineering, emphasizing approximation but not boundary conditions on or scope of applicability, and it did not produce the paradigmatic consensus they had hoped for, indeed it evoked sharp criticism (Carroll, 2006; Carroll and Campbell, 1986).

Comment 9

 

But the Card, Moran and Newell work did provide a touchstone and focus for other theory-based work during the 1980s; it clearly raised the possibility of a theory-based paradigm, and it attracted many other voices to the debate.
When I met Long, we were both engaged in trying to work out what kind of a discipline or project HCI was, or could be seen as, and what kinds of knowledge or theory it used or could use. We were trying to describe how cognitive science knowledge might emerge from and be applied to HCI design work.

Comment 10

We were pursuing different answers, the nature of which I will address presently.

2.1. Defining the HCI discipline

Long’s keynote address at the Nottingham conference, the paper was written with John Dowell, contrasted three conceptions of HCI as a discipline – craft, applied science, and engineering (Long and Dowell, 1989). Long and Dowell define discipline as a particular body of knowledge supporting specific practices directed at solving a general problem. I like very much the approach of trying to be explicit as possible about definitions. This work bluntly proposes a boldly minimalist schematization of discipline.

Comment 11

Long and Dowell define the general HCI problem as ”the design of humans and computers interacting to perform work effectively” (p. 13). They characterize and analyze three disciplinary conceptions of HCI. First, they suggest that HCI can be pursued as a craft practice, relying on implicit, informal, and experiential knowledge. They discuss the example projects of Prestel videotex (Buckley, 1989) and of the Ded display editor (Bornat and Thimbleby, 1989) as paradigmatic. They argue that a craft practice of HCI cannot be effective: Because its knowledge is implicit, informal, and experiential, its knowledge cannot be operationalized: ”it cannot be directly applied by those who are not associated with the generation of the heuristics or exposed to their use” (p. 18). Moreover, because craft knowledge is heuristic, ”there is no guarantee that practice applying HCI craft knowledge will have the consequences intended” (p. 119). In other words, craft practice is ineffable and unreliable.

Comment 12

Second, they consider a disciplinary view of HCI as applied science, relying on knowledge in the form of theories, models, and principles used to formulate and investigate hypotheses, predictions, and explanations. They concede that sciences, like psychology, can be applied to HCI, giving the example of the role of confirmatory feedback in guiding sequences of behavior. However, they argue, such general scientific principles cannot be directly and deductively applied in a specific design because they do not ”prescribe the feedback required . . . to achieve effective performance of work” (p. 20). In other words, the context in which the scientific principles were formulated and developed are necessarily different than the ones that arise for a particular design application, and specifically so with respect to supporting effective work outcomes in the context of the design.

Comment 13

Long and Dowell also consider the related strategy of constructing guidelines/principles which are themselves grounded in scientific theory. They discuss Hammond and Allinson’s (1988) computer assisted informal learning system with respect to the theory-based design principle ”provide distinctive multiple forms of representation” as an example. Long and Dowell argue that such a principle cannot directly guide design, since neither it nor the theories that underwrite it are defined, operationalized, or generalized with respect to effective performance of work activity.

More generally, Long and Dowell argue that although HCI as an applied science describes knowledge more explicitly and more generally, and supports derivation of theory-based guidelines, it ultimately fails in the same way craft practice fails as a disciplinary model for HCI: applied science does not describe how to support particular work activity effectively. As a consequence, the use of science in design must always be empirically mediated by implementation, evaluation, and iteration.

Long and Dowell turn finally to their preferred disciplinary model for HCI: engineering. They state that engineering distinctively solves design problems ”by the specification of designs before their implementation” (p. 24). They describe engineering knowledge as principles that allow ”designs to be prescriptively specified for artifacts, or systems which when implemented, demonstrate a prescribed and assured performance” (p. 24). Finally, they state that engineering can deal systematically with complexity: ”Designs specified at a general level of description may be systematically decomposed until their specification is possible at a level of description of their complete implementation” (p. 24).
Long and Dowell are optimistic about the engineering conception, but provide few details. Indeed, they concede that engineering principles of the sort they require do not exist (as of 1989). They give two examples that they consider promising: Dix and Harrison (1987) and Dowell and Long (1989). Curiously, they do not mention the engineering models of Card et al. (1983). Further, they suggest – actually quite like Card et al. (1983) – that the most promising niches for this disciplinary model to be realized would be in highly practiced, expert performance of relatively low-level tasks domains in which ”human behavior can be usefully deterministic” (p. 27).

Comment 14

One of the key constructs that differentiates the engineering disciplinary model of HCI from the applied science and craft models is that the former incorporates an explicit model of the application domain. The craft practice and applied science disciplinary models have no notion of boundary conditions, applicability, or context built into them. Nevertheless, Long and Dowell are confident that engineering principles of the sort they imagine for HCI would be generalizable knowledge, that application of the principles would be direct and indeed specifiable, and effective.

Comment 15

2.2. Problems with Long and Dowell’s conception of discipline

The best thing about the Long and Dowell paper is that it clearly and forcefully sets forth definitions, makes sharp distinctions, and reasons toward a clear programmatic conclusion and recommendation for how HCI ought to organize itself as a discipline. I will return to this point in closing, but let me say now that I regard all that as very important and constructive. HCI is still much in need of clarifying its foundations, and making progress on that will require definitions, distinctions, and argumentation. I believe that Long and Dowell’s paper provides intellectual scaffolding to construct such a debate. Although, as I will presently make clearer, I feel that the paper did not achieve it goals, I think the goals are valuable and that the general versions of the questions raised are still quite open to debate and in need of investigation.

Comment 16

Long and Dowell’s specific argument, however, fails in important ways. First, its general conception of discipline is unwieldy, and the authors do little to ease this concern. They present their conception of discipline in the most general terms, and then hurry onwards without actually describing the HCI discipline in any empirical detail. This omission matters, because the second failure of their paper is that the scoping of HCI they presume is more narrow than the reality of HCI. Moreover, the manifold ways in which HCI has broadened since 1989 make it ever more difficult to see how to extend the Long and Dowell analysis to contemporary HCI. Third, the characterizations and arguments directed at the craft and applied science models for an HCI discipline are dismissive and inadequate. They just do not make the case that these paradigms are inappropriate or necessarily ineffective. Finally, the argument for an engineering disciplinary model of HCI rests on an academic idealization of what engineering is like. No wonder they could not find examples of it.

Comment 17

As is often true, the rub comes with unpacking the details. Surely disciplines codify and use knowledge through practices addressing disciplinary problems. So one would have to say far more to have said anything. For example, we might assume that ”practices” is a union of typical and/or critical workflows, information flows, roles and divisions of labor, and other social arrangements in and around work activity. Indeed, this seems a rather minimal notion of practices. But even this is more than encyclopedic in scope. Such an amorphous and expansive conception of practices is a burden for all would be framework makers, and not just for Long and Dowell. However, the challenge Long and Dowell more uniquely inflict on themselves is that they insist that the job of HCI is to render practices in an explicit specification. They do not address this explicitly, but it seems to me that they would have to require that the practices be made explicit through some kind of hierarchical task analysis. In 1989, this would have been seen as reasonable, perhaps even as too obvious to belabor. But this is no longer true. Comprehensive task analysis, outside of restricted safety critical interactions, is moribund (Carroll, 2002). Even its most enthusiastic proponents concede that it is not used (Diaper, 2002).

Comment 18

More pointedly, such a conception of practices rendered explicit through tedious recursive decomposition is empirically wrongheaded. Studies of technical practices in general (Latour, 1987; Latour and Woolgar, 1986; Orr 1996) and of practices in HCI settings specifically (Bentley et al., 1992; Heath and Luff, 2000; Suchman, 1987) refute the possibility that actual domain practices could ever be meaningfully specified in this manner. Practices cannot be specified a priori because at any reasonable level of complexity they depend on the improvisations of people and the local culture of groups. Neither of these can be analyzed a priori or generally. Any programme that requires this level of specification has stumbled on the starting blocks.

Comment 19

Finally, it is important to emphasize that discarding brittle and a priori notions of discipline, practice, and even knowledge does not leave us paradigmatically impaired. Quite to the contrary, empirical approaches have already taught us much. If we want to develop a set of models for disciplines, we should study what scientists, engineers, designers, and other technical persons do, and how they do it (Latour, 1987; Latour and Woolgar, 1986). It is true that empirical approaches do not easily lead to simple type contrasts, such as Long and Dowell’s three paradigms for disciplinary projects, but they do produce exemplars that can be used as models.

Comment 20

Finally, it is important to emphasize that discarding brittle and a priori notions of discipline, practice, and even knowledge does not leave us paradigmatically impaired. Quite to the contrary, empirical approaches have already taught us much. If we want to develop a set of models for disciplines, we should study what scientists, engineers, designers, and other technical persons do, and how they do it (Latour, 1987; Latour and Woolgar, 1986). It is true that empirical approaches do not easily lead to simple type contrasts, such as Long and Dowell’s three paradigms for disciplinary projects, but they do produce exemplars that can be used as models.

Comment 21

2.3. Problems with Long and Dowell’s scoping of HCI

Long and Dowell spend quite a bit of discussion on the importance of identifying general disciplinary problems. They assert that each discipline has one, though it may be decomposable into subproblems with corresponding sub-disciplines. The general problem of HCI is stated as ”humans and computers interacting to perform work effectively” (p. 13). In their text, it is clear that Long and Dowell recognized that this was a very general general problem, essentially a cross-product of all topics involving humans and their organizations, computers – including embedded and networked devices, and all work activities in which the former employ the latter.

Comment 22

Long and Dowell explicitly identify HCI as a design domain. They give an alternate wording of their own general HCI problem highlighting this: ”the design of humans and computers interacting to perform work effectively” (p. 13). The general problem is
decomposed into ”the design of humans interacting with computers” and ”the design of computers interacting with humans”; they associate the former general problem with Human Factors or Ergonomics, and the latter with Software Engineering. All of these restatements and decompositions of the general problem focus on the effective performance of work activity; the phrase ”to perform work effectively” ends each version of the general HCI problem.

Long and Dowell’s conception, sweeping though it may be, is far too narrow. HCI in the reality of its practice addresses many activities that are not work. Indeed, it is rarely pointed out, but obviously true, that the importance of play, leisure, education, and myriad other non-work activities to HCI undermines many of the 1980s conceptions of HCI that came from ergonomics and human factors, but also from Activity Theory, work psychology, and other continental frameworks that are singularly focused on the workplace. Work is important, but all work and no play would have made HCI a far more dull enterprise. Fortunately, it didn’t turn out that way.

Comment 23

In their scoping of HCI, Long and Dowell seem to over focus on a particular methodological challenge of the 1980s. During this early period, designers and developers were often construed as customers or recipients of HCI methods and techniques. And it was generally believed that these counterparts wanted and needed HCI methods that were so well specified that they could be effectively put into practice by people not trained or experienced in HCI itself, namely, the designers and developers. This kind of boundary–mediated relationship still exists in some consulting arrangements, but it is no longer an appropriate view of HCI or its goals. Some of these boundaries have dissolved as HCI has come to directly include a greater diversity of professionals, and to produce professionals with more diverse skills. But perhaps more fundamentally, the goal of codifying formal methods that could be applied ”knowledgefree” to crank out good systems is misguided. We return to this point below.

Comment 24 The issue of rescoping the general problem of HCI to include non-work activity has a more specific problematic consequence for Long and Dowell. Once we broaden the definition of the general problem of HCI – appropriately – to include play, leisure, education, and so on, the very important qualifier effectively becomes much more difficult to understand. Yet, as we will see, Long’s conception of HCI, and his position regarding the question of what approaches to HCI could be appropriate, had everything to do with understanding and operationalizing effectiveness.

Comment 25

2.4. Problems with Long and Dowell’s conception of craft

The defining characteristic of a craft discipline is that craft knowledge is implicit, informal and acquired from experience. Long and Dowell did not give an example of traditional craft practice, but this would have been useful. Craft practice is the most developed paradigm for technology development in human history and there are many examples of it.

Comment 26 An excellent example is George Sturt’s (1923) book ”The Wheelwright’s Shop”. Sturt inherited a shop, and decided to learn why the wheelwrights made wheels and carts as they did. One of the design features for which he sought rationale was that traditional English carts have slightly bowl-shaped wheels, mounted so that the portion below the axel is perpendicular to the ground. Sturt was surprised to find that he got several different answers from his expert wheelwrights. He was told by various master wheelwrights that such wheels better accommodate the cooling of iron tires, that they have a smaller turning radius, that they better tolerate sideward swaying of a cart, and that they allow the cart body to be wider at the top, and thereby allow larger loads. This example illustrates how implicit, informal, experiential knowledge can guide a practice.

Comment 27

Long and Dowell take a skeptically rationalist view that such codification of knowledge is problematic, perhaps because it cannot be conveyed in written form. But it is arguable that this is a superior form of technical practice. Sturt’s account suggests how multiple converging rationales might make practices more robust as they are passed from expert to apprentice. The wheelwright craft that Sturt investigated had been maintained successfully for more than century. It was manifestly effective.

Comment 28

Another perspective on alternative HCI paradigms is to ask what sorts of specific affordances a craft practice entails that applied science and engineering do not. One answer is that knowledge is objectified less and proceduralized more in a craft practice. Sturt’s wheelwrights had one another; they worked collectively. They were less in need of engineering standards or explicit theories and guidelines. Their practice had evolved through generations to a level of design refinement not considered by Long and Dowell. The features of the cart wheels could not be derived linearly from single guidelines or sources of rationale, they were over-determined.

Comment 29

Indeed, this line of thinking seems more relevant than ever to the contemporary configuration of HCI. Many more designers participate in HCI than did in the late 1980s. The paradigm of design work is not evolutionary like wagon wheels, but it is a craft practice. It is true that design is typically taught in a studio paradigm, through participation and enactment as opposed to lecture and discussion. And it is true that design knowledge consists in heuristic concepts and techniques rather than deductive principles and laws. Long and Dowell take this as a crippling epistemological limitation, but they argue from an a priori conception of knowledge and the use of knowledge that requires specification and logical derivation.

Comment 30

Such a requirement would only make any sense if we could be assured that such an option exists. Although Long and Dowell hope it might exist, they are unable to give even one example.

Comment 31

And in any case, even if we could define a positivistic programme for design along these lines, why would we want to, given that a successful paradigm for the design profession already exists and is already contributing broadly to HCI design?

Comment 32

I will suggest later that it might make sense to take cognizance of actual HCI design practices in conceptualizing models for a discipline of HCI.

Long and Dowell’s specific critical analysis of craft is directed most specifically at Bornat and Thimbleby’s (1989) development of the early display editor Ded. In contemporary terminology, Bornat and Thimbleby employed evolutionary prototyping, a method in which designers create a running system and then successively revise their design to respond to user experience. In hindsight, this example is unfortunate with respect to Long and Dowell’s case that craft practice cannot produce generalizable knowledge. For indeed, Ded is an early instance of a design paradigm for text editors that became utterly pervasive throughout the world. It would be difficult to find a better example of design knowledge that proved generalizable, applicable, and effective.

Comment 33

Long and Dowell assume that if a system design is iteratively developed that ipso facto it can only make use of implicit, informal, and experiential knowledge, and cannot be based at all on explicit science or principles. This is bizarre, and all the more bizarre because Bornat and Thimbleby clearly label some of the ideas they articulated and refined in this project as ”theories”. We should take them at their word rather than insist, with Long and Dowell, on a false dichotomy between designs that have an iterative process and designs that embody explicit general knowledge (or for that matter heuristic knowledge). This dichotomy is not consistent with design practice. Indeed, it sets an impossibly high standard for the successful use of knowledge in design, namely that the knowledge must be applied a priori through logical derivation and never be wrong, never need to be adjusted. It is important to keep the severity of this standard in mind because it seems likely that no use of knowledge in the history of HCI, and perhaps any complex design domain, has ever attained this standard.

Comment 34

2.5. Problems with Long and Dowell’s conception of applied science

Long and Dowell argue that the general problem of scientific disciplines is to predict and explain phenomena, not to specify designs that support working effectively. Thus, scientific knowledge can help us predict and explain, but not prescribe designs. This seems an unwarrantedly limited view of the bounds of knowledge and creativity.

Comment 35

If by prescribing designs, Long and Dowell really do mean logically derive designs, then one would have to say that scientific knowledge also cannot serve prediction or explanation, since these applications of scientific knowledge, even safely restricted to basic science discourses, are invariably creative, and not purely mechanical endeavors.

Comment 36

Moreover, as I will emphasize in the immediately following section, there is no known way to prescribe designs in this limited sense, thus failing to prescribe designs must be seen as a vacuous failure.

I am of course aware that the ascription of positivism is now regarded as discourteous, but I do think that Long and Dowell are venturing into the hoary traditions of positivism. Applying knowledge is frequently a creative endeavor, in science or anywhere else. Knowledge does not come with rules of application, rather these are argued for and constructed as knowledge is put into use. In contemporary epistemology, we alter our conceptions about knowledge and its application when confronted with insights and successes. We should not turn away from insights and successes because they fail to follow a priori rules. We should alter the rules.

Comment 37

In their particular analysis of Hammond and Allinson’s theorybased design of a computer-aided learning system, Long and Dowell rather freely admit that the sophisticated use of psychological theory in the design ”might have been expected to modify learning behavior towards that of the easier recall of materials” (p. 22). This seems to contradict their general stance that science cannot prescribe designs, but nevertheless I agree with them. However, they go on to make an interesting distinction. They say that the theories Hammond and Allinson appealed to do not directly ”address the problem of the design of effective learning” (p. 22), and that resultingly Hammond and Allinson’s design might support more effective recall (the specific consequence that the theories did address), but still fail to support effective learning. This is cutting things pretty finely. Most psychologists would take better recall to be a learning achievement, though it is true that better recall is not necessarily indicative of better learning in every sense, or more specifically, of effective learning with respect to particular criteria or learning objectives. Accordingly, Long and Dowell conclude, Hammond and Allinson’s design work would necessarily have to progress via prototyping, evaluation, and iteration.

Comment 38

This argument is exceedingly peculiar. By granting that the theory-based design approach could reasonably expect to realize certain specific consequences for users (enhanced recall), Long and Dowell are essentially granting exactly what Hammond and Allinson claimed. Namely, they are granting that science can be applied in design; that designs can embody principles derived from scientific theory, and that consequences for users of the design can be anticipated from the theory. What they are balking at is the claim that scientific theory could completely specify a design, including all of its detailed consequences for users (cf. ”the design of effective learning”), a rather bold claim that, to my knowledge, no one ever made.

The shortcoming of Hammond and Allinson’s theory-based approach, namely, not being able to precisely specify the design of effective learning, entails that they must augment theory-based guidance with direct empirical approaches – prototyping, evaluation and iteration. But this is precisely how theory-based design in HCI has always worked (Johnson et al., 1989). Given that there is no way to prescribe designs, and that all design of any complexity must be empirical in just this sense, then Long and Dowell’s critique of the disciplinary model of applied science vis-à-vis HCI design is an argument about a straw man.

Comment 39

2.6. Problems with Long and Dowell’s conception of engineering

A touchstone example of software engineering in the real world is Brooks’s (1975) ”Mythical Man–Month”, essays written primarily about one of the largest software engineering projects in history, IBM Operating System 360. On Long and Dowell’s view of engineering we might expect to read about how the OS 360 design was successively decomposed until its subsystems could be completely specified, about how the system was explicitly and completely specified before it was implemented, and about how engineering principles were used to ”prescribe and assure” its performance before it was designed and implemented. But as everyone knows, the design of OS 360 did not work like that.

Comment 40

The complex and iterative nature of actual engineering processes is over-determined, as Brooks describes and as many subsequent studies have elaborated. For example, in a top–down framework, requirements guide the development of early designs. However, in practice, early designs typically cause requirements to be further developed, altered, and even abandoned. Design also helps to identify new requirements that were not part of the original project mandate, but which may be quite essential once they are identified. Brooks emphasizes that these iterative relationships obtain not merely because requirements happened not to be noticed, but because they cannot be identified until the early design enables their discovery. Today, none of this is shocking. Linear waterfall models of design and development have been succeeded by models including feedback and iteration.

Comment 41 Again turning the clock back to the mid-1980s, the apparent disconnect between Long and Dowell’s view of engineering, and Brooks’ case study of engineering practice in a large software development project, becomes more comprehensible. Early methodological conceptions of HCI were frequently oriented to system development processes that had traditionally overlooked consideration of users and other human stakeholders. Ironically, because HCI was not integrated into these processes, and often had no standing in the organizations practicing these processes, HCI methods were developed with an inadequate understanding of their ultimate context of use. Brooks’ view of OS 360 is an insider’s view, and in many respects an exposé. Part of the reason that Brooks’ book is a classic is that when published it was so iconoclastic, refuting core conceptions about software engineering. Long and Dowell were addressing engineering from an external view. They were trying to conceptualize an HCI discipline that could effectively contribute to system engineering processes as officially described.

Comment 42

There is, of course, an engineering paradigm for HCI. It developed in the 1990s, and it is interesting to observe how it differs from the conception of Long and Dowell. Usability engineering is one of the core clusters of method and process in HCI (Nielsen, 1994; Rosson and Carroll, 2002). Notably the sense of engineering in this stream of activity is systematic but it is fundamentally empirical. Its primary focus is methods to directly involve users via participatory design and analysis, and to assess user experiences through surveys and interviews, many kinds of scenario exercises, and direct evaluations, including thinking aloud studies, throughout every stage of the system development process, from
requirements identification through to documentation design. It uses models, for example like GOMS, but in relatively limited ways.

Comment 43

3. A hundred flowers that blossomed

In closing their paper, Long and Dowell (1989) considered whether craft practice, applied science and engineering could function together as a mutually supportive ensemble of disciplinary models. They point out that the three paradigms use and produce knowledge and results that are not always mutually intelligible. However, they counter-argue that the three paradigms together would better exploit what is known and what is practiced in HCI, and that integrating the three disciplinary models could encourage an HCI community ”superordinate to any single discipline conception” (p. 30). While I have tried to scrutinize many of the technical points and arguments in the paper, I think Long and Dowell ultimately were led to a larger truth, one that is strongly evidenced in the HCI we see today.

Comment 44 During the 20 years since Nottingham, HCI has changed in many ways. The lens of the Long and Dowell paper highlights three vectors of change that are intriguing and challenging, and that help to sound echoes of the Long and Dowell paper. First, it has become ever clearer that craft is the primary source of innovation in HCI. The primary role of science in HCI is to help us understand these innovations after they occur. Second, applied science in HCI provides explicit foundation for engineering models. These first two points emphasize how we need to link up the disciplinary models for HCI, instead of examining and evaluating them separately as competitive paradigms.

Comment 45

 

Third, the science foundation for HCI is incredibly rich and fragmented, more than anyone expected in the mid-1980s, and perhaps with more to come. We live in a time when the nature of science itself has been deeply questioned. Some of the debates occurring now in HCI with regard to its proper footing in science make the mid-1980s theory crisis seem mild indeed.

Comment 46

3.1. Craft innovations drive HCI science

The original vision for HCI was that cognitive science theory would produce or guide cognitive engineering of better systems. This programme was affirmed and pursued zealously throughout the 1980s, and through to the present. Good examples of this paradigm do exist, I believe. The Hammond and Allinson paper that Long and Dowell deconstructed is a wonderful example. However, it is also easy to be skeptical of ”science based design” as a general disciplinary model for HCI. Long and Dowell were articulating that skepticism.

Comment 47

Throughout the history of HCI the truly game-changing innovations have tended to be craft based. The pivotal design concept of direct manipulation, and the early scientific accounts of it are a case in point (Shneiderman, 1983; Hutchins et al., 1985). Direct manipulation is the style of computer interaction in which a person manipulates data and functions through gestures with display objects, for example, pointing and clicking in windows with a mouse — as contrasted with referring to data and functions by name in typed command strings. The principle object of these analyses, the point-and-click graphical user interface was developed years before the original scientific accounts, and indeed, direct manipulation is still being theorized and further developed (e.g., Plouznikoff et al., 2005). The graphical user interface as a design concept was developed pretty much through craft innovations during the decade from the mid-1960s through the mid-1970s (e.g., Buxton et al., 2005; Myers, 1998).

Comment 48

It is interesting that even the early theories of direct manipulation emerged long after the technology innovation itself. Thus, far from determining or even guiding the technology development and user interface design, the theories served the purpose of interpreting, consolidating, and abstracting the lessons from craft innovation to help move HCI research and development work forward in a more explicit and deliberate manner. This is very valuable; it allows for an engineering practice to be codified from the more implicit craftwork.

Comment 49

I describe the direct manipulation example because it is and was so central to the development of HCI. However, similar patterns can be seen in the development of HCI science and theory for other key concepts and techniques in HCI design. For example, Blackwell (2006) provides a vivid exegesis of how craft and science have shaped the use of metaphor in HCI through the course of three decades. The history of HCI science is one of technology innovations mysteriously popping out of craftwork, and then eventually being noticed, analyzed, and codified in models and theories.

Comment 50

3.2. Applied science provides explicit foundation for engineering models

The most ambitious articulation of the relationship between cognitive science and its application is that of a reciprocal relationship at the level of models (Norman, 1982). This is to make a distinction between specific (one-off) design applications of cognitive science, like the Hammond and Allinson computer assisted learning system, and systemic applications in which the science is a foundation for an engineering model that can be more generally applied. Card et al. (1983) development the Model Human Processor (MHP) and Goals, Operators, Methods and Selection rules (GOMS) model for analyzing routine human–computer interactions is a very early example of applied cognitive science theory in HCI that also provided direct guidance for a set of engineering models.

Comment 51

As I briefly noted above, these models were somewhat narrowly scoped, but seen in the context of cognitive science ca. 1980, this was some of the most comprehensive applied science work ever attempted. The model explicitly integrated many components of skilled performance – perception, attention, short-term memory operations, planning, and motor behavior – to produce predictions about expert performance in real tasks.

The MHP/GOMS model, taken as an engineering model, was an advance over prior human factors approaches in that it explicitly described the cognitive structures underlying manifest behavior. In other words, it was an engineering model directly and explicitly grounded in scientific understanding of information processing psychology. But as a scientific account, this model was a huge step forward also: Cognitive science models and theories up to that point had not attempted such a level of integration. The comprehensiveness of these early models vis-à-vis cognitive science can be seen as directly caused by their purpose vis-à-vis cognitive engineering. In order to generate detailed quantitative predictions about user performance in realistic contexts, HCI models must make explicit assumptions about a wide range of human characteristics.

Comment 52

 

Not all engineering models in HCI are as focused and narrow as MHP/GOMS. For example, the various developments of usability engineering could be considered a collection of engineering models. The usability engineering textbook I wrote with Rosson (Rosson and Carroll, 2002) quite explicitly presents a system development lifecycle engineering model, a wide range of systematic methods that usability engineers can follow to better assure that their designs are useful and usable to people. Our conception of engineering is heavily influenced by Brooks; we emphasize many approaches to prototyping, and emphasize throughout the book that one of the primary mistakes to avoid is premature commitment: thinking that the first passably-acceptable solution generated is the stopping point for design. If I were to write that book today I would significantly broaden its treatment of nuances of quality in the user experience. We emphasized satisfaction, but qualities like fun and engagement deserve more consideration. Of course this would make usability engineering even more dependent on applied science, and even more empirical.

Comment 53

Just recently, the ACM SIGCHI Symposium on Engineering Interactive Computing Systems (http://eics-conference.org/2009/) was launched, but again the notion of engineering in this conference series is far broader than that of mechanical derivation of solutions with predetermined properties.

Comment 54

3.3. The science foundation for HCI is incredibly rich and fragmented

The science foundation of HCI in the early 1980s was cognitive science, chiefly cognitive psychology. But this quickly changed. Even by the end of the 1980s, many other approaches with roots in social psychology, sociology and anthropology were moving to the center of HCI. A technological reason for this was that collaborative systems were emerging and raised many questions that could not be articulated in a cognitive paradigm. More importantly, HCI was reaching out to understanding technology in use, and actually contexts of use nearly always involved multiple people, work practices, organizational structures and myriad factors beyond the purview of individual cognition.

Suchman’s (1987) study of photocopier use was iconic. She described a variety of usability problems with advanced photocopier user interfaces. The problems she identified were fairly typical of HCI studies of the time (Carroll, 2003). But her approach and analysis were distinct in important ways. First, she studied people doing real work in a workplace context, not in a laboratory setting, as was typical at the time. Indeed, many of her participants were scientists at Xerox PARC going about their research work, and occasionally struggling to make copies. Second, she analyzed the interaction between the person and the machine as a sort of conversation that can fail when the actions of the participants are not intelligible to one another. This raised the level of analysis to that of human agency, as opposed to operating characteristics of the mind-as-a-computer (memory limitation, incorrect rules, etc.). Third, she directed her analysis to very fundamental issues in cognitive science. Thus, based on the amount of creative improvisation she observed in people trying to fathom and use photocopiers, she concluded that the concept of plans as causal accounts of human action was fundamentally flawed. Plans might be resources for action, but could not determine action in circumstances of any significant complexity.

Comment 55

By the end of the 1980s, HCI had become an international research community. Threads of work that had been going on in the United States and in Europe were increasingly brought together through Europeans visiting the mostly-American Association for Computing Machinery (ACM) CHI Conference and Americans visiting the mostly-European International Federation of Information Processing (IFIP) INTERACT Conference. For example, Bjerknes et al. (1987) published their collection on participatory design. Although they do argue that involving users directly in the inner sanctums of design deliberation is technically effective, the primary theme and argument in their book links participatory design to democracy, and asserts that sharing power with users in the design process is a better moral choice. Issues of self-determination in the workplace and power sharing and participation in software design cannot be articulated in a purely cognitive HCI.

Comment 56

These developments contributed to a scientific foundation far more rich, far more diverse than the starting points of the early 1980s. By the end of the 1990s, HCI looked quite different. Social psychology concepts like production blocking, conformity, social loafing, and social pressure had become as commonplace as memory capacity, consistency, and index of difficulty in the 1980s.

Activity theory and distributed cognition were established paradigms in theory, in many ways eclipsing information processing psychology as the establishment in theory. Ethnographical fieldwork had become a methodological touchstone for understanding usability.

Comment 57

HCI theory during this period was highly successful, in the sense of producing explanations and principled descriptions of human– computer interaction contexts (themselves largely produced through craft innovations) that have had great impact on cognitive science. Just as the MHP/GOMS model had led cognitive science in the 1980s, Suchman’s analysis of situated actions, distributed cognition and activity theoretic models, and studies of computer-mediated collaboration had substantial influence throughout cognitive science. For example, in 1993 a special issue of Cognitive Science, the field’s flagship journal, was addressed to reconsideration of Suchman’s 1987 book on the field of cognitive science. Similarly, a special issue of the Journal of the Learning Sciences, the flagship cognitive science journal in learning, was directed to reflection on Suchman’s contribution in 2006.

Comment 58

 

3.4. Design rationale as theory: the task-artifact framework

My own contribution to the Nottingham conference was more similar to Long’s than I realized at the time. Like Long, I also sought to formulate a programme for HCI.

Comment 59

Like Long, I had concluded that HCI could not comprehensively be constructed as applied cognitive science. In my paper (Carroll, 1989), I suggested that the most effective role for science in HCI design might be to interpret designs-in-use, to codify the knowledge implicit in designs so that it could be used more explicitly in future designs. I suggested that bringing this interpretative work into the design process itself might be the closest we could get to theory-based design. I saw this as augmenting the paradigm of craft practice to demystify the how and why, quite analogous to what George Sturt was trying to with the craft of wheelwrights.

Comment 60

Much like Sturt, I suggested that designed artifacts ought to ”read” as theories, that system design and development outcomes should be directly leveraged as knowledge outcomes (Carroll and Campbell, 1989). Implemented designs have nice properties, considered as knowledge outcomes: they are precise and complete; that is, they are complete enough to run and do whatever it is that they do, and they cannot leave things vague or make unrealistic simplifying assumptions the way a discursive theory can.

Comment 61

Designs must take a stand on every issue they encounter. Designs seamlessly integrate ideas from many sources and from many levels of analysis; that is, a design may embody ideas from GOMS with respect to keystroke-level interactions, ideas from Activity Theory about leveraging cultural practices in new work designs, and ideas from social psychology about how collaborators can quickly come to trust one another. Discursive theories are notoriously bad at spanning levels of analysis; indeed, many take it as given that levels of analysis can never be spanned. Finally, designs are also unavoidably testable, that is, when people use them, their use generates consequences, vivid, concrete and often poignant.

The main thing that makes designs poor as theories is that they are difficult to read. The propositions of an artifact’s theory are implicit, after all; they must be constructed by an analyst. And there’s the rub. How can we identify the theory that is implicit in an implemented design? My answer was design rationale, the documentation traditionally generated in the design process describing the issues, decisions, choices, and consequences that were considered, pursued, abandoned, and/or implemented. Of course, because design rationale is documentation, it is often viewed as tedious, boring, and bureaucratic. Moreover, because most designs ultimately fail in one way or another, creating an explicit and thorough design rationale is analogous to carefully leaving your fingerprints all over a spot you know will most likely be a crime scene.

Comment 62

In Longian terms, I wanted to imagine ways to better integrate HCI craft practices with applied social and cognitive science so as to do the least violence to the manifest effectiveness of HCI craft practices, but at the same time help those practices to be more deliberative, more auditable, and more manageable.

Comment 63

The key criterion for me was intelligibility to the practices and values of HCI designers. Thus, I eventually built my own methodological prescriptions out of scenario-based design, emphasizing the practical utility of narrative representations as well as their analytic utility in suggesting and contextualizing rationales (e.g., Carroll, 2000).

During the years after Nottingham, and to some extent caused by prodding from Long, I kept at this line of thinking, eventually reaching the programmatic claim that in HCI design rationale is the theory (Carroll and Rosson, 2003). During the early 1990s, I had many enjoyable interactions with Long. My recollection is that Long accepted that my approach integrated craft and applied science, but, perhaps not surprisingly, he felt our ”design rationale as theory” programme was underspecified as a disciplinary model, and that, in particular, it needed to be more explicit about defining effectiveness. Our discussion never came to an ending.

Comment 64

3.5. Making sense of HCI

In 1989, many in the field felt that HCI needed a better-defined paradigm, or as Long and Dowell termed it, a better-defined disciplinary model. For even in 1989, HCI was plainly a thriving and growing socio-technical endeavor that was diversifying much more than it was converging. Perhaps framing disciplinary models is just the human impulse to closure, to create figure from ground. Perhaps it is just researchers doing what they do. With another 20 years of hindsight, we can see more plainly now that HCI continues to diversify more than to narrow. Not only do Long and Dowell’s original contenders live on, but we have many variants of each.

Long and Dowell’s technical analysis wound up being more nihilistic than they most likely intended. I think this was because they faithfully and energetically applied overly rigid and a priori models of all three of their disciplinary models for HCI. To me this is the tragic pattern of positivism, which I take here as a paradigm– defining concern with how propositions are generated (discovery procedures, predictive models), warranted (usually in observable empirical phenomena), and logically related (e.g., by derivation or generalization). Positivism, as far as I can tell, is motivated by good and noble impulses: Escape subjectivism and capture universal truths, provide empirical foundation for knowledge statements, produce systematic, cumulative, and integrative knowledge-generating practices and knowledge descriptions and explanations (e.g., science) and so forth. The problem for positivism, and reason I see it as tragic, is that subjectivism is inescapable. Knowledge depends on context, on point of view, on history, on meaning making practices that are partially ineffable, and on levels of analysis that are incommensurable. Indeed, as emphasized by every philosopher since Kuhn (1962) science is a social institution, and what is regarded as sound, even what is regarded as true, is socially constructed.

Comment 65

My personal construction of this is that positivism belongs to that interesting category of wrong ideas we need to value. Positivism so overworries methodological issues that it produces results that are unproblematic but of little consequence. Still we are well advised to orient to positivist objectives. We should do so knowing that to take these objectives too seriously, too rigidly, will lead to paradigmatic dead ends. Throwing positivist cautions away can lead to empirical programmes for which we cannot know what methods were actually employed, what data were gathered, or what the results are really about.

Comment 66

Long and Dowell, I think, became snarled in an intellectual trap of their own design in characterizing the three disciplinary models for HCI. They characterized models that no one followed, and that no one ever has followed. They laid down positivistic criteria for disciplinary models that I suspect cannot be satisfied, and that, in any case, do not accord with what anyone actually does or has done in the craft, science, and engineering of HCI.

Comment 67

But they were in good company. On my side of the Atlantic, Allen Newell articulated an interestingly comparable disciplinary programme in his 1985 opening plenary address at the ACM CHI Conference (Newell and Card, 1985). Newell’s talk presented a vision of extending the early GOMS work into a much more comprehensive paradigm for science in HCI. He memorably said that psychology might be ”driven out” of HCI in the future if it were not pursued in a quantitative modeling paradigm (aka, ”hard science”). The talk provoked much controversy, discussion, and new research. It led to alternate proposals, modified proposals, replies and rejoinders (Carroll and Campbell, 1986; Newell and Card, 1986). Again, through the benefit of hindsight, we can see that Newell overstated the ”hard science” threat. His programme was ignored, yet psychology continues to thrive in HCI. As I look back, I think that what Newell really wanted was a science of HCI in which psychology (and other cognitive sciences) could play a central role. Newell’s real worry, I think, was not that the psychology of HCI might take a qualitative turn. He was worried about the interdisciplinary power balance between computer science and its human science partners. I have elaborated this historical reflection in Carroll (2006).

In my view, Newell and Long’s contributions must be taken in historical context. They were addressing the still-current threat of methodological fragmentation. They had helped to found HCI in the 1970s, and were trying to ensure that HCI could continue to prosper as it had in the early 1980s. Both proposed to achieve this through normative disciplinary frameworks. And both went a bit too far in this regard.

Comment 68

The conclusion I take away is that we should regard HCI as a sort of meta-discipline. I call it a community formed around the ever-expanding concept of usability (Carroll, 2009), because I think it is really just this shared pre-theoretic interest and commitment that causes HCI to cohere at all. HCI has no single disciplinary problem or specified set of practices, and certainly no single conception of effectiveness. Instead, the boundaries of HCI have expanded as the notion of usability became richer.

Comment 69

Usability was originally articulated naively in the slogan ”easy to learn, easy to use”. The blunt simplicity of this conceptualization gave HCI an edgy and prominent identity in computing. It served to hold the field together, and to help it influence computer science and technology development more broadly and effectively. However, inside HCI the concept of usability has been reconstructed continually, and has become increasingly rich and intriguingly problematic. Usability now often subsumes qualities like fun, well-being, collective efficacy, aesthetic tension, enhanced creativity, support for human development, and many others. The trajectory of its core concept explains how and why the HCI community has continued to grow and diversify. It also explains why a priori frameworks, such as those articulated by Long and by Newell, tend to look dated almost before they are formulated. More importantly, a dynamic view of usability suggests that what we have seen for the past three decades may just continue. Perhaps usability will always develop as our ability to reach further toward it improves.

Comment 70

This picture of HCI as a diverse community orienting to a concept whose meaning changes through time is unsettling. It implies that the methodological fragmentation Long addressed in his 1989 Nottingham keynote is endemic to HCI, not so much a problem to be remedied, but a characteristic to be accepted and leveraged, or at least coped with.

Comment 71 In either case, I think it remains useful to try to articulate disciplinary models – models that are evidenced in current practices, models from areas neighboring HCI that arguably could address current challenges in HCI, if they were to be adopted, and perhaps even models that we just invent. Some of these models could be the pure types that Long and Dowell worked with, perhaps too rigid to be implemented, but useful as analytic tools to characterize the more hybrid forms that can actually be observed.

Comment 72

Acknowledgements

This paper draws on several my own previous meditations on the history and foundations of HCI: Carroll, 1997, 2002, 2006, 2009. I thank the editors for organizing this project. I especially thank Peter Wright and two anonymous reviewers for their cheerfully insightful deconstructions of my essay. I believe that moments of reflection on the contributions of those who have helped to lead us in the recent past are not only appropriate celebrations, but also directly useful for us toward making sense of what we have done, and what we are doing now. Finally, I want to acknowledge and thank John Long, with whom I had a very stimulating debate at various conferences and workshops during 1988– 1993. These interactions were very helpful to me in motivating and focusing my own thinking and writing. Frankly, I think I benefited more, though John always seemed ready for another round. As far as I can remember and reconstruct, no minds were changed in these debates, but I recognize more clearly now that that is not necessarily the most important outcome of such a debate. The writing of this essay was supported in part by the Edward M. Frymoyer Chair Endowment.

References

Bentley, R., Hughes, J.A., Randall, D., Rodden, T., Sawyer, P., Shapiro, D., Sommerville, I. 1992. Ethnographically-informed systems design for air traffic control. In: Proceedings of the 1992 ACM Conference on Computer-Supported Cooperative Work, Toronto, Ontario, Canada, November 01–04, 1992. CSCW ’92. ACM, New York, NY, pp. 123–129. DOI: http://doi.acm.org/10.1145/143457.143470.

Bjerknes, G., Ehn, P., Kyng, M. (Eds.), 1987. Computers and Democracy — a Scandinavian Challenge. Aldershot, Avebury, VT.

Blackwell, A.F., 2006. The reification of metaphor as a design tool. ACM Transactions on Human–Computer Interaction 13 (4), 490–530.

Bornat, R., Thimbleby, H., 1989. The life and times of Ded, text display. In: Long, J.B., Whitefield, A.D. (Eds.), Cognitive Ergonomics and Human–Computer Interaction. Cambridge University Press, Cambridge.

Brooks, F., 1975. The Mythical Man–Month. MIT Press, Cambridge.

Buckley, P., 1989. Expressing research findings to have a practical influence on design. In: Long, J.B., Whitefield, A.D. (Eds.), Cognitive Ergonomics and Human–
Computer Interaction. Cambridge University Press, Cambridge.

Buxton, W., Baecker, R., Clark, W., Richardson, F., Sutherland, I., Sutherland, W., Henderson, A., 2005. Interaction at lincoln laboratory in the 1960’s: looking forward – looking back. In: CHI ’05 Extended Abstracts on Human Factors in Computing Systems, Portland, OR, USA, April 02–07, 2005. CHI ’05. ACM, New
York, NY, pp. 1162–1167.

Card, S.K., Moran, T.P., Newell, A., 1983. The Psychology of Human–Computer
Interaction. Erlbaum, Hillsdale, NJ. Carroll, J.M., 1989. Feeding the interface eaters. In: Sutcliffe, A.G., Macaulay, L.A. (Eds.), People and Computers V. Cambridge University Press, Cambridge, pp. 35–48.

Carroll, J.M., 1997. Human–computer interaction: psychology as a science of design.
Invited Chapter for the Annual Review of Psychology 48, 61–83.

Carroll, J.M., 2000. Making Use: Scenario-Based Design of Human–Computer Interactions. MIT Press, Cambridge, MA.

Carroll, J.M., 2002. Human–Computer Interaction. Encyclopedia of Cognitive Science. Macmillan/Nature Publishing Group, London.

Carroll, J.M., 2003. Situated action in the zeitgeist of human–computer interaction.
The Journal of the Learning Sciences 12 (2), 273–278.

Carroll, J.M., 2006. Soft versus hard: the essential tension. In: Galletta, Dennis,Zhang, Ping (Eds.), Human–Computer Interaction in Management Information Systems. In: Zwass, Vladimir (Ed.), Advances in Management Information Systems Series. Armonk, NY, M.E. Sharpe, Inc., pp. 424–432.

Carroll, John M., 2009. Human computer interaction (HCI). Interaction-Design.org, <http://www.interaction-design.org/encyclopedia/human_computer_interaction_ hci.html>. (Retrieved 19.05.09).

Carroll, J.M., Campbell, R.L., 1986. Softening up hard science: reply to Newell and Card. Human–Computer Interaction 2, 227–249.

Carroll, J.M., Campbell, R.L., 1989. Artifacts as psychological theories: the case of human–computer interaction. Behaviour and Information Technology 8, 247– 256.

Carroll, J.M., Rosson, M.B., 2003. Design rationale as theory. In: Carroll, J.M. (Ed.), HCI Models, Theories, Models and Frameworks: Toward a Multidisciplinary Science. Morgan-Kaufmann, San Francisco, pp. 431–461.

Diaper, D., 2002. Scenarios and task analysis. Interacting with Computers.

Dix, A.J., Harrison, M.D., 1987. Formalizing models of interaction in the design of a display. In: Proceedings of Intract’87 Second IFIP Conference on Human–
Computer Interaction. Elsevier Scientific, Amsterdam, pp. 409–413.

Dowell, J., Long, J., 1989. Toward a conception for an engineering discipline of
human factors. Ergonomics 32, 1513–1535.

Grudin, J., 2005. Three faces of human–computer interaction. IEEE Annals of the
History of Computing 27 (4), 46–62.

Hammond, N.V., Allinson, L.J., 1988. Development and evaluation of a CAL system for non-formal domains: the hitch-hikers guide to cognition. Computers and Education 12, 215–220.

Heath, C., Luff, P., 2000. Technology in Action. Cambridge University Press, Cambridge. Hutchins, E., Hollan, J., Norman, D., 1985. Direct manipulation interfaces. Human–
Computer Interaction 1, 311–338. Johnson, J., Roberts, T.L., Verplank, W., Smith, D.C.,

Irby, C.H., Beard, M., Mackey, K., 1989. The xerox star: a retrospective. Computer 22 9 (11–26), 28–29. Kuhn, T., 1962. The Structure of Scientific Revolutions. University of Chicago Press, Chicago.

Latour, B., 1987. Science in Action: How to Follow Scientists and Engineers Through
Society. Harvard University Press, Cambridge, MA.

Latour, B., Woolgar, S., 1986. Laboratory Life: The Construction of Scientific Facts, second ed. Princeton University Press, Princeton, NJ.

Long, J.E., Dowell, J., 1989. Conceptions of the discipline of HCI: craft, applied science and engineering. In: Sutcliffe, A., Macaulay, L. (Eds.), People and Computers IV, Proceedings of the Fifth Conference of the BCS HCI SIG. Cambridge University Press, Cambridge, pp. 9–32.

Myers, B.A., 1998. A brief history of human computer interaction technology. ACM Interactions 5 (2), 44–54.

Newell, A., Card, S., 1985. The prospects for psychological science in human– computer interaction. Human–Computer Interaction 1, 209–242.

Newell, A., Card, S., 1986. Straightening out softening up: response to Carroll and Campbell. Human–Computer Interaction 2, 251–267.

Nielsen, J., 1994. Usability Engineering. Morgan Kaufmann, San Francisco. Norman, D.A., 1982. Steps toward a cognitive engineering: design rules based on analyses of human error. In: Proceedings of the 1982 Conference on Human Factors in Computing Systems, Gaithersburg, Maryland, United States, March 15–17.

Orr, J.E., 1996. Talking About Machines: An Ethnography of a Modern Job. Cornell University Press, Ithaca, NY.

Plouznikoff, N., Plouznikoff, A., Robert, J.-M., 2005. Object augmentation through ecological human–wearable computer interactions. In: IEEE International Conference on Wireless and Mobile Computing, Networking and Communications, 22–24 August, WiMob’2005, pp. 159–164.

Rosson, M.B., Carroll, J.M., 2002. Usability Engineering: Scenario-Based Development of Human–Computer Interaction. Morgan-Kaufmann, San Francisco.

Shneiderman, B., 1983. Direct manipulation: a step beyond programming languages. IEEE Computer 16 (8), 57–69.

Sturt, G., 1923. The Wheelwright’s Shop. Cambridge University Press, London.

Suchman, L.A., 1987. Plans and Situated Actions: The Problem of Human–Machine Communication. Cambridge University Press, New York.