P. TIMMER and J LONG

Ergonomics & HCI Unit, University College London, 26 Bedford Way, London, WC1H 0AP

Expression de l’efficacité des horizons de planification

RÉSUMÉ

L’évolution technologique est souvent motivée par des ‘problèmes’. Pourtant, l’expression de ces problèmes en termes de performance des systèmes de travail n’est souvent qu’anecdotique ou implicite. Cette recherche propose une méthode explicite pour exprimer l’efficacité d’un système de travail. La méthode est illustrée sur un système de travail de gestion du transport. L’intérêt particulier de ce domaine concerne la façon dont l’interaction opérateur-technologie soutient efficacement la planification à l’avance (sous la forme d’un horizon de planification). La méthode est composée de quatre étapes. Premièrement, le comportement de planification à l’avance est conceptualisé. Un aspect critique de la méthode est la ‘Théorie des Horizons de Planification de l’Opérateur’ ainsi que ‘l’extension’ et ‘l’opportunité’ d’horizons de planification particuliers. Deuxièmement,  le domaine de travail est modélisé, afin d’établir la qualité du travail effectué par le système de travail. Troisièmement, les comportements qui soutiennent la planification efficace sont modélisés. Finalement, une comparaison est faite entre la qualité réelle du travail effectué et la qualité désirée. Si la performance tombe en-dessous d’un niveau désiré, les comportements du système de travail contribuant à l’inefficacité sont analysés. Si une planification inefficace est identifiée (c’est-à-dire un problème), la méthode soutient la recherche des origines du problème, ainsi que la construction d’une théorie causale. Bien que l’illustration ne porte que sur la planification d’un système de travail de gestion du transport, les étapes de la méthode sont proposées pour soutenir plus généralement l’expression de l’efficacité des systèmes ou autres.

Mots-clés:            Horizon de Planification; Efficacité d’un système de travail;  Contrôle du trafic aérien.

I.         INTRODUCTION

I. 1.            MOTIVATION AND OVERVIEW

This paper describes a method that enables the expression of: a) the plans of a process operator, and how far into the future those plans extend; and b) an assessment of how adequate those plans are, for ensuring that work goals are attained. The method is illustrated using an Air Traffic Management microworld. The need to express operator plans, and their extension, arises when technologies are being developed to support the planning of interventions with a dynamic evolving process; domains where process evolution needs to be anticipated, and process interventions need to be planned to address anticipated process states. By associating each individual plan with an assessment of adequacy, design problems may be characterised, problems that may be alleviated by technological support. Where the expression of plans shows those plans to be inadequate for attaining management goals, then new technologies can be proposed that may result in more effective operator planning behaviour.

In general, the evolution of a human-technology worksystem may either be problem-driven or technology-driven (Woods & Roth, 1988). Problem-driven evolution arises when a specific problem is attributed to the design of a technology, and the technology is then redesigned (or replaced) to remedy that specific problem. Such problems are frequently expressed by operators in the form of a subjective, experience-based report or anecdote, which may or may not lead to some in-depth analysis of the problem. Alternatively, technology-driven evolution may arise when a technology is redeveloped (or replaced) simply because redevelopment (or replacement) is possible, that is to say, not in the light of a specific ‘problem’ (such as inadequate planning). As the aim of the method proposed here is to relate operator planning (and plans) to effective or ineffective performance (intervention outcomes), and given that ineffective performance is considered a problem in need of a (technological) solution, the method may be understood as a system development tool, to support problem-driven evolution at the early stage of ‘problem formulation’ (Rasmussen, 1992; Woods & Roth, 1988). The method therefore assists in the process of progressing from operator anecdotes about problems, to a structured and more formal analysis and expression of those problems. The need to express planning problems, and thereby evolve technological solutions, arises in traditional process control domains such as Air Traffic Management (ATM), Railway Signal Management (RSM), nuclear power generation, and so forth. Throughout the paper, method application will be illustrated with reference to an ATM-like microworld of sufficient complexity to demonstrate the phenomena of concern to the method.

Before continuing, a definition of the term ‘design problem’ is offered.  A design problem is considered to exist, and therefore acts as a motivator for problem-driven evolution, when a desired level of performance is not being achieved by a human-technology worksystem. That worksystem may then be termed ‘ineffective’, as desired performance is somehow compromised. Where a design problem is believed to exist, a method is needed for expressing that problem (its severity, frequency, behavioural causes, etc.) in a manner that will contribute towards its solution. Outlining such a method is the aim of this paper. Being able to express worksystem ineffectiveness is particularly valuable when problem-driven evolution occurs within safety critical domains. For domains where the consequences of ineffective human-technology interaction have serious potential outcomes for human life, being able to express whether or not particular interactions are effective, supports reasoning about the adequacy of the technology in question. Within the context of Cognitive Engineering, such expression and reasoning is termed ‘diagnosis’ (Dowell & Long, 1998; Rasmussen, 1986) and, as shall be illustrated in the paper, can support formulation of the design problem that a re-designed technology should solve (Dowell, 1998).

The emphasis on ‘design’ and ‘design problems (and solutions)’ characterises the present approach as one of engineering, that is, contributing to the design of effective worksystems (Amalberti & Deblon, 1992; Dowell & Long, 1998; Flach, 1998; Hollnagel, 1998; Rasmussen, 1986; Reason, 1998; Vincente, 1998; Woods, 1998), rather than as one of science, that is, understanding the phenomena associated with worksystems and their behaviours (Barnard, 1991; Meyer & Kieras, 1999). Within the design approach, the present can be more precisely characterised as ‘design for effectiveness’, seeking to use the design primitives of ‘work’, ‘worksystem’, and ‘performance’ to motivate the acquisition and validation of design knowledge, to diagnose and solve design problems (in contrast, for example, to ‘human performance’, expressed as some form of speed and errors (Reason, 1998)).

Problems of interaction arise with many different types of technology. Here, particular consideration is given to problems arising with technologies that support planning ahead. In presenting the method, at the first stage, a theory is presented, for use when modelling how far ahead an operator plans. In contrast to other work on planning (Amalberti & Deblon, 1992; Boudes & Cellier, 1997; O’Hara & Payne, 1999), this theory makes reference to a plan’s ‘extension’, and its ‘adequacy’ (how well the plan ensures work goals are achieved). A method for expressing the effectiveness (or otherwise) of a plan necessarily requires consideration of the plan’s extension (how far into the future the plan accounts for an intervention), and whether or not the plan is adequate to ensure goals are attained.

The aim of this paper is therefore to propose a method for expressing the adequacy of operator planning, with special interest in capturing instances of ineffective planning (diagnosing design problems). The presented method comprises four stages. In the first stage, planning behaviours of interest are conceptualised by a domain-independent Theory of the Operator Planning Horizon, and requirements for modelling an operator’s planning horizon are generated from that theory. During the second stage, the work carried out by the human-technology (planning) worksystem is considered, and captured by a model of the domain. Here, the ATM-like microworld is described. In the third stage, models of operator planning horizons are constructed. Finally, these two types of model, of the domain (work) and of planning horizons (plans and planning behaviour), are considered alongside each another, and diagnostic assessments are made as to whether or not the plans formed were effective (i.e., whether or not there is a design problem concerning planning). When a problem is identified, a causal theory is constructed to relate how operator-technology interactions contributed to the occurrence of that problem. The four stages together constitute a method that helps to establish an explicit link between planning behaviours and the quality (effectiveness) of work executed by the worksystem, a relation frequently addressed in only an implicit fashion (Boudes & Cellier, 1997). The method therefore addresses the construction and use of models during the design process, a tradition well established within HCI research (Blandford & Young (1993); Card, Moran & Newell (1983)). In the analysis of operator-technology interactions, ‘effectiveness’ is considered a primitive (ontological) entity, alongside: human (planning) behaviour; technological behaviour (and how it supports planning); and details of the work performed (Dowell & Long, 1998). The paper is structured to reflect the stages of the method outlined above, and concludes with a discussion of the method.

I.2            ATM-LIKE MICROWORLD

For the purposes of method illustration, a laboratory-based ATM-like microworld was used. The microworld was constructed on the basis of an observational field study at Ringway Control Centre in Manchester, and possesses selected characteristics of the operational system that make it suitable for illustrating the method (Dowell, 1998; Long & Timmer, 2001). The domain is dynamic and imposes a significant planning burden upon the operator, who must anticipate the future state of air traffic, establish goals, collect and integrate data from different sources, plan tactical interventions to two aircraft variables (altitude and speed), and finally intervene with aircraft. Aircraft response to worksystem intervention is time-lagged, and the quality of aircraft management, in terms of aircraft safety and expedition (fuel use, progress to plan, minimum interventions), is calculated by the simulation software, following an interactive scenario.

The managed domain, containing aircraft, beacons, airways and so forth (Dowell, 1998), is generated by the simulation software and displayed on a computer-based radar. Paper-based flight progress strips are used, as observed in the operational worksystem, and document aircraft entry and exit states, aircraft identity and route. The strips are annotated after each intervention with details of aircraft state changes. Interventions to aircraft altitude and speed can be made via menu selection on the radar, rather than by ground-to-air radio. Through interaction with these technologies, the operator’s task is to ensure the safe and expeditious management of aircraft across the sector. Aircraft traverse a sector along airways, moving from an entry beacon to an exit beacon, via an intermediate beacon (see Figure 1). Between eight and ten aircraft are managed during a scenario. Operators are naïve subjects, trained in the management procedures necessary to ensure aircraft safety and expedition, and with some practice in the management task.

Figure 1. Topographic representation of the simulated sector

Figure 1 Représentation topographique du secteur simulé

While the simulation constitutes a considerable simplification of operational ATM, it is sufficient for method illustration. Planning can be observed, of interventions to (process) object variables, and is available for analysis. Likewise, desired levels of worksystem performance can be specified, actual levels of performance measured, and effectiveness or ineffectiveness thereby expressed. Providing these criteria are met, it is proposed that the method may be applied to other domains (operational or microworld). Having considered the microworld of interest, the method is presented in four stages. First, consideration is given to conceptualising the operator planning horizon. Second, the work carried out by the microworld operator is considered (Stage 2), after which models of planning horizons are constructed (Stage 3). Finally, ineffectiveness is diagnosed (Stage 4).

II.         THEORY OF THE PLANNING HORIZON

II.1            PLANNING SCOPE

The term ‘planning’, within the Cognitive Ergonomics / Science / Psychology literature is used to refer to a wide range of human operator behaviours: problem-solving (O’Hara & Payne, 1999); task scheduling (Hayes-Roth & Hayes-Roth, 1979); decision-making (Pietras & Coury, 1994); or anticipation (Boudes & Cellier, 1997, 1998). Additionally, planning can refer to an immediate action (Shallice, 1982), or an action after some temporal delay (Amalberti & Deblon, 1992). In consequence, some clarification of the scope of the term, as used here, is required. Planning refers to the formation of plans for actions/interventions with aircraft at some point in the future. An important distinction here, therefore, is between pre-planned interventions and unplanned interventions (referred to in this paper as management by ‘instant execution’). Instant execution occurs where an intervention is specified, and then immediately executed by the operator. As such, the term refers to the formation of real-time control decisions, with no forward-looking time delay between the specification of an intervention and its execution. In contrast, planning is always for some future intervention (ahead), that is, not the next action undertaken by the operator, after specifying the intervention. The planning ahead of interventions is commonly: (i) viewed as more efficient (i.e., involving fewer interventions) than instant execution, when implementation costs are high (O’Hara & Payne, 1999); (ii) associated with expertise and strategic thinking (de Groot, 1978); and therefore (iii) considered a form of best practice. For example, de Groot showed that grandmasters plan ahead to a depth of six or seven chess interventions, in contrast to novices, who may only consider one or two interventions ahead. The benefits of such ability are clearly demonstrated by game outcome; the grandmaster is a ‘master’ for good reason.

The development of technology, for the management of dynamic processes, reflects the general desirability of planning interventions further into the future. The ATM-like microworld, used in this paper, serves only to illustrate the method. However, within operational Air Traffic Management in recent years, the concept of ‘gate-to-gate’ aircraft route planning has emerged. This concept requires that the planning of aircraft interventions no longer be devolved to the level of sector management, but rather be considered for all sectors traversed by aircraft during flight, by a separate team of specialist ‘multi-sector planners’. Again, this strategy suggests that planning further ahead is considered generally to be a desirable feature of dynamic process management, a feature for which technological support is sought (David, 1997; Miaillier, 1998).

II.2            PLANNING THEORIES

Models of human and machine planning have a long history (Hoc, 1988; Miller, Galanter, & Pribram, 1960; Sacerdoti, 1977), such models drawing to a greater or lesser extent, on planning theory  (Suchman, 1993, 1987; Vera & Simon, 1993). For the purpose of expressing planning effectiveness, at Stage Three of the method, a model is constructed that represents an operator’s plans for future interventions with aircraft. Model construction is informed by a Theory of the Operator Planning Horizon. The theory is largely a synthesis of parts of other planning theories, with some additions to make it suitable for expressing planning effectiveness. Such theory-driven modelling is possible because the simulation presents the operator with a well-defined problem space, and the task is completed by operator-technology interaction. There are no human-to-human collaborative processes influencing the planning task (Hughes, Somerville, Bentley, & Randall, 1993). As such, the method addresses planning as work at the level of micro-level mental processes, rather than as a macro-level community activity (Engeström, 2000). There is only one agent of management, who plans, anticipates, intervenes, and so forth.

From planning theory, a number of stable phenomena have been documented, of both planning behaviour, and of plans themselves, that may be assumed when modelling operator planning for dynamic process management.  For example, planning is most frequently characterised as being a form of goal-oriented behaviour (Newell & Simon, 1972). Having expressed the goals of management within the microworld as the maintenance of aircraft safety and expedition, it is anticipated that individual plans for interventions (to speed and altitude of individual aircraft) can be associated with these management goals.

During the planning process, a number of mental representations (knowledge sources) are known to be utilised, mental representations of (1) the domain state (Moray, 1992; Rasmussen, 1986), and of (2) how to interact with technologies/devices (Young, 1983) to bring about transformation of that domain (Payne, Squibb, & Howes, 1990). A mental representation of the state of the domain corresponds to what the operator knows, from moment to moment, about the state of the managed traffic – the work being undertaken (Dowell & Long, 1998). This representation is frequently referred to as the operator’s ‘picture’ (Cox, 1992; Whitfield & Jackson, 1982), and it is used to predict the future states of aircraft on the sector, and thereby plan.

Different types of plan are known to exist (Hoc, 1993), two possible types being plans for: (1) high-level process management; and (2) particular interventions to particular process objects. Here, the plans of concern are scoped to the latter class, plans for interventions with simulated aircraft (worksystem actions that transform the domain). In addition to mental representations of the domain (used in planning), the operator requires representations concerning how to interact with worksystem devices (Young, 1983), so that the detailed specification of actions, which bring about interventions, can be constructed.

Hayes-Roth and Hayes-Roth (1979) have shown that human planning is opportunistic, in that individuals plan when opportunity and necessity demand. Operators may not maintain a complete, coherent and well integrated set of plans for all aircraft managed within the sector. Rather, plans may exist for some aircraft, known to be particularly problematic (top-down planning), but as domain events arise (or are predicted to arise) that affect management goals, plans may be constructed in a reactive manner (bottom-up). It is therefore not the case that what is planned (at ‘planning time’) is always executed (at specified ‘execution time’). Rather a range of outcomes may be observed as plans may be discarded or repaired (Hoc 1988; Woods, 1988), due to failures in information acquisition and subsequent anticipation, or decay (i.e., be forgotten (Timmer & Long, 2000)). Having considered some of the relevant planning literature, for the management of dynamic processes, the Theory of the Operator Planning Horizon can now be presented.

II.3            THEORY OF THE OPERATOR PLANNING HORIZON

The first stage of the method, for expressing the effectiveness of a planning horizon, involves conceptualising the planning phenomena of interest. For this microworld, conceptualisation involves scoping operator planning behaviour, observed in the simulation, with respect to the planning literature, and synthesising existing theory with concepts necessary to express the effectiveness of a planning horizon. The outcome here is the Theory of the Operator Planning Horizon (TOPH), which makes explicit the planning phenomena of concern to the research. The TOPH may be stated, in domain-independent terms, as follows:

• An interactive (planning) worksystem formulates plans for interventions to a domain that are intended to attain management goals.

• Plans are formed by planning behaviour that requires mental representations of: i) the domain; and ii) the devices (and how to interact with those devices, to bring about interventions).

• Plans specify: i) interventions to domain object attribute values; and ii) a ‘triggering condition’ for plan execution.

• One or more plans, that refer to the same domain object, constitute the planning horizon for that object.

• A planning horizon may be described in terms of its ‘extension’.

• A planning horizon’s extension is expressed in terms of the future state of the domain object in question, if all planned interventions are executed.

• The extension of a planning horizon may be described as being adequate or inadequate for ensuring that the goals of management are met. The adequacy of a planning horizon’s extension is determined by the individual plans it contains, that is to say, whether or not when implemented, those plans ensure management goals are attained. If an operator’s planning horizon extension is adequate, and assuming the planned interventions are executed, the horizon will support effective management.

The theory places emphasis upon the planning worksystem’s goals, mental representations, and plan details. The concept of an horizon’s ‘extension’ arises from the need to express how far into the future plans (within an horizon) account for the state of a managed object. For example, two planned interventions to an aircraft (in the microworld) may be sufficient to ensure that the aircraft leaves the sector in its planned ‘exit’ state. The horizon (comprising those two planned interventions) may then be said to ‘extend to’ that aircraft’s exit state (object goal state, at the time of planning). Alternatively, horizons for aircraft may extend to states, associated with particular beacons, or parts of airways. Expressing the planning horizon’s extension in terms of the state of an object may be contrasted with other theories, such as Boudes and Cellier’s (1997) Theory of Anticipation Range. In their theory, the extension of a set of plans is expressed in terms of time, for example, plans that extend over the next 5 minutes for a given aircraft (see also Anderson & Settle, 1996). Finally, given the present work’s focus upon expressing problems with plans, the notion of planning extension adequacy is critical and novel. The adequacy of an extension is a concept that is used to relate plans for particular interventions to management goals, and thereby enable the expression of whether or not the specified plans for an aircraft ensure safety and expedition for that aircraft, over the extension of the plans. In conclusion, while the theory embodies some familiar concepts from planning theories (e.g., use of mental representations in planning, existence of triggering conditions, and so forth), it also possesses some novel concepts that are important for the expression of planning effectiveness.

Use of the term ‘horizon’, to refer to the limits of a mental behaviour, is not new. Hutchins (1990) describes an ‘horizon of observation’ in ship navigation, and more recently Wong, Sallis and O’Hare (1997) have discussed the planning horizon with respect to ambulance dispatch, yet in a non-technical manner, without defining what is meant by the term. Hence the need to conceptualise the term, and thereby generate a theory. Perhaps the most complete and explicit exposition of an alternative theory to TOPH, that examines the horizon of some form of mental behaviour, is Boudes and Cellier’s (1997, 1998) Theory of Anticipation Range, which possesses two components: (1) a mechanism for anticipating future domain object states; and (2) a temporal horizon. In contrast, the TOPH possesses three components: (1) planning behaviour (including plans); (2) horizon extension (in terms of future object states, rather than time); and (3) the adequacy of horizon extension (explicitly addressed).

II.4            REQUIREMENTS FOR A MODEL OF THE PLANNING HORIZON

The TOPH conceptualises the planning phenomena of interest, for the modelling of an operator’s horizon. From the theory, requirements may be generated, concerning behaviours that a model of a planning horizon should capture (at Stage 3), if the model is to represent accurately an horizon. These requirements are:

• Models need to reference interventions with objects (aircraft) that can be associated with management goals (of safety and expedition).

• Models need to reference the operator’s mental representation of the state of the domain at the time of plan formation.

• Models need to specify the details of planned interventions, and the conditions for triggering such interventions.

• Planning horizon models need to be specified for each managed object, and moment-to-moment changes of horizon extension need to be expressed in terms of a change in state of the object to which reference is made – domain objects here may be abstractions, i.e. groups of functionally related objects (aircraft).

• Models need to specify all the interventions that actually take place with an object (planned and unplanned), and their impact upon management effectiveness, such that the adequacy of an horizon extension may be expressed retrospectively.

With respect to the microworld, it can be seen that following the TOPH, a model needs to represent: (1) operator plans for a particular aircraft (one aircraft per horizon); (2) the operator’s mental representation of the state of the domain at the time of planning; and (3) the interventions actually carried out. Such a model addresses the planning concern of this research. To examine the effectiveness of the plans formed, a model of the planning horizon needs to be considered alongside a second model, a model that captures the quality of work carried out (i.e. the extent to which the aircraft objects were actually managed in accordance with management goals). This second model of work quality is termed a domain model, and the focus for Stage 2 of the method. In the next section, a microworld-generated domain model is considered.

III.         DOMAIN MODEL

In the microworld, a computer-based simulation generates a radar image of the managed sector, updated as aircraft traverse the sector, or change speed/altitude in response to operator interventions. In addition to generating this image, after each operator intervention, the simulation calculates new domain model values for a hierarchy of aircraft attributes that may have been altered as a consequence of the intervention. At the lowest level of the hierarchy are what Dowell (1998) calls PASHT attributes, standing for: Position; Altitude; Speed; Heading; and Time.  From these low level attributes, intermediate attributes are calculated for aircraft: Progress (flight duration); Fuel Use; Separation and Number of Manœuvres. Finally, at the apex of the attribute hierarchy, values for aircraft safety and expedition are calculated from intermediate attribute values. Therefore, an intervention to change an aircraft altitude will be recorded in the domain model at the PASHT level, and consequences of that aircraft climbing/descending to the new altitude will be calculated in terms of progress and fuel use, and ultimately any consequences for aircraft safety and expedition at the new altitude (and during the ascent or descent) will be calculated. Following the last intervention with each aircraft, the set of attribute values calculated represent the final ‘actual’ values for: Progress, Fuel Use, and so forth; for that aircraft’s passage across the sector. In addition to calculating these values after each intervention, for each attribute the model possesses a goal value. These goal values are calculated, by the simulation software, based upon an optimal (goal) path across the sector. Therefore, given ‘actual’ attribute values (from management scenarios) and ‘goal’ values (from an optimal scenario), comparisons may be made, from intervention to intervention, between the actual and goal states of managed aircraft. Large discrepancies between actual and goal values will constitute the starting point for diagnosing worksystem design problems.

Figure 2. Part of a domain model for a single intervention with aircraft BAN

Figure 2. Partie du modèle du domaine pour une intervention unique sur l’avion BAN

Figure 2 shows part of a domain model for aircraft BAN, following an operator intervention to slow down BAN from 900Kmph to 720Kmph (the change to BAN’s PASHT attribute – Speed – is shown at A). At B and C, only Intermediate attribute values are shown, from which values for safety and expedition can be calculated. Bold values in brackets, alongside each attribute name, represent goal values for that attribute. Unbracketted values show calculated predictions for each attribute, given the aircraft’s state, following the most recent intervention. A set of such values, for all aircraft after each intervention, constitutes the domain model of interest. Therefore, as a consequence of an intervention to reduce BAN’s speed, its predicted Progress across the sector is slowed, from 1170secs to 2220secs; and Fuel Use is greatly reduced, as 720Kmph is the cruising speed for all aircraft (the speed at which fuel use is optimised). Within the model the Separation attribute shows the aircraft is safe, no separation conflicts with other aircraft have been predicted, given the new speed value. As a consequence of the intervention, the Number of Manœuvres attribute is increased by one.

When such data concerning the actual state of an aircraft are compared with goal values for that aircraft, an analysis of the quality of aircraft management is possible, both from moment to moment, and at the end of the management session. The data for BAN here show that prior to the speed intervention, BAN’s time to progress across the sector was 1170sec. When compared to BAN’s goal value for progress, this value indicates BAN to have been progressing too quickly across the sector, and due to exit the sector 1260secs earlier than planned (the difference between actual and goal values). The cost associated with such fast passage is clear from the fuel consumption figures, which show the aircraft, prior to intervention, consuming 328 more units of fuel than the goal value. Both before and after the intervention, BAN’s separation value is safe, and so BAN’s superordinate safety attribute can likewise be calculated as being safe – BAN is in a safe state, both before and after the intervention. Following the intervention, BAN’s progress and fuel consumption are more closely aligned with goal figures.

To establish instances of ineffective management, some criteria need to be applied to the size of any discrepancy between goal and actual attribute values. Here, it is assumed, for the purpose of expressing effectiveness, that if values for an aircraft’s progress or fuel consumption exceed 10% (either side) of the goal value, ineffective management has occurred. Likewise, if aircraft separation is violated, ineffectiveness has occurred. Finally, the Number of Manœuvres value should not exceed 3 (i.e. 50% in excess of the goal value shown). Consideration will now be given to the impact these criteria have (for intermediate attribute values), on high level attributes of safety and expedition. If separation is violated, the attribute value for safety likewise reflects this state (Safety = Unsafe). If any one of the values for: Progress; Fuel Use; or Number of Manœuvres exceed specified criteria, the aircraft may likewise be considered ‘unexpeditious’, i.e. the management goal of expedition is not being attained.

In conclusion, data from the microworld-generated domain model concern the quality of aircraft management, and instances of attribute values violating criteria may be associated with instances of ineffective management – unsafe aircraft, aircraft progressing too fast or slow, consuming too much or too little fuel, or undergoing intervention too often. Expressing the effectiveness of an operator’s planning horizon, therefore, involves associating domain model data with particular interventions, and establishing whether or not those interventions were planned.  In the next section, Stage 3 of the method for expressing the effectiveness of planning horizons is presented. A second model, a model of an operator’s planning horizon, is discussed. Stage 3 is followed in Stage 4, by an analysis of the effectiveness of that horizon, when the planning horizon model is considered in the light of data from the domain model (Stage 2), which contains data similar to that discussed above.

IV. Model of a Planning Horizon

IV.1 Data Requirements for Model Construction

From the Theory of the Operator Horizon, a number of requirements were identified for a model of the horizon. Each requirement is now considered in turn, and the means for obtaining data to address the requirement discussed.

Requirements

• Models need to reference interventions with objects (aircraft) that can be associated with management goals (of safety and expedition).

In the microworld, all interventions are made with the radar device, and are therefore observable. Continuous observation of operator interaction with worksystem technologies yields such data.

• Models  need to reference the operator’s mental representation of the state of the domain at the time of plan formation.

Operators establish the state of the domain by referencing worksystem devices. By observing head movements and pointing actions to fields on a Flight Progress Strip, it is possible to establish what data concerning the domain are being acquired by the operator. Traditionally, concurrent verbal protocols provide a rich source of data concerning what an operator knows about a problem, and how that knowledge is used in problem solving. Verbal protocol data, in addition to observation of operator head and hand movement, yields data that assist in inferring the operator’s changing mental representation of the domain. In addition, the mental representation can be inferred from interventions. For example, if an operator intervenes with an aircraft to give it a cruising speed, it is possible to infer that the operator knew that aircraft’s speed was not the desired cruising speed, prior to intervention.

• Models need  to specify the details of planned interventions, and the conditions for triggering such interventions.

In the absence of planning tools that support the explicit documentation of a set of plans, operator plans for interventions remain in the head of the operator. In consequence, only verbal protocol data can reveal the details of such plans, and associated triggering conditions.

• Planning horizon models need to be specified for each managed object, and moment-to-moment changes of horizon extension need to be expressed in terms of a change in state of the object to which reference is made.

Each model should reference an individual aircraft. As the horizon changes, the state of the aircraft in question can be established with reference to the computer-generated domain model for that scenario (see Section III).

• Models need to specify all interventions that actually take place with an object (planned and unplanned), and their impact upon management effectiveness, such that the adequacy of an horizon extension may be expressed retrospectively.

To satisfy the first requirement, all interventions are documented. The impact of each intervention, upon management effectiveness (how well the actual state of the aircraft matches its goal state), can be established with reference to the domain model generated for the management scenario.

Therefore, with regard to acquiring data for the purpose of modelling operator planning horizons, observational data, of operator hand and head movements are required (for information search and interventions), plus concurrent verbal protocol data (for plans and evidence of the content of the associated mental representation). With such data, a model of the operator’s planning horizon can be constructed, as in Figure 3, which separates each of these classes of data.

IV.2            MODEL OF AN OPERATOR PLANNING HORIZON

Figure 3 presents a model of an operator’s planning horizon for an aircraft LOG. The ‘Plan/Execution’ column of the model records all human-technology worksystem goals that concern interventions. Data in this column distinguish goals for immediate execution (in bold), from plans for future interventions (in italics). The ‘Intervention’ column documents those goals for interventions that were actually implemented.

Figure 3. Model of an operator’s planning horizon for aircraft LOG

Figure 3. Modèle pour un opérateur d’un horizon de planification pour l’avion LOG

Goals that concern interventions are formed, given a particular mental representation of the state of the aircraft in question, at the time of planning or instant execution. The model records such mental representations in the ‘category’ column. The set of possible categories reflect the range of possible states of the objects in the domain (Timmer & Long, 1997). Aircraft may therefore be ‘Active’, when they arrive on the sector, or ‘Incoming’, prior to arrival. Once active, they may be ‘Safe’ or ‘Unsafe’, ‘Expeditious’ or ‘Unexpeditious’ with respect to their speed (‘Unexpeditious (Speed)’) or altitude (‘Unexpeditious (Altitude)’). Once an aircraft is at its exit altitude and cruising speed, no further interventions are required as it is in its exit/goal state (‘Active Aircraft (Exit)’). The final ‘Encode’ column records device information fields that were searched by the operator as a means to forming a mental representation (category) of the state of an aircraft, for the purpose of goal/plan specification or monitoring.

The model for aircraft LOG is considered in detail. The model commences (Line 1) by showing that at the beginning of the management session, having encoded LOG’s Flight Progress Strip (FPS), the operator knows that LOG is an incoming aircraft. No plans are formed at this stage. LOG then enters the sector (Line 2), and by encoding LOG’s radar trace, the operator’s mental representation of the state of LOG is updated, from ‘Incoming Aircraft’ to ‘Active Aircraft’. Referencing again LOG’s FPS (Line 3), the operator establishes LOG as travelling at 900Kmph, and at an altitude of 13,000ft. Given LOG’s excessive speed, the aircraft is mentally represented as being unexpeditious with respect to its speed, and an intervention is immediately formed (Line 3) and executed (Line 4) to slow the aircraft down, and transform its state to ‘expeditious’ with respect to its speed. This intervention is an example of instant execution, and represented within the model as such. One consequence of the intervention is that the operator’s mental representation of LOG is now updated, to reflect its new expeditious state. LOG is safe and expeditious, resulting in the operator forming a default plan, to leave LOG in its current state for the foreseeable future. The model then shows that LOG is left to progress from its entry beacon to the intermediate beacon ‘Delta’, and then on to its exit beacon ‘Epsilon’, at altitude 13,000ft and speed 720Kmph. Once at its exit beacon (Line 5), the operator encodes FPSs for both LOG, and a second aircraft SAM, and realises that both aircraft occupy the same altitude, and safety conflict is possible. LOG is therefore mentally re-categorised as ‘Unsafe’, and a plan is formed to give LOG its exit altitude of 4,000ft later (in the near future). The model then shows (Line 6) that the operator’s mental representation of: (1) the state of LOG; and (2) the plan for future intervention (formed at Line 5), are both forgotten (decay) and the plan to ‘Give LOG (its exit) altitude later’ is re-formed (at Line 7). On the occasion of this re-planning, the model shows that no safety conflict with SAM was identified, and LOG was represented once more as a ‘Safe Expeditious Aircraft’. The next line of the model (Line 8) shows that the operator appears to have forgotten (again) the state of LOG, and by referencing data from FPSs (Line 9), reforms the mental representation that LOG is a safe expeditious aircraft. The plan formed at Line 7 is not assumed to have decayed on this occasion, as the model shows no further re-planning of the intervention, until it is executed at Line 10. The final line of the model shows that LOG is given its exit altitude as planned, and the default plan to leave LOG until it exits the sector is formed, as the aircraft is in its exit state, and no further interventions are necessary.

If the model is considered with respect to how it documents the extension of the operator’s planning horizon for LOG, the following observations can be made. At Line 4, a plan is formed to ‘Leave LOG’. Given that LOG is flying at a high altitude, and at cruising speed, and is safe (i.e. is an ‘Active Safe Expeditious Aircraft’), LOG may be left in such a state, until near its exit beacon, and then it should be given its exit altitude. The plan to ‘leave’ LOG may therefore be said to extend to ‘near LOG’s exit beacon’. At Line 5, when LOG is near its exit beacon, a further plan is formed to give LOG its exit altitude. At Line 5, with such a plan, the operator’s planning horizon may be described as extending to ‘LOG’s exit state’ (i.e. to the end of its management). At Line 7, the new duplicate plan (of the plan formed at Line 5) has an identical extension to that of the plan at Line 5. Therefore, with a model of an operator’s planning horizon, from moment to moment, it is possible to express the ‘extension’ of the plans formed, either in terms of: a) the state of the aircraft in question, for example, the plans extend to LOG’s exit state; or b) some position on the sector at which the aircraft’s state must change, for example, plans for aircraft LOG extend to near LOG’s exit beacon’. A description of the extension of a planning horizon, therefore, arises from the details of the operator’s plans for a given domain object (aircraft) at a given moment. As plan details change, so too does the horizon’s extension.

V.         EXPRESSING HORIZON EFFECTIVENESS

Having conceptualised the planning horizon (Stage 1), described how a domain model captures work quality (Stage 2), and illustrated how a planning horizon is modelled (Stage 3), in Stage 4 of the method, expression of the effectiveness of that horizon is undertaken. In the first instance, the model of the planning horizon is considered, line-by-line, alongside objective data from the domain model, concerning aircraft states after each intervention. The actual state of aircraft (from the domain model) can then be compared with the operator’s inferred mental representation of the state of that aircraft (‘Category’ column of a planning horizon model), and plans and interventions considered, to establish whether or not the plans/interventions were effective. Once completed, and in the event of an instance of ineffectiveness occurring, some expression of its cause can be undertaken.

The Theory of the Operator Planning Horizon expresses the concept of effectiveness (attaining management goals), in terms of the adequacy of an horizon’s extension. Adequacy of an horizon’s extension relates to the quality of work that will be brought about by the worksystem, if the plans that make-up the planning horizon are executed. An horizon’s extension may be considered adequate, if the plans (that make-up the horizon), when executed, lead to the attainment of worksystem goals of safety and expedition. Adequacy is, therefore, a difficult attribute of an horizon to assess. At one moment the horizon may extend to an Aircraft’s (Aircraft X) exit state, and be adequate to ensure goals are met. At another moment, a second Aircraft Y may be given the same altitude as Aircraft X, thereby rendering the plans, that make-up Aircraft X’s planning horizon extension, inadequate for ensuring the maintenance of safety. Re-planning is then necessary. Existing plans may need to be discarded, given the new unsafe state of the aircraft. For each line of the model in Figure 3 (starting at Line 4), an assessment of adequacy is made, following a short description of the plan being assessed.

• Line 4

Encode       Intervention          Category                                           Plan/Execution

description

LOG is given speed 720Kmph (instant execution), and a plan is formed to ‘Leave LOG’ at speed 720Kmph, and altitude 13,000ft (i.e. as an Active Safe Expeditious Aircraft). A single plan is formed that extends to near LOG’s exit beacon.

adequacy

Following the intervention to LOG’s speed, at the time of plan formation, the domain model’s prediction of LOG’s state is presented in Figure 4.

Figure 4. Domain model performance data for LOG

Figure 4. Données du modèle de performance du domaine pour LOG

The domain model shows that LOG is:

• safe in its new state

The planning horizon at Line 4 is made-up of a single plan, to leave LOG, and given that LOG is safe in its current state, it may be said that the horizon extension, at this point, is adequate for ensuring that aircraft safety is maintained.

• expeditious in its new state

Projected progress is within 10% of criterion. Projected fuel use is 18.8% in excess of criterion. Given LOG’s speed and high altitude, it would seem the projection of excessive fuel use refers to fuel already consumed, before the intervention, i.e. when travelling at 900Kmph earlier in the management scenario (see Line 3, Figure 3). The plan to leave LOG near its exit beacon is therefore adequate for ensuring aircraft expedition.

• Line 5

Encode                  Intervention    Category                    Plan/Execution

description

LOG progresses across the sector to its exit beacon. A plan is formed to give LOG its exit altitude of 4,000ft, ‘later’. This single plan extends to LOG’s exit state. While ‘later’ is a triggering condition of minimal specification, the assumption that the horizon extends to LOG’s exit state is considered justified, because all necessary interventions have been specified to ensure LOG leaves the sector in its exit state.

adequacy

At planning time, and as a consequence of the adequacy of the planning horizon’s extension at Line 4, LOG is safe and expeditious. This plan is actually executed at Line 10. It is therefore possible to consult the domain model’s assessment of LOG’s safety and expeditiousness after this intervention (Figure 5), and thereby assess the adequacy of the plan extension.

Figure 5. Performance data for LOG after the last intervention

Figure 5. Données de performance de LOG après la dernière intervention

From Figure 5, it is therefore possible to say retrospectively:

• the planning horizon extension (to LOG’s exit state) will ensure its safety

• while progress and exit altitude are as planned, LOG’s fuel

consumption increases with this intervention (40% in excess of the

criterion value). It would therefore appear that this horizon extension is not adequate for ensuring that aircraft expeditiousness is maintained.

• Line 6

Encode           Intervention               Category                              Plan/Execution

The details of the plan to ‘Give LOG altitude 40, later’ appear to have decayed, and so the model shows the operator has no plans at this time.

• Line 7:

Encode                    Intervention     Category                  Plan/Execution

description

The intervention is re-planned, as at Line 5.

adequacy

As at Line 5.

No further plans are formed. At Line 10, the plan formed at Line 7 is executed. The domain model, as discussed at Line 5, shows LOG is unexpeditious with respect to its fuel use (40% excess).

From this line-by-line analysis, it is clear that the effective management of aircraft LOG’s fuel consumption did not take place as desired. In consequence, the domain model shows LOG to have been unexpeditious as it exited the sector, having consumed 40% more fuel than desired (the bracketed goal value for fuel use in the domain model). Given this ineffectiveness, the following expression of the problem, with this ATM planning task, is possible:

The problem of managing aircraft expedition with respect to fuel use may originate, as in the case of LOG, with aircraft entering sectors at high speed (greater than cruising speed), thus already rapidly consuming large quantities of fuel. The timely reduction of aircraft speed can alleviate this problem. However it would seem a greater part of the problem arises from judging the appropriate moment to intervene with aircraft, to allocate low exit altitudes. Managing aircraft at cruising speed, as high as possible for as long as possible, is the best strategy for minimising fuel consumption. With a reduction of altitude comes a commensurate increase in fuel consumption. If an aircraft is to leave a sector at a low altitude (e.g., for airport approach), careful judgement is required as to when to execute such an intervention, so that the aircraft exits the sector at the exit altitude. If such an intervention is executed too early, the aircraft will fly for an extended duration at a low altitude, consuming higher quantities of fuel. In the case of LOG, this problem would appear to have been the most important. The operator’s plan to ‘Give LOG Altitude 4,000ft, later’ lacked accurate reference to a position within LOG’s final airway, when the intervention would be made, ‘later’ merely meaning ‘near LOG’s exit beacon’. It is assumed that in consequence, LOG was given its exit altitude too early, and flew too low for too long, to maintain an expeditious level of fuel consumption. While worksystem technologies supported the operator in forming a plan for the correct intervention, they offered no support for judging the most appropriate moment for plan execution.

This expression of the causes of ineffectiveness accounts for ineffectiveness in the management of a single aircraft. It is proposed that for each additional instance of ineffectiveness, attributed to fuel use, a similar analysis be undertaken. When considering the data over a number of instances of the same problem, a general expression of the problem is possible (Timmer, 1999). Nevertheless, from consideration of the expression above, it is proposed that problem-driven technological evolution can benefit from problem expressions, similar to that illustrated. In the case of the evolution of technologies to support better fuel use management, redesign may commence with consideration of how to support operator judgement in timing the moment of low-level aircraft descent, prior to exit.

V.         DISCUSSION

In this paper, a method has been proposed and illustrated for expressing human-technology worksystem effectiveness during a planning task. The method involves stages of: conceptualising behaviour of interest; modelling the domain to measure work quality; modelling planning behaviour; and considering work quality alongside worksystem behaviour, to enable an expression of effectiveness. Using an illustration of the poor quality management of expedition, ineffectiveness was identified, and expressed in terms of worksystem behaviour, and in a manner proposed to support problem-driven technological evolution.

Of the method, a number of observations may be made. The method is comprised of a set of general stages, rather than a set of detailed procedures. For example, following conceptualisation, a domain model needs to be constructed. The illustration discusses one such model (not how it was constructed), and it is accepted that there are many alternatives to the one discussed. For the method to express effectiveness successfully, it is merely a requirement that a domain model measure work quality in some way, using some performance criteria for judgement of ineffectiveness (design problems). Without such measurement, comparing worksystem behaviour with work quality during Stage 4 is not possible. One advantage of expressing the method in this manner is that it extends the method’s scope of application. An ATM-like microworld was the focus of the illustration, and planning ahead conceptualised with respect to the planning behaviour likely to be observed during management of that microworld. Using another domain (the domain of Railway Signal Management (RSM) has also been analysed), it is possible to envisage other behaviours being conceptualised, for example, the notion of planning extended to include strategic plans, (for example to maximise train throughput or even out train flow), as well as plans for discrete interventions (tactical) (for example, to particular signals). Provided such planning is modelled accurately at Stage 3, the logic of the method, and successful expression of effectiveness, should be maintained.

The success with which the method can be migrated from a microworld to an operational environment, is largely a function of the extent to which a) the planning horizon modelling requirements can met, and b) the target domain modelled (and performance measured). In current operational ATM, for example, the method, as it stands, will not scale-up, as the verbal protocol data (revealing planning behaviour) can not be obtained without task interference. In the future, with Datalink technology, method application to operational ATM may be more feasible. In RSM, the voice channel is largely free, except for control-to-train communication, and control room communication – here also, management is largely undertaken by a single signalman (with some communication with other managed line sections and a supervisor).

When this work is considered alongside the work of Boudes and Cellier (1997, 1998), concerning controller ‘anticipation range’ in ATM, some similarities and differences can be identified. Theoretically, Boudes and Cellier’s work has strong similarities, in that they too seek to establish relationships between operator mental representations of the domain and devices, and plans for interventions and flight strip management. However, the method presented here, for modelling the effectiveness of planning horizons, possesses a number of novel components. Firstly, it attempts to characterise the extension of a planning horizon in non-temporal terms, but rather in terms of the future states of intervened aircraft (whether or not aircraft will be safe or unsafe, and to what future point (on the sector) do such plans extend). Secondly, the method possesses a domain model, with which to assess the adequacy of plans for future interventions. The domain model is crucial to the successful execution of the method, as it enables an assessment to be made of how well the operator is planning, and how adequate the specified plans are for ensuring that management goals are achieved. Without a domain model, a method can only determine that an operator is planning. No assessment of the quality of those plans can be made.  As the focus of the method is to support the development of technologies that will improve operator planning abilities, the domain model is crucial for determining existing planning ineffectiveness and how such ineffectiveness can be overcome through re-design.

To conclude, this paper tries to make explicit a number of important relationships that need to be established before the expression of effectiveness is possible. The Theory of the Operator Planning Horizon tries to make clear the behavioural phenomena of concern, and enables the clear derivation of requirements for data that constitute a representation of such an horizon. Likewise, the domain model enables the identification of problems of worksystem performance, and subsequent construction of a worksystem model to establish the behaviour that brought about less than the desired quality of work. Establishing relationships from the data serves to augment the subjective (anecdotal) recall of operator ‘problems’ with technologies, and offers some basis for quantifying problems, and the subsequent generation of priorities for re-design. Such a method is therefore considered a useful tool, to compliment existing design practices, to support the explicit expression of the magnitude of a design problem. As such, it is considered to have advanced the ‘design for effectiveness’ approach, since without a well-specified expression of the design problem, there can be no (known) design solution, nor acquisition and validation of design knowledge supporting the transition from one to the other (Long & Timmer, 2001). Phenomena-driven and human performance-driven approaches are unable to support such a design origin and transition.

VI.         REFERENCES

Amalberti, R., & Deblon, F. (1992). Cognitive modelling of fighter aircraft process control: a step towards an intelligent on-board assistance system. International Journal of Man-Machine Studies, 36, 639-671.

Anderson, B. F., & Settle, J. W. (1996). The influence of portfolio characteristics and investment period on investment choice. Journal of Economic Psychology, 17, 343-358.

Barnard, P. (1991). Bridging between basic theories and the artifacts of human-computer interaction. In J. M. Carroll (Ed.), Designing interaction: Psychology at the Human-Computer Interface. Cambridge, UK: Cambridge University Press.

Blandford, A., & Young, R. M. (1993). Developing runnable user models: separating the problem solving techniques from the domain knowledge. In J. L. Alty, D. Diaper, & S. Guest (Eds.) People and Computers VIII, Proceedings of HCI’93. Cambridge, UK: Cambridge University Press.

Boudes, N., & Cellier, J-M. (1997). Evaluating the concept of anticipation range in air traffic control. Paper presented at the Ninth international Symposium of Aviation Psychology, Columbus, Ohio, May.

Boudes, N., & Cellier, J.-M. (1998). Étude du champ d’anticipation dans le contrôle du trafic aérien. Le Travail Humain, 6, 29-50.

Card, S. K., Moran, T. P, & Newell, A. (1983). The Psychology of Human- Computer Interaction. Hillsdale, New Jersey: Lawrence Erlbaum Associates.

Cox, M. (1992). The cognitive aspects of the air traffic control task: a literature review. IAM Report No. 718. Psychology Divison, RAF Institute of Aviation Medicine.

David, H. (1997). Radical revision of en-route air traffic control. EUROCONTROL Experimental Centre Report No. 307. http:/www.eurocontrol.fr/public/reports/eecreports/1997/rep307

de Groot, A. D. (1978). Thought and choice in chess. The Hague: Mouton.

Dowell, J., & Long, J. (1998). Conception of the cognitive engineering design problem. Ergonomics, 41, 126-139.

Dowell, J. (1998). Formulating the cognitive design problem of air traffic management. International Journal of Human-Computer Studies, 49, 743-766.

Engeström, Y. (2000). Activity theory as a framework for analyzing and redesigning work. Ergonomics, 43, 960-974.

Flach, J. M. (1998). Commentary. Cognitive Systems Engineering: putting things in context. Ergonomics, 41, 163-167.

Hayes-Roth, B., & Hayes-Roth, F. (1979). A cognitive model of planning. Cognitive Science, 3, 275-310.

Hoc, J-M. (1988). Cognitive Psychology of Planning. London: Academic Press.

Hoc, J-M. (1993). Main features of the human supervision of a long response latency process. In Proceedings of the IEEE/SMC’93 Conference on Systems, Man and Cybernetics, pp. 114-119. Piscataway, NJ: IEEE.

Hollnagel, E. (1998). Commentary. Comments on ‘Conception of the cognitive engineering design problem’ by John Dowell and John Long. Ergonomics, 41, 160-162.

Hughes, J. A., Somerville, I., Bentley, R., & Randall, D. (1993). Designing with ethnography. Making work visible. Interacting with Computers, 5, 239-253.

Hutchins, E. (1990). The technology of team navigation. In J. Galegher, R. E. Kraut, & C. Egido (Eds) Intelligent Teamwork: Social and Technological Foundations of Cooperative Work, pp.191-220. Hillsdale, New Jersey: Lawrence Erlbaum Associates.

Long, J., & Timmer, P. (2001). Design problems for research: what can we learn from ATM-like microworlds. Le Travail Humain, 64.

Meyer, D. E., & Kieras, D. E. (1999). A computational theory of executive cognitive processes and multiple-task performance: Part 1 Basic Mechanisms. Psychological Review, 104, 3-65.

Miaillier, B. (1998). ATM Strategy for 2000⁺: Volumes 1 and 2. EUROCONTROL Report FCO.ET1.ST07.DEL02. http://www.eurocontrol.be/ded/atmstrat

Miller, G. A., Galanter, E. & Pribram, K. H. (1960). Plans and the Structure of Behaviour. New York: Holt, Rinehart & Winston.

Moray, N. (1992). Mental models of complex dynamic systems. In P. Booth, & A. Sasse (Eds). Mental Models and Everyday Activities, Second Interdisciplinary Workshop on Mental Models, pp. 103-132. Cambridge, UK, March.

Newell, A., & Simon, H. A. (1972). Human Problem Solving.  Englewood Cliffs, NJ: Prentice-Hall.

O’Hara, K, P., & Payne, S. J. (1999). Planning and the user interface: the effects of lockout time and error recovery cost. International Journal of Human-Computer Studies, 50, 41-59.

Payne, S. J., Squibb, H. R., & Howes, A. (1990). The nature of device models: the yoked state space hypothesis and some experiments with text editors. Human-Computer Interaction, 5, 415-444.

Pietras, C. M., & Coury, B. G. (1994). The development of cognitive models of planning for use in the design of project management systems. International Journal of Human-Computer Studies, 40, 5-30.

Rasmussen, J. (1986). Information processing and human-machine interaction: An approach to cognitive engineering.  New York: North Holland.

Rasmussen, J. (1992). The ecology of work and interface design. In A. Monk, D. Diaper, & M. D. Harrison (Eds.), People and Computers VII. Proceedings of the HCI’92 Conference. Cambridge, UK: Cambridge University Press.

Reason, J. (1998). Commentary. Broadening the cognitive engineering horizons: more engineering, less cognition and no more philosophy of science, please. Ergonomics, 41, 150-152.

Sacerdoti, E. D. (1977). A Structure for Plans and Behaviour.  New York: Elsevier.

Shallice, T. (1982). Specific impairments of planning. Philisophical Transactions of the Royal Society of London, B 298, 199-209

Suchman, L. (1987). Plans and situated actions: the problem of human- machine communication. Camridge, UK: Cambridge University Press.

Suchman, L. (1993). Response to Vera and Simon’s situated action: a symbolic interpretation. Cognitive Science, 17, 71-75.

Timmer, P. (1999). Expression of operator planning horizons: a cognitive engineering approach. PhD Thesis, University of London.

Timmer, P., & Long, J. (1997). Separating user knowledge of domain and device: a framework. In H. Thimbleby, B. O’Conaill & P. Thomas (Eds.) People and Computers XII. Proceedings of HCI’97, pp379-395. London, UK: Springer.

Timmer, P., & Long, J. (2000). Plans versus outcomes: establishing the costs of planning. In E. Hollnagel, P. Wright, & S. Dekker, (Eds.) Proceedings  of the Tenth European Conference on Cognitive Ergonomics, pp158-165. Linkoping, Sweden: European Association of Cognitive Ergonomics.

Vera, A. H., & Simon, H. A. (1993). Situated Action: a symbolic interpretation. Cognitive Science, 17, 7-48.

Vincente, K. J. (1998). Commentary. An evolutionary perspective on the growth of cognitive engineering: The Risø genotype. Ergonomics, 41, 156-159.

Whitfield, D., & Jackson, A. (1982). The Air Traffic Controller’s ‘Picture’ as an Example of a Mental Model. In G. Johannsen, & J. E. Rijnsdorp, (Eds.) Analysis, Design, and Evaluation of Man-Machine Systems, pp. 45-52. New York: Pergamon.

Wong, W., Sallis, P., & O’Hare, D. (1997). Eliciting portrayal requirements: experiences with the critical decision method. In H. Thimbleby, B. O’Conaill, & P. Thomas, (Eds.) People and Computers XII. Proceedings of HCI’97, pp. 397-415.  London, UK: Springer.

Woods, D. D., & Roth, E. M. (1988). Cognitive Systems Engineering. In M. Helander (Ed.), Handbook of human-Computer Interaction, pp. 3-43. Amsterdam: North Holland.

Woods, D. D. (1988). Commentary: Cognitive Engineering in complex dynamic worlds. In E. Hollnagel, G. Mancini, & D. D. Woods (Eds.) Cognitive Engineering in Complex Dynamic Worlds. London: Academic Press.

Woods, D. D. (1998). Commentary. Designs are hypotheses about how artifacts shape cognition and collaboration. Ergonomics, 41, 168-173.

Young, R. M. (1983). Surrogates and mappings: two kinds of conceptual models for interactive devices. In D. Gentner, & A. L. Stevens (Eds.), Mental Models, Ch. 3. Hillsdale, New Jersey: Lawrence Erlbaum Associates.

John Long

Emeritus Professor, Ergonomics and HCI Unit, University College London, 26, Bedford Way, London, WC1H OAP, UK ( Now at UCL Interaction Centre)

Keywords: HCI Engineering, Domain of work, Design problems

Abstract

Usability continues to be central to Human-Computer Interaction (HCI), in spite of recent developments associated with user experience, user pleasure, etc. The Internet continues to dominate the introduction of a wide range of new technologies and rightly has become central to HCI. One current view of HCI, then, is in terms of the usability of Web pages. This view is appealing and there can be no doubt that usable Web pages would be a laudable achievement for HCI This paper supports such a view, but argues that it is too limited. HCI, as an engineering discipline, supports user interaction design, but for effectiveness. The latter necessarily includes usability – how easy it is to use a computer, but also task quality – how well the work is performed by the interactive user-computer worksystem. Easy-to-use computers may produce poor work, hard-to-use computers may produce good work. Usability alone fails to distinguish between these two cases, but differentiation is essential for HCI design and evaluation, because both usability and task quality need to be as desired. This paper proposes a domain approach to make good the limitation of HCI as the usability of Web pages. The domain of work is represented as object, attributes, values and their desired changes – task quality. Implicit domain models (medical reception; military command and control; and domestic energy management) and explicit domain models (off-load planning; emergency management; and air traffic management) are described. Each model is illustrated in terms of its potential support for design as the diagnosis of design problems and the prescription of design solutions, both as they relate to the task quality of the associated domains of work. It is concluded that HCI is indeed more than the usability of Web pages and that the domain approach to making good the limitation shows promise. In particular, implicit domain models are considered to support design in the short-term, while explicit domain models are considered to support design and research in the long-term.

1. Introduction

Usability and Web pages continue to be central to HCI. However, the view that HCI is no more than the usability of Web pages is rejected as too limited. Nevertheless, usable Web pages constitutes a laudable goal for HCI. This paper proposes a domain approach to HCI to make good this limitation.

1.1 Usability and HCI

HCI has evolved considerably since those early days. A number of developments have extended its scope. For example, computer-supported co-operative work (CSCW) emphasises the interdependance between computer users. Ubiquitous computing (Ubicomp) emphasises the variety of computing devices and their locations. Virtual reality (VR) emphasises the importance of ‘presence’ in its simulations. However, in spite of these developments, HCI continues to retain usability as a critical expression of performance.

The concept of usability has also evolved since the early days of HCI. Pleasure has been proposed as a concept, relating people to computing products, which goes beyond usability (Green and Jordan, 2002). User experience claims to be a more complete expression of the same relationship (Light & Wakeman, 2001). However, these new concepts do not claim that usability is unimportant, or even unnecessary, only that there are additional people-product relations. Usability, then, remains central to HCI. It is concluded that in spite of extensions to both the scope of HCI and usability, over the years since their inception, the latter remains central to the former.

1.2 Web Pages and HCI

New technologies continue to characterise and to extend the scope of HCI. Personal computers soon became networked. Broadband communication integrated data processing and communications. Multi-media synthesised text and graphics. Multi-modality combined user inputs from speech, gesture and keyboard. Other new technologies include: distributed networks; mobile computing and telephony; smart buildings; wearable computing; augmented reality, etc. However, some of these new technologies are only in their infancy. Other technologies have proved less successful than originally expected. Yet other technologies have achieved only modest market penetration. In contrast, the new technology, that is the Internet, has continued to develop, offers a number of successful services (although not all are succesful) and has achieved deep market penetration. It is unsurprising, then, that Web pages have rightly become central to HCI.

1.3 Usability, Web Pages and HCI

Given the centrality of both usability (Section 1.1) and Web pages (Section 1.2) to HCI, one might be forgiven for thinking that HCI is essentially the usability of Web pages. This view is appealing and there can be no doubt that usable Web pages would be a laudable and notable achievement for HCI. The notability would derive both from the importance of the achievement for users and from the increased competition of other professionals, such as graphic designers, software engineers, marketeers and financiers to influence the design and evaluation of interactive computer systems. The view that HCI is the usability of Web pages is shared here. However, the proposition is understood as ‘HCI is not less than the usability of Web pages’, but not ‘HCI is only the usability of Web pages’. The scope of HCI must include usability, a primary expression of performance and Web pages, the dominant New Technology. HCI, then, is not limited to, but more than the usability of Web pages. The aim of this paper is to propose an approach – that of the domain – for HCI to go beyond the usability of Web pages.

2. General Domain Approach to HCI

The limitations of the view that HCI is no more than the usability of Web pages are analysed in terms of: completeness; coherence; and fitness-for- purpose. A domain approach is proposed, which makes good these limitations in terms of the three criteria.

2.1 Limited View of HCI

It has been argued earlier that the view of HCI as only the usability of Web pages is too limited. It is worth setting out these limitations in more detail. The latter both provide support for the argument and set up criteria against which to assess alternative approaches to HCI -such as that of the domain, as proposed here. The criteria are: completeness; coherence; and fitness-for-purpose. These criteria derive from the concept of a discipline, whose knowledge, acquired by research, supports its practices (Long, 1996).

First, the usability of Web pages is an incomplete expression of HCI performance. Ideally, usable interactive user-computer systems also achieve their goals. However, HCI evaluations routinely identify usable systems, which fail to achieve their goals and hard-to-use systems, which do achieve their goals. Usability, then, can only constitute one aspect of performance. There also needs to be some expression of the work an interactive system performs and how well it performs that work. The usability of Web pages is too limited a view of performance. If the Web pages provide a service, such as e-commerce, performance needs to express how well the service is effected, for example, goods purchased.

Usability also fails to reflect the computer’s contribution to performing work. For example, a hard-to-use system might be made more easy to use by simplification of the interface. Simplification might be achieved by reducing inappropriate computer behaviours (and so the code supporting them) or increasing appropriate behaviours (and so the code supporting them). In both cases, usability is the same, but the computer contribution differs. This difference should be reflected in a more complete expression of the interactive worksystem’s performance.

Second, the usability of Web pages is an incoherent expression of HCI performance. Usability appears to be a property of the computer. For example, a hard-to-use computer can be re-designed to be easier to use. But computers can also become easier to use by training, learning and practice with no change in the computer itself. Training, learning and practice all modify properties of the user and not the computer. So, usability would be more coherently expressed, if it referred to the user contribution to the interaction. For example, if usability were expressed as effort, or somesuch, the latter might vary as a function of re-design, training, etc. Third, and last, the usability of Web pages is not fit-for-purpose for expressing HCI performance. HCI, as an engineering discipline (Long and Dowell, 1989), acquires and validates knowledge by research to support HCI design. HCI design (and evaluation) can be understood as the diagnosis of design problems and the prescription of design solutions. HCI, then, attempts to ‘design for effectiveness’, indeed to  ‘design users interacting with computers to perform effective work’, (Dowell and Long, 1989 and 1998). The usability of Web pages fails to express such effectiveness and so is not fit-for-purpose for supporting HCI performance.

In summary, then, the usability of Web pages is neither complete, coherent, nor fit-for-purpose for expressing HCI performance for an HCI discipline intent on designing for effectiveness. Those three criteria need to be met by any alternative approach to HCI, including the domain approach, proposed here.

2.2 Particular Domain Approach to HCI

The domain approach to HCI originates in proposals made by Dowell and Long (1989 and 1998). According to the latter, HCI is an engineering discipline, whose research acquires knowledge to support the solution of the general problem of design, having the particular scope of users interacting with computers to perform effective work. Following Dowell and Long, users interacting with computers constitute an interactive worksystem, composed of at least two separate, but interacting, sub-systems, namely users and computers. Such interactive systems have a domain of application. The domain of work is where work originates, is performed and has its consequences. Goals are allocated to worksystems by organisations. A domain is distinct from, and delimits, a worksystem. Worksystems are designed to perform effective work.

Work is conceived as comprising one or more objects, constituted of attributes, which have values. Goals express a requirement for change in the values of these attributes. Interactive subsystems consist of user behaviours interacting with computer behaviours. These behaviours are supported by mutually exclusive user structures and computer structures and are executed to perform work effectively. Effectiveness is expressed by the concept of performance, that is, how well a system achieves its goals – task quality, and the system costs that are incurred in so doing. Costs are incurred by both the user and the computer, may be physical and mental/cognitive/abstract and can be thought of as workload – for present purposes. The domain approach thus adds the separate concept of work to the system effecting the work. In addition, it applies the concept of work as object/attribute/value transformation (by the worksystem) to performance. It further distinguishes user and computer workloads. The conception is entirely motivated and rationalised by the requirements of an HCI engineering discipline to support design for performance.

This view of HCI is considered to meet the three criteria of completeness, coherence and fitnessfor- purpose, identified earlier and so make good the limitations of the view of HCI as only the usability of Web pages.

First, the view is complete. Performance of an interactive work-system is expressed in terms of task quality – how well the work is carried out and worksystem costs, the workload involved in carrying out the work that well. System workload is composed of user workload and computer workload. If usability is considered to be expressed by user workload, then task quality and computer workload are the additional concepts, required for a complete view of HCI.

Second, the view is coherent. User workload is a property of the user; computer workload is a property of the computer. An increase or decrease in the one can be accompanied by an increase or decrease in the other. In general, the trend in current design is to increase computer workload for a decrease in user workload (given the decreasing cost of computer hardware and software). This current trend in design is well expressed by differentiated worksystem costs, but not by usability, either alone or as a presumed property of the computer. Further, design, training or user selection can all be expected to change user workload, but only design can change computer workload.

Third, and last, task quality and differentiated system workload are fit-for-purpose for expressing HCI performance, because design problems can be diagnosed as ineffective performance, and thus design solutions can be prescribed as effective performance. HCI, as an engineering discipline, can, be supported in its intent ‘to design for effectiveness’. Performance-driven design is taken to be the hall-mark of an engineering discipline (Newman, 1994) In conclusion, then, the domain approach to HCI proposed here meets the three criteria of completeness, coherence and fitness-for-purpose that were used to identify the limitations of the view of HCI as only the usability of Web pages. This domain approach to HCI is now illustrated.

3. Domain Approach Illustrations

Each illustration of the domain approach to HCI describes a domain model and its potential to support design as the diagnosis of design problems and the prescription of design solutions. Only task quality is illustrated as an expression of performance. User and computer costs as workload refer to the worksystem and not the work. User workload is a more coherent expression of usability. Only task quality derives from the domain and so goes beyond usability. Hence, the focus of the illustrations on task quality.

3.1 Implicit Domain Model Illustrations

Modelling the domain of work, following the domain conception of Dowell and Long (1989), necessarily produces an explicit domain model. However, the domain conception can also be used informally to identify domain aspects, implicit in descriptions of interactive worksystems. Such descriptions derive from domain expertise and may be provided by domain experts or worksystem documentation, such as operational procedures, training manuals, etc. Implicit domain models are important for three reasons. First, they are readily available, given access to domain experts and/or worksystem documentation. Second, implicit domain models can be used to support design and evaluation as illustrated here. Third, such models provide the starting point for developing explicit domain models. Implicit domain models use a wide range of representations, including text, diagrams, etc. Note that because the domain models are implicit, these representations include descriptions of the worksystem and other aspects, excluded from the domain conception of Dowell and Long.

3.1.1 Domain of Medical Reception

Domain models of Medical Reception (MR) are implicit in the documentation of medical reception itself (for example, Drury, 1981; Jeffreys and Sachs, 1983, etc.) MR involves combinations of people and office devices (including computers, at least in the UK) in the support of interactions between medical practitioners and their patients in medical general practices. Receptionists are central to the organisation of MR. For example, by 1981 over 70% of medical general practitioners in the UK employed receptionists to operate doctor patient appointment systems, in a group practice (comprising three or more doctors), during surgeries (that is, when the doctors see patients). Most of the receptionists’ time is spent dealing with: requests for surgery appointments and home visits, either by telephone or in person; patients, who turn up with or without appointments; telephone requests to speak to doctors and other medical health workers; registration of new patients; and other patient enquiries and complaints. When making an appointment, the receptionist tries to satisfy the patients’ request for a particular doctor, without keeping them waiting for longer than is acceptable. (At busy times, there may be ten or more patients queuing for an appointment, either on the telephone or in person.) It is difficult, under such conditions, to keep track of the order in which patients present themselves, by telephone or in person, so receptionists can become confused and either have to interrupt and recommence appointment-booking, keeping patients waiting longer than is acceptable or fail to respect the ‘first-come-first-served’ principle underlying ‘fair’ queuing.

Domain models can also be implicit in the transcribed videotapes of MR observational studies (Hill, Long, Smith and Whitefield, 1995). Consider this example of protocol data from the latter study:

Telephone 1    : flash

Receptionist 1 : pick up receiver Telephone
: (over telephone) say: “Can I help”
: pick up green card (?baby registration card) from hatch
: read green card : put green card on desk top
: (over telephone) say: “No, I’m sorry I’ve got nothing”.

Nurse                 : look at nurse appointment book.

Receptionist 1 : (over telephone) say: “Did you say it’s an eye infection?”
: (over telephone) say: “I’ve got literally nothing”.
: (over telephone) say: “All I can offer you is 11:45 this morning – an emergency                                            appointment”.

Receptionist 2 : search prescription – box

Receptionist 1 : (over telephone) say: “Or 10 past 10 with Dr. J tomorrow morning”.

Receptionist 2 : take out prescription

Receptionist 1 : (over telephone) say: “O.K., your name again?”
: write in appointment book
: (over telephone) say: “O.K., 11:45 Dr. I”.
: (over telephone) say: “Thank you. ‘Bye”.
: replace receiver Telephone 1.
: (to hatch) say: “Have you not got a card when you registered the baby

On the basis of MR documentation and the protocol data, the following set of user requirements might be expressed by a receptionist domain expert: “Booking appointments works well, if only one patient is waiting and there are no interruptions. If many patients are waiting, on the telephone and at the hatch (that is, in person), or following an interruption by patient, nurse, doctor or other receptionist, we are often confused as to where we are in the booking procedure. We often take patients out of turn or have to start booking again. This results in us having to work harder than is reasonable and to keep the patients waiting longer than is acceptable to them. Further, we often fail to meet the desires of patients as concerns the particular doctor, the time, or both booked for the appointment. We need some computer help to deal with waiting patients and interruptions, so that our work and the patient waiting times are reduced to acceptable levels”.

It should be noted that neither the MR documentation, the protocol data, nor the receptionist domain expert refer explicitly to the domain, as conceived by Dowell and Long. However, their conception can be used to make explicit the domain, implicit in the description of worksystem behaviours.

For example, following Hill et al (1995), ‘suitability-of-appointment-for-patient’ could be conceived as one attribute of a ‘medical case object’, having values of ‘suitable’ or ‘unsuitable’. For the protocol data, the patient would obviously have preferred an appointment with Dr. J. rather than Dr. I., although an earlier appointment with the latter (for an eye infection) was preferable in this instance to a later appointment with the former. Likewise, an expedition-of-appointment-for-patient’ attribute might have the values ‘timely’ or ‘untimely’. The interruption of a nurse or doctor would have delayed the booking of the appointment described by the protocol data, so rendering it untimely.

Such an informal interpretation of the implicit MR model could be used to support design (and evaluation) in terms of design problem diagnosis and design solution prescription. The design problem and solution would be expressed in terms of the task quality of the MR domain of work.

As concerns design problem diagnosis, the earlier receptionist user requirements identify both acceptable user waiting times and meeting patients’ doctor preferences as candidate problems. In terms of the implicit MR domain model, the design problem could be expressed as the undesired MR task quality of (doctor) suitability and appointment expedition. The problem could be quantified, for example, appointment-booking achieving 90 per cent (doctor) suitability and 90 per cent expedition timelines.

As concerns design problem prescription, computer-based decision support might be provided for the receptionist. The support could record the order of patient arrival and store patients’ doctor preferences, using the latter to prompt preferred appointments. The additional computer support might be expected to increase MR task quality of (doctor) suitability and expedition to a desired level.

The illustration shows how the domain approach goes beyond usability (workload). The latter is implicated in the receptionist user requirements (“This results in us having to work too hard…”). It is also implicated in the additional computer decision support. However, in neither case does usability express MR task quality. (Doctor) suitability and appointment expedition express how well the MR work is performed and not the usability/workload of the MR system performing the work that well. The two concepts, of course, may be intimately related.

3.1.2 Domain of Military Command and Control

The domain of military command and control (C2) carries out two types of work – planning for armed conflict and operating for armed conflict. Nation states pursue their interests by the use of their resources, both human and nonhuman. Users may be political, military, diplomatic, etc. Resources include land, sea, air, space, installations, etc. Domain models of C2 are implicit in military history, doctrine, manuals, domain expert descriptions and military operating procedures. This illustration of military C2 uses the Vincennes incident as described in the official report of the associated enquiry (USDOD, 1988).

The incident took place during the Iran/Iraq war, initially a land battle. Iraq attempted to disrupt Iran’s oil trade, launching air attacks against its oil installations. In response, Iran disrupted oil transport in the Persian Gulf. The US response was to send naval forces to ensure oil supplies. The Iraqi Air Force successfully attacked Iranian forces near the North Persian Gulf. Iranian retaliation of small boat attacks on commercial shipping was expected. The incident occurred when the USS Vincennes – a cruiser – mistakenly shot down civilian Iran Air Flight 655, with total loss of life, while simultaneously engaging a group of Iranian small boats.

The domain model of military C2 is implicit in the expert military descriptions, which appear in the official report. Colbert and Long (1995) summarised the events of the incident as they occur in the report descriptions. Here is a selective extract (the time of the event precedes the events):

0620 The small boats and the Vincennes begin to close ———-.

0647 Flight 655 takes off and is detected by the Vincennes as an ‘unknown, presumed enemy’.

0649 Flight 655 adopts its flight path, which is towards the Vincennes. The Vincennes challenges its air contact (actually Flight 655), but receives no reply. For a moment, Vincennes’ air contact appears electronically to identify itself as a military aircraft (due to freak weather conditions?).

0651 One of the Vincennes’ guns jams when one of the small boats is about to adopt a dangerous position ———. Further challenges to the air contact receive no reply —–. Flight 655’s altitude is misread. The air contact is perceived as diving towards the Vincennes; in fact, it is climbing away from it.

0654 Two surface to air missiles are launched by the Vincennes ——. The missiles destroy Flight 655. All passengers and crew are killed.

The summary does not refer explicitly to the domain of C2, as conceived by Dowell and Long. However, their conception can be used to make the implicit domain model explicit. For example, following Colbert and Long (1995), armed conflict objects could be conceived as constituted of: friends; hostiles; and neutrals objects. All three objects have attributes of vulnerability and involvement. In addition, the friends object has the attribute of power (to realise its interest) and the hostiles object has the attribute of threat (to frustrate the friend’s interest). Attributes could have the values of low, medium and high (or indeed any finer gradations, as required). The C2 expert descriptions of the report can be re-expressed to render the domain model explicit as follows:

0620 The involvement, power and vulnerability of the friend (Vincennes) and the hostile increase again.

0647 The involvement of the neutral (Flight 655) and the friend begins to increase.

0649 The involvement and vulnerability of the neutral, and the friend’s power with respect to it, increases rapidly.

0651 The power of the friend with respect to the neutral temporarily falls sharply——. The involvement and vulnerability of the neutral and the friend’s power with respect to it continue to increase rapidly.

0654 The friend’s power with respect to the neutral is realised, with catastrophic results.

The design problem, as concerns task quality, can be diagnosed as the involvement of the neutral (Flight 655) and the realisation of the friend’s (Vincennes’) power with respect to the neutral. Both the latter’s involvement and the friend’s power with respect to the neutral have undesired high values. As a result, task quality is not as desired, and so constitutes a design problem.

As part of the prescription of a possible design solution, air contact altitude might be more clearly presented, for example, using colour or three-dimensional coding. The Vincennes would then have perceived Flight 655 as climbing away from it and not diving towards it. As a result, the neutral’s involvement would have decreased and the friend’s power would not have been realised. Neutral involvement and friend’s power values, and so task quality, would be as desired.

The illustration shows how the domain approach goes beyond usability. The latter might well be implicated in the Vincennes’ misreading of Flight 655’s altitude, but altitude is co-extensive with neither neutral’s involvement nor friend’s power realisation. Usability tells us how easy or difficult it is to perform work, not how well the work is performed.

3.1.3 Domain of Domestic Energy Management

Energy has multiple uses in the home – heating, cooking, lighting, etc. A major use – in the UK at least – is the heating of the house to maintain the comfort of the people living there. Management consists of the planning and control of the heating to ensure people’s comfort. Management is typically effected by ‘central’ heating systems, consisting of a boiler, which heats water, a tank, which stores it and pipes, which conduct the water to the radiators. Users interact with a controller to set and to programme the heating. The hot water heats the radiators, which in turn heat the house, which determines the comfort of the occupants.

Elsewhere (Stork, Middlemass and Long, 1995), a set of user requirements was elicited for such a central heating system. They are as follows: “The domestic routine of A occasionally requires him to remain at home to work in the mornings, rather than at his office. However, if A leaves after 8 o’clock in the morning, or stays at home to work, then the house is too cold until he turns the gas-powered central heating back on. If he expects to be at home a short time after 8 o’clock, then he often uses the one-hour boost facility on the heating controller to turn the heating back on, which can result in him being too cold, if he is at home longer than expected. A’s ability to work is adversely affected by being cold and having to control the heating. A finds it difficult to plan much in advance, either whether he is staying at home to work, or if he stays, how long he will stay to work. The current gas bill is acceptable, and an increase could be tolerated, although a decrease would be desirable”.

The user requirements make some explicit reference to the domain, for example, “the house is too cold” and “him being too cold”. The references are, however, incomplete. Following Stork and Long (1994), a more complete and explicit domain model might have two main physical objects: A and the house. A has a physical attribute of temperature and an abstract attribute of comfort. The attribute of comfort is related to the attribute of temperature, having a range of acceptable temperatures (for example, 35.75c to 37.5c).

The second physical object is the house, which has physical objects that are the rooms. The rooms have a physical attribute of their temperature and physical objects as radiators. The radiators have a physical attribute of their temperatures. The temperature is related to the temperature of A (an approximately linear relationship) and the temperature of the radiators.

In terms of this explicit domain model, a design problem exists, because the current values of the temperatures of the radiators result in the value of the comfort attribute of A being ‘not comfortable’, at some times, that is, task quality is too low. A design solution prescribed by Stork et al, (1995) involved programming the heating to be on for weekday mornings with an additional remote-heating controller, having an advance button and a bright status light, installed by the front door. The temperatures of the radiators and rooms, and so the house and A, were maintained, such that A’s value remained comfortable, that is, task quality was as desired.

The illustration shows how the domain approach goes beyond usability. Usability/workload is implicated in the user requirements. For example, “A finds it difficult to plan much in advance” and “A’s ability is affected by —— having to control the heating”. However, these aspects do not refer to A’s comfort, the maintenance of which, as desired, constitutes the work of the domestic energy work-system. The design solution likewise. However, the two aspects of performance, and so effectiveness, although related, are differentiated in the domain approach. The rationale is that any design change, such as the addition of a front door controller might affect usability/workload and task quality differentially.

The illustration of how implicit domain models, such as medical reception, military command and control and domestic energy management, when informed by the explicit domain approach of Dowell and Long, can inform design as the diagnosis of design problems of task quality and the prescription of design solutions for task quality, are now complete.

In all cases, the illustrations show how the domain approach goes beyond usability, and given the range of applications, beyond the usability of Web pages. Usability/workload expresses the performance of the work-system; task quality the performance of the work. Together they constitute effectiveness. We turn next to consider how explicit domain models go beyond usability.

3.2 Explicit Domain Model Illustrations

As indicated earlier, applying the domain conception of Dowell and Long necessarily produces an explicit model of the domain of work. Explicit models have also been referenced (Hill et al., 1995; Colbert and Long, 1995; and Stork et al, 1995). The illustrations, which follow show how explicit models support design and in so doing, go beyond usability.

3.2.1 Domain of Amphibious Landing Off-load Planning

As indicated earlier, military C2 carries out planning for armed conflict. Amphibious landing off-load planning is one such type of planning. An amphibious landing may here be considered an attack (that is, armed conflict) against a potentially hostile shore, launched from the sea, and involving air, sea and land forces. It includes the movement ashore of a landing force, embarked on transport ships and naval vessels, by means of amphibious vehicles, landing craft, and helicopters. The landing force arrives ready for combat ashore, and at beaches and landing zones (rather than ports or airfields) (Evans; 1990). Critical for the landing are the off-load plans (see Figure 1), which represents a typical traditional off-load plan as a table (upper figure) and as a gantt chart (lower figure). These plans specify who is to go ashore (left-hand columns of the table) and where and when (right-hand columns of the table). They also specify who is to take the landing force ashore and how the force is to be grouped tactically  (middle columns) .

Figure 1. Typical traditional off-load plan
as a table (upper figure) and as a gantt chart (lower figure)

An explicit domain model of amphibious offload planning (following Colbert and Long, 1995 and Long, 2000) is shown in Figure 2. The plan has both physical and abstract aspects, the latter represented at different levels of description. Plan object attributes comprise: scope; content; and view. Scope has time and object values. Content has the values of conflict object goal states etc, view has the values of table, gantt chart etc.  
Figure 2 Domain model of amphibious off-load planning

Following Long (2000), decision problems, such as off-load planning, require three decision types: solution selection; solution construction; and problem elaboration. Traditional off-load plans (Figure 2) provide only limited support for these decision types. Colbert and Long (1995), using the explicit domain models of armed conflict and off-load planning, diagnosed a design problem, indicating the task quality of such traditional off-load plans are not as desired. The design problem identified, as undesired, aspects of both plan content and plan scope. Concerning plan content, there was a failure to achieve 100% availability by the specified deadlines and rates of lift in terms of man/hours. Concerning plan scope, there were too many errors, related to time and object values.

Figure 3 Re-designed interface for amphibious off-load planning

Again, using the domain models of armed conflict and off-load planning, Colbert and Long proposed a re-designed interface, constituting the prescription of a design solution (shown in Figure 3). The interface not only shows the off-load plan to date, but also: next load pending; next load options; and next load assessments. The next load option exploits the domain model of off-load planning, for example, contents. The next load assessment exploits the domain model of armed conflict, for example, power, safety, cohesion and fatigue and the domain model of off-load planning, for example, lift. The redesign was intended to provide improved support for plan solution selection, solution construction and problem elaboration and so to achieve desired content deadlines, and lift and a reduction in scope errors.

Colbert and Long conducted an evaluation of the re-designed off-load planning interface. The results were as follows. Content deadlines were not met as desired, 91.5% not 100% of content was made available by the deadline. The planned rate of lift, however, was as desired – the 267 men/hour achieved fell between the desired criteria of 255 -275 men/hour. Likewise, scope errors of 0.4 fell below the desired criteria of 1.8. We can conclude on this basis that off-load plan quality was as desired, except as concerns content availability by specified deadline. Further redesign would be required to meet this performance criterion.

However, what this illustration is intended to show is not the success of the redesign itself, but that the domain approach goes beyond usability and that explicit domain models have the potential to support design. First, off-load task quality was expressed as desired content availability and rate of lift, together with scope errors. Elsewhere, Colbert and Long evaluated usability/workload in addition to task quality. Users’ rating of workload was an average 3.0,  which was less than the acceptable criterion of 3.3, and so as desired. Thus, the redesign affected task quality and user costs differentially, so illustrating how the domain approach goes beyond usability. Both are required to express work-system performance and so to support design. Second, both domain models of armed conflict and of off-load planning contributed to the design by suggesting object/attribute/values, such as plan content and scope. Such aspects undoubtedly also contributed to the effects on the performance of task quality and user costs/workload.

3.2.2 Domain of Emergency Management

As in many countries, the UK has a system for the co-ordination of the emergency services in response to disasters, such as explosions, airplane crashes, etc. This system is called the Emergency Management Combined Response System (EMCRS). This system manages, that is, plans and controls, agencies, such as Fire and Police, when they respond to disasters. The UK EMCRS was set up to support better coordination between agencies.

The EMCRS is a command and control system with a three-level structure – operational, tactical and strategic. The system has objectives (embodied in plans), common to all agencies. These objectives are (in descending order of priority): to save life; to prevent escalation of the disaster; to relieve suffering; to safeguard the environment; to protect property; to facilitate criminal investigation and judicial, public, technical or other enquiries; and to restore normality as soon as possible (Home Office, 1994). The individual agencies relate their own individual plans by means of the shared EMCRS plans to interact effectively. Each agency plan specifies a set of functions, for example: Fire Service – rescuing trapped casualties; preventing escalation of the disaster, etc.

An explicit domain model of EMCR (following Hill and Long, 1996 and 2001) is shown in Figure 4. The model comprises physical and abstract domain objects, having attributes and values. For example, the lives object has a survival/evacuee status attribute, having the values of: rescued and not rescued. The disaster scene object has a site attribute, having the values of preserved and unpreserved. And the disaster character object has a fire-type status attribute, having the values of controlled and uncontrolled. The transformation of the object/attribute/values by the EMCRS determines the values of the stability and normality attributes of the disaster object. These values constitute EMCR task quality.

Hill and Long conducted an observational study, involving two stagings of an inter-agency combined response training scenario, which took place at the Home Office Emergency Planning College. The 60 trainees were members of the emergency services and local authority emergency planning officers. The training scenario concerned the derailment of an oil-tanker train on a bridge over a busy main road and a market. The data from the study were used to construct the explicit EMCR domain model. The data can also be used to diagnose a design problem and to prescribe a design solution to illustrate the contribution of the domain model to design.

Figure 4 Domain model of emergency management combined response system

One design problem derived from a behaviour conflict between the Police, Fire and Ambulance Services. The conflict concerned the relations between the Services’ trampling of the disaster site, and the preserved/unpreserved values of the site attribute of the disaster scene object (see earlier and Figure 4). In the training scenario, the Police declare the site a crime scene as vandalism is suspected to be the cause of the tanker-train derailment. As a result of the declaration, the Fire and Ambulance Services are required, by the EMCRS plan, not to trample the scene in the course of their functions (see earlier), as any evidence of a crime must be preserved “to facilitate criminal investigation” (see EMCRS objectives earlier). However, not trampling the site, that is, moving more carefully and so more slowly, delays the rescue of casualties by the Ambulance Service (a primary function) and hinders prevention of escalation of fires by the Fire Service (also a primary function). The Police Service behaviours of preserving the site of the disaster scene conflict with the Ambulance Service behaviours of casualty rescue, and the Fire Service behaviours of fire containment. In terms of EMCR task quality, there is an undesired survivor/evacuee attribute value of not rescued and an undesired fire-type attribute of uncontrolled. Task quality is not as desired and so constitutes a design problem.

The training data do not make clear, whether this design problem is related to planning or control, training or selection of service personnel and so do not suggest a specific design solution prescription. However, it is plausible to conceive that, in any re-design of the EMCRS plans, the disaster scene object site attribute could take on values of high, medium and low preservation. In the training scenario, the train and the track might be adjudged high preservation, the rest of the bridge medium preservation and the main road low preservation value. The additional trampling gradation would be expected to increase the survivor/evacuee attribute value of not rescued for the Ambulance Service and the fire-type attribute of uncontrolled for the Fire Service. The gradation, however, would also be expected to reduce the facilitation of criminal investigation. In other words, the original desired task quality would not be achieved by the re-design, but the task quality would be superior to the actual task quality, observed in the training scenario.

The additional gradation of trampling might make the EMCRS plan more or less usable, implicating more or less workload for the Police, Ambulance and Fire Services. Be that as it may, the illustration makes clear how the EMCR task quality of the domain approach, expressed as survivor/evacuee rescued and the management combined response system fire-type controlled, goes beyond usability and EMCR systems go beyond web pages.

Figure 5. Domain model of reconstructed air traffic management

3.2.3 Domain of Air Traffic Management

Air traffic management (ATM) is here understood as the planning and control of air traffic. Operational ATM manages air traffic, for example, Manchester Ringway Control Centre in the UK. The Centre manages a terminal manoeuvring area with 9 beacons, more than 2 airways, 1 stack, and 2 exits. Its traffic can be characterised as: departing; arriving, overflying; “low and slow”; high-level bunching and so forth. The management involves track and vertical separation rules (Dowell, 1993 and 1998). Planning is supported by “flight strips” and controlling by radar.

Dowell developed an explicit domain model of Manchester Ringway ATM and a simulation thereof – reconstructed air traffic management (RATM) (Dowell, 1993 and 1998). The model, following Long and Timmer (2001), appears in Figure 5. The model comprises physical and abstract objects. Airspace objects include beacons, for example, Alpha, Beta, etc. Aircraft objects include aircraft, for example, TAW, etc. The intersection of airspace objects and aircraft objects results in air traffic event objects with attributes of: position; altitude; speed; heading; and time. Transformation of the values of these attributes result in air traffic vector object attributes of safety, with values of flying time and vertical separation; and expedition, with values of flight progress; fuel use; manoeuvres; and exit variation. Safety and expedition express ATM task quality. Timmer and Long (1996 and 1997) conducted an observational study, using the RATM simulation. An extract from their data, involving the aircraft TAW is shown in Figure 6. The data can be used to diagnose design problems. For example, in the case of TAW, the data indicate its initial state to achieve desired task quality, the separation goal of false (that is, there is desired separation). However, following an operator intervention to improve fuel use, by changing TAW’s altitude (fuel use decreases with increases in altitude), TAW’s actual task quality becomes less than desired, the separation goal false being actually 1830 (that is, less than desired separation). This difference between actual and desired RATM task quality, constitutes the basis for an instance of a RATM design problem.

A further instance of a design problem, diagnosed by means of the domain model (in conjunction with a RATM worksystem model, proposed by Timmer and Long (2002)) appears in Figure 7. The RATM data have been collapsed to highlight the design problem diagnosis. The final state of the data shows actual performance to be less than desired Figure 5. Domain model of reconstructed air traffic management. performance. ZEN is safe, but unexpeditious (excessive progress and fuel use). The likely reason is that earlier, ZEN was expeditious, but unsafe (see domain model attribute values of Figure 4). The operator intervenes to speed up ZEN, so making it safe. However, the operator fails to update the flight progress slip, which continues to show ZEN at its cruising speed of 720. Exceeding the cruising speed results in excessive progress and fuel use. The operator’s earlier recognition of ZEN as an active, safe and unexpeditious (as concerns speed) aircraft, but subsequent failure to increase to slow down ZEN, is assumed to be associated with the decay of the unexpeditious category in memory and the flight progress strip showing (incorrectly) ZEN to be flying at cruising speed.

Figure 6 Reconstructed air traffic management data showing
Domain model attribute states and RATM Worksystem behaviours

Although Timmer and Long do not prescribe a design solution in detail, they do speculate about RATM re-design options. For example, a procedure, requiring immediate flight progress strip update, might solve the design problem. Alternatively, or in addition, an explicit display by the RATM interface of speed, progress and fuel use, which link position, altitude, speed, heading and time attributes to safety and expedition (see Figure 4) might also solve the design problem. If so, RATM actual performance, as concerns task quality, would equal desired performance.

In both cases, the new flight strips procedure and the re-designed interface would have implications for the usability/workload of the RATM system/operator. However, any such implications are not co-extensive with RATM task quality as expedition and safety. The illustration, thus, makes clear how the domain approach, expressed as task quality, goes beyond usability and how the explicit domain model can support design.

Figure 7 Reconstructed Air Traffic Management design problem (collapsed data)

4. Discussion and Conclusions

Both implicit and explicit domain models constitute HCI design knowledge. Such knowledge is intended to support both design and research. HCI design, as shown by the earlier illustrations, can be conceived as the diagnosis of design problems and the prescription of design solutions (Long, 2001). HCI research can be conceived as the acquisition and validation of design knowledge (Long, 2001). Implicit and explicit models are now assessed for their potential to support design and research.

The illustrations from the domains of medical reception, military command and control and domestic energy management all suggest implicit domain models have potential to support design as both diagnosis and prescription. In contrast, these implicit models would appear to have little potential for informing research, other than as a point of departure. Validation of the design knowledge requires its conceptualisation, operationalisation, test and generalisation (Long, 2001). Since these models are implicit, they are not (explicitly) conceptualised. Hence, they cannot be operationalised, tested and generalised, and so validated.

The illustrations from the domains of off-load planning, emergency management and air traffic management all suggest explicit models have potential to support design as diagnosis and prescription. Note, however, that these explicit domain models express only task quality, that is, performance. They do not describe the worksystem behaviours, neither those of the user nor of the computer, determining performance. Additional models are required to represent the latter. The explicit models would also appear to have the potential for informing research

As the domain concepts are explicit, the models meet the requirement of research for conceptualisation. Since conceptualised, the models offer the potential for operationalisation, test and generalisation, and so validation. It is concluded that both implicit and explicit domain models of work have the potential to support design practice as diagnosis and prescription. However, the nature of the support is different. Implicit models support (re-)design of the instance (that is, the particular work system, for example, the medical prescription system of Drs. I. and J. (see earlier) ). In contrast, explicit domain models express the instance as a member of the class (that is, the domain model of air traffic management of which Manchester Ringway is an example (see earlier) . Explicit model support for design is thus more than implicit model support. In contrast, explicit model support is more limited, as it expresses only performance (as task quality) and not the worksystem itself. As concerns research, only explicit domain models have the potential to provide support, because implicit models cannot be validated, in the absence of conceptualisation. Implicit domain models, however, can be the starting point for explicit domain models, developed by research.

It is concluded, generally, that implicit domain models are needed in the shorter term, to inform design of individual work systems. Explicit domain modelling is in its infancy. However, explicit domain models offer the possibility of validation by research and so a better guarantee, in the longer term, of support for both design and research. The importance of explicit domain models needs to be underlined and its further development encouraged. However, both implicit and explicit domain models can be considered as part of the domain approach to HCI. This approach accepts that usability and Web pages are rightly both central to HCI at this time. However, task quality – how well the work is done, derived from domain models, that is, performance, goes beyond usability, conceived either as a property of the computer or as the property of the user, that is, workload. The domain approach also goes beyond the new technology of Web pages. The Web is essentially an information provider with design problems and solutions, concerning access, navigation, etc. It also currently offers, in addition, a range of services, for example, ecommerce, insurance, banking, holiday and flight booking, etc. However, work such as air traffic management, emergency management and amphibious off-load planning are well beyond its current capability. It is concluded, then, that the domain approach goes beyond the usability of Web pages – the thesis of this paper. Finally, the arguments and the illustrations suggest that the domain approach shows promise in its support for an engineering HCI discipline, whose aim is design for effectiveness and whose design knowledge offers a better guarantee of its application.

Acknowledgements:

For conceptual foundations – John Dowell. For the research – all first authors of illustrations. For paper realisation – Doris; Natalie; Rhunette; Beverley; Minna and Jacky.

4. References

BEKKER, M.M. & LONG, J. User Involvement in the Design of Human-Computer Interactions: Some Similarities and Differences between Design Approaches. In Proceedings of the British Computer Society Human-Computer Interaction Specialist Group, 2000.

CARD, S.K., MORAN, T.P. & NEWELL, A., The Psychology of Human-Computer Interaction, New Jersey, Erlbaum, 1983.

COLBERT, M. & LONG, J. Towards the Development of Classes of Interaction: Initial Illustration with reference to Off-Load Planning. In Behaviour & Information Technology, 15 (3), 149-181, 1995.

Home Office. Dealing with Disaster, 2nd ed. Home Office Publication, London, HMSO, 1994.

DOWELL, J. Cognitive engineering and the rationalisation of the flight strip. Unpublished PhD Thesis, University of London, 1993.

DOWELL, J. Formulating the design problem of air traffic management. International Journal of Human-Computer Studies, 49, 743-766, 1998.

DOWELL, J. & LONG, J. Target Paper: Conception of the Cognitive Engineering Design Problem. Ergonomics, 41 (2), 126-139, 1998.

DOWELL, J. & LONG, J.B. Towards a conception for an engineering discipline of human factors. Ergonomics, 32, 1513-35, 1989.

DRURY, M. The medical secretary’s and receptionist’s handbook, 4th ed. London, Bailliere Tindall, 1981. EVANS, M.H.H. Amphibious operations. London, Brassey’s, 1990.

GREEN, W. S., & JORDAN, P. W. Pleasure with products: Beyond usability. London, Taylor & Francis, 2002.

HILL, B. & LONG, J. A Preliminary Model of the Planning and Control of the Combined Response to Disaster. In Proceedings of ECCE 8, 57-62, 1996.

HILL, B. & LONG, J. Performance Modelling in the Domain of Emergency Management. In M.A. Hanson, (Ed.). Contemporary Ergonomics 2001, 165 – 170. London, Taylor & Francis, 2001.

HILL, B., LONG, J., SMITH, W. & WHITEFIELD, A. A Model of Medical Reception – The Planning and Control of Multiple Task Work. Applied Cognitive Psychology, 9 (1), 81-114, 1995.

JEFFREYS, M. & SACHS, H. Rethinking General Practice: Dilemmas in Primary Health Care. London, Tavistock, 1983.

LIGHT, A. & WAKEMAN, J. Beyond the interface: users’ perceptions of interaction and audience on websites. Interacting with Computers, 13(3), 325-351, 2001.

LONG, J. Specifying Relations between Research and the Design of Human-Computer Interactions. International Journal of Human- Computer Studies, 44 (6), 875-920, 1996.

LONG, J. Cognitive Ergonomics: some lessons learned (some remaining). In M.A. Hanson, (Ed.), Contemporary Ergonomics 2001, 263 – 271. London, Taylor and Francis, 200.

LONG, J. & TIMMER, P. Design problems for cognitive ergonomics research: What we can learn from ATM-like micro-worlds. Le Travail Humain, 64(3), 197-222, 2001.

LONG, J.B. Domain Approach for Decision Support for Planning and Control: a Case-Study of Amphibious Landing Off-Load Planning. In Proceedings of APCHI 2000, 2000.

NEWMAN, W. A preliminary analysis of the products of HCI research, using pro forma abstracts. In Proceedings of CHI ’94, 278-284, 1994.

STORK, A. & LONG, J.B. A Specific Planning and Control Design Problem in the Home: Rationale and a Case Study. In Proceedings of the International Working Conference on Home-Oriented Informatics, Telematics and Automation, 1994.

STORK, A., MIDDLEMASS, J. & LONG, J. Applying a Structured Method for Usability Engineering to Domestic Energy Management User Requirements: a Successful Case Study. In Proceedings of HCI’95, 367-385, 1995.

TIMMER, P. & LONG, J. Expressing the effectiveness of planning horizons. Le Travail Humain, 65(2), 103-126, 2002.

TIMMER, P. & LONG, J. Integrating Domain and Worksystem Models: An Illustration from Air Traffic Management. In Proceedings of the Conference on Domain Knowledge in Interactive System Design, 194-207, 1996.

TIMMER, P. & LONG, J. Separating User Knowledge of Domain and Device: A Framework. In Proceedings of HCI’97, 379- 395, 1997.

S. Cummaford and J. Long

Ergonomics & HCI Unit, University College London, 26 Bedford Way,
London, WC1H 0AP

ABSTRACT

Current HCI design knowledge is generally not well specified and thus not validatable. There is a need for more formal design knowledge, which can be validated, such that guarantees may be developed. This need would be met by Engineering Design Principles (EDPs). EDPs support the specification then implementation of a class of design solution for a class of design problem within the scope of the EDP. A conception of the general EDP (GEDP) is proposed here, illustrated with reference to internet-based transaction systems. The GEDP is derived from the conception of the general design problem of an engineering discipline of HCI, and the general design solution, as conceptualised here. This conception of the GEDP, it is argued, is sufficiently formal to support the initial operationalisation of class-level design problems to support the development of class-level EDPs. A strategy for developing class-level design problems is proposed and illustrated with reference to transaction systems. This strategy appears promising for the development of class-level EDPs, supported by empirical guarantees.

Keywords Engineering, design principles, conception, human-computer interaction, performance guarantees, internet transaction systems.

NEED FOR HCI ENGINEERING DESIGN PRINCIPLES

Current best practice in HCI design has produced many technologies that interact with the user to perform effective work. However, the knowledge applied in the design of these technologies is all-toooften not explicitly stated and so not formally conceptualised, although it may be successfully operationalised by designers. Reliance on such ‘craft’ skills militates against the identification, and so the validation, of successful design knowledge and, as a result, its take-up and re-use. The lack of validation and the consequent ineffective development of design knowledge thus leads to slow and inefficient HCI discipline progress (Long 1996).

There is a need for more formal HCI design knowledge, that is, whose conception is coherent, complete and fit-for-purpose, such that guarantees may be developed and ascribed. HCI Engineering Design Principles (EDPs) would meet this need by establishing these guarantees on the basis of analytic and empirical testing, leading to their validation. EDPs are explicit, and so validatable, prescriptions of substantive and methodological design knowledge which, when applied to a design problem within the scope of the principle, would support the specification then implementation of a design solution with guarantee (Dowell & Long, 1989). The development of such knowledge would thus increment the knowledge of an engineering discipline of HCI and would be fit-for-purpose, by providing support with a better guarantee for the practices of solving HCI design problems.

The benefits of such EDPs would be numerous. By employing design knowledge, which has already been shown to support the development of successful design solutions to design problems of a similar type, the need for iterative system development would be reduced. The first iteration would benefit from previous solutions. The re-use of such design knowledge would thus reduce the development time for a technology for which a principle had been formulated. Consequently, the cost of iterative usability testing would be reduced. Furthermore, the structuring of EDPs, at varying levels of generality, would support the re-use of successful design knowledge in new design problems, providing these problems could be characterised similarly at a general level. EDPs would also facilitate design knowledge organisation, by offering a structure with which to taxonomise acquired design knowledge, relating to classes of design problem.

This paper seeks to inform EDP development by conceptualising design knowledge sufficient to prescribe a general design solution (GDS) for a general design problem (GDP). Conceptualisation of such design knowledge, once the general EDP (GEDP) has been established as applicable, is required by EDP development to make explicit the concepts, which need to be instantiated as class-level EDPs (CEDPs). The process of EDP validation comprises four stages: conceptualisation; operationalisation; test; and generalisation (Long, 1996). These four stages support the development of formal, and so validatable, HCI engineering design knowledge. Conceptualisation supports the identification of promising knowledge, which guides instantiation of the GEDP, to produce a CEDP. Instantiated CEDPs may then be operationalised, tested and generalised. These four stages of validation support the ascription of performance guarantees.

This paper begins with an expression of the GDP of an engineering discipline of HCI, as proposed by Dowell and Long (1989; 1998), which is then used to inform the conceptualisation of the GDS, the components of the GEDP and the relationships between the GDP, the GEDP and the GDS. GEDP concepts are illustrated with respect to internet-based transaction processing systems (transaction systems). These transaction systems are in widespread use in electronic commerce, e.g. the order collation and payment system in an internet bookshop. The second section addresses CEDP development issues. Two contrastive strategies are presented, the ‘initial instance’ strategy (Stork & Long, 1994) and the ‘initial class’ strategy, as proposed here. These strategies are assessed and the ‘initial class’ strategy is developed by consideration of class-level design problems (CDPs) and an approach to developing CDPs to support the development of CEDPs. CDPs are illustrated with respect to transaction systems.

PRACTICES SUPPORTED BY EDPS

Long and Dowell (1989) characterise disciplines as comprising: a general problem with a particular scope; practices which provide solutions to the general problem; and knowledge, which supports those practices. EDPs would be the knowledge of an engineering discipline of HCI. They argue that disciplines, and so knowledge, may be characterised by the completeness with which solutions are specified, supporting the practices of implement and test, if incomplete; or specify then implement, if complete. EDPs seek to support the practices of specify then implement by employing formal, and so validatable, design knowledge, which offers complete, coherent, prescriptive design support. The efficacy of prescriptive design knowledge may be seen in more mature engineering disciplines (such as electrical engineering), in which discipline knowledge supports the complete and coherent specification of design solutions prior to implementation.

CONCEPTION OF THE RELATIONS BETWEEN GDP, GEDP AND GDS

To support the development of such EDPs, Dowell & Long (1989) proposed a conception of the GDP of an engineering discipline of HCI in terms of: work; the interactive worksystem; and performance, as task quality and worksystem costs. This conception of the GDP is summarised below to inform the conception of the GDS and GEDP as proposed here. By conceptualising the relations between the GDP, the GEDP and the GDS, coherence and completeness may be assessed. The conception of the GDS is proposed here in terms of: work; the interactive worksystem; and performance, as task quality and worksystem costs. The concepts of the GDP are thus recruited into the conception of the GDS. The concepts of the GDP form criteria for the success of the GDS; use of the same concepts supports assessment of the success of the GDS in satisfying the criteria in the GDP. The conception of the GEDP of an engineering discipline of HCI is proposed here in terms of: scope, comprising a class of users, a class of computers and a class of achievable performances; substantive component; methodological component; and guarantees. These relationships between the concepts of the GEDP and their relationship to concepts contained in the GDP and GDS are discussed later. The discussion of these relationships supports the conceptualisation of complete and coherent relations between the conceptions of the GDP, GEDP and GDS of an engineering discipline of HCI.

GENERAL DESIGN PROBLEM

A design problem 1 expresses an inequality between actual performance (Pa) and desired performance (Pd) of some interactive worksystem (i.e. Pa >Pd with respect to some domain; a successful design solution specifies some interactive worksystem (hereafter worksystem) which achieves the desired performance (Pa = Pd) with respect to some domain. Worksystems comprise users and computers, both of which have structures supporting behaviours. The desired performance is expressed as work, achieved to a desired level of task quality, whilst incurring an acceptable level of costs to the worksystem. Work is expressed as transformations of the attribute values of objects in the domain of the worksystem. These domain transformations (Dowell & Long) inform the conception of the GDS and GEDP and  are in italics on first exposition. Engineering Design Principles are achieved at some desired level of task quality (Tq), whilst incurring some acceptable level of costs to the user (Uc) and the computer (Cc). Attributes are features of domain objects, which afford transformation by the worksystem. The goals of the worksystem are defined as a product goal, which is a transformation of object attribute values. Realisation of a product goal may involve the transformation of many attributes and their values, these transformations being termed task goals. Thus, a product goal may be re-expressed as a task goal structure, which specifies the order and relations between a number of task goals, sufficient to achieve the product goal. As more than one task goal structure may be sufficient to achieve a product goal, it is necessary to distinguish between alternative task goal structures in terms of task quality. Task quality describes the difference between the product goal and the actual transformation specified by a task goal structure. This concept supports evaluation of alternative structures of this type (Dowell & Long, 1989). The worksystem comprises one or more users interacting with one or more computers, each of which is characterised by structures which support behaviours. Desired performance is thus effected by a particular class of user and computer structures, supporting behaviours, which achieve domain transformations, whilst incurring some acceptable level of costs. Worksystem structures are necessary to support behaviour, e.g. knowledge of financial transactions is necessary to support transacting behaviours. Worksystem behaviours involve the transformation of object attributes and their values, e.g. transferring ownership of goods from the vendor to the customer in transaction processing may be expressed as transforming the attribute ‘owner’ from value ‘vendor’ to value ‘customer’ for domain object ‘book x’.

SUBSTANTIVE AND METHODOLOGICAL EDPS

Dowell and Long assert that EDPs may be either substantive or methodological. They state that substantive EDPs “prescribe the features and properties of artefacts, or systems that will, constitute an optimal design solution to a general design problem.” Methodological EDPs “prescribe the methods for solving a general design problem optimally.” (Dowell & Long, 1989). In the conception of EDPs proposed here, substantive and methodological knowledge is assumed to be unitary. The issue of whether optimal solutions are commensurate with EDPs is not addressed, as the guarantees ascribed here would be derived from empirical testing. Thus, these guarantees cannot be claimed optimal, but empirically established.

NEED FOR CONCEPTION OF GDS AND GEDP

The Dowell and Long conception supports the development of EDPs by offering a coherent and complete expression of the GDP to be addressed by the GEDP. However, the relationship between the concepts of the GDP and the concepts of the GEDP are not formally conceptualised. Furthermore, the conception of the GDS is implicit. Thus, the relationship between the GEDP and the GDS is not formally conceptualised. The GDS is conceptualised below, as required to inform the development of EDPs.

CONCEPTION OF GDS

A design solution 2 contains the specification of a worksystem for which the actual performance equals the desired performance (i.e. Pa = Pd), as stated in the design problem. Worksystems comprising users and computers are conceptualised as structures, which support behaviours, which interact to perform work in a domain. Work is expressed as transformations of object attribute values to achieve task goals, which comprise a task goal structure. The quality with which the task goal structure achieves the product goal specified in the GDP is expressed as Tq and the costs incurred by the users and computers are expressed as Uc and Cc.

CONCEPTION OF GEDP

A conception of the GEDP may be considered complete, if its expression is sufficient to identify the applicability of the GEDP, in terms of the GDP. It may be considered coherent, if its expression is sufficient to prescribe design knowledge for specifying the GDS. The ascription of performance guarantees must also be explicitly conceptualised for coherence. The conception of EDPs proposed here includes the concepts of: scope; comprising a class of users, a class of computers and a class of achievable performances; substantive component; methodological component; and performance guarantees. This conception of the GEDP 3, it is argued, is sufficiently coherent and complete to support the initial operationalisation, test and generalisation of EDPs, and so is potentially fit-forpurpose. The concepts of the GEDP are formally conceptualised at a level commensurate with the conception of the GDP. This conception of the GEDP supports carry-forward of coherent and complete design knowledge by supporting the expression of design knowledge, at the appropriate level of generality. This knowledge supports the operationalisation of CEDPs, and its success determines whether it is fit-for-purpose. CEDPs formally specify the relationship between a CDP and a corresponding CDS. The concept of classes supports the representation of design knowledge at different levels of generality. These classes are 2 Concepts from Dowell & Long which have been recruited into the conception of the GDS and the GEDP are in bold italics on first exposition.

Concepts which are novel to this paper are in bold on first exposition. Cummaford and Long 82 identified by reference to the scope of CEDPs – class hierarchies are not intended to constitute a taxonomy of all possible CDPs, but rather only of those CDPs for which a CDS exists. The ultimate success of a CEDP is measured by the performance achieved by the specific technologies supported by its application. The associated guarantees are based on the empirical testing of a series of instances of CEDP application. The coherent and complete conceptualisation which guides this operationalisation may thus be assessed for fitnessfor- purpose. The concepts of the GEDP are informed by the concepts of the GDP and GDS, as the purpose of the GEDP is to identify its applicability to the GDP and prescription of the GDS, if applicable. The GEDP supports the prescription of a GDS, which achieves the desired performance stated in the GDP, if identified as applicable. Scope of the GEDP Specifying criteria for identifying design problems, to which an EDP may be applied, ensures that the knowledge is applied only to those design problems for which it supports the specification of a design solution. Design problems contain not less than one or more users, interacting with not less than one or more computers, and some desired performance. The scope of the GEDP thus comprises a class of users (U-class), a class of computers (C-class) and a class of achievable performances (P-class). If the user and computer in the design problem are members of U-class and C-class respectively, and the desired performance stated in the design problem is a member of P-class, then a design solution would be produced and the actual performance of the solution would equal the desired performance stated in the design problem. The relationship between Uclass, C-class and P-class is developed by empirical testing of the implemented design solutions, produced by CEDP operationalisation. If the user, computer or the desired performance are outside the scope of the principle, then there is no guarantee that the design solution may be specified then implemented. For transaction systems, the criteria for establishing U-class membership would establish the minimum structures and behaviours required for some user, in conjunction with some member of C-class, to achieve a performance which is a member of P-class. Such structures might include knowledge of financial transactions with card-based payment technologies. Supported behaviours might include matching goods descriptions to their shopping goals. The criteria for C-class membership might include structures such as a Virtual Shopping Cart, and supported behaviours, such as real-time processing of payments via the internet. P-class would specify the product goal, e.g. support the exchange of resources for currency, which could be achieved by members of U-class and C-class, to a desired level of task quality, whilst incurring an acceptable level of costs.

Substantive and Methodological Components of GEDP

EDPs contain substantive and methodological design knowledge which may be applied to any design problem within the scope of the EDP. The substantive component is characterised by the conceptualisation of user and computer structures and behaviours, comprising the worksystem, which are present in some instance of the class of users (Uclass) or class of computers (C-class) respectively. The methodological component supports the conceptualisation of a task goal structure, comprising task goals, to be effected by the worksystem, which achieves the product goal stated in P-class. The product goal specifies the work to be effected in the domain by the worksystem, in terms of object attribute value transformations. The structures and behaviours specified in the substantive component are sufficient to achieve the task goal structure specified in the methodological component to an acceptable level of task quality, whilst incurring some acceptable level of costs; where task quality and worksystem costs are members of P-class. This sufficiency is supported by empirical testing of a CEDP, which indicates whether the GEDP is fit-for-purpose. Support for the conceptualisation of user and computer structures and behaviours may take the form of models of interaction between user and computer, expressed as structures supporting behaviours. In the case of transaction systems, a mercantile model of the stages of a transaction to support the specification of behaviours, sufficient to achieve the required domain transformations, would constitute candidate substantive knowledge. One such model (Kalakota & Whinston, 1996) characterises a transaction as: prepurchase determination (information seeking); purchase (agreement of a contract for exchange); and postpurchase interaction (exchange, and evaluation of the product). This model might indicate that information seeking behaviours are necessary to complete a transaction, these behaviours being supported by structures, e.g., purchasing goals, hands. Summary of GEDP Conception This paper proposes a conception of EDPs within which guarantees may be developed for formal HCI engineering design knowledge. The conception proposed thus far comprises the following concepts and relationships: For any design problem {user, computer, Pd} and any EDP {U-class, C-class, P-class, substantive component, methodological component} If user is a member of U-class and computer is a member of C-class,  then user structures and behaviours, and computer structures and behaviours, stated in the substantive component are present. If user structures and behaviours and computer structures and behaviours specified by the substantive component are present then the task goal structure specified by the methodological component is achievable. If the task goal structure specified in the methodological component is effected by a worksystem comprising the structures and behaviours specified in the substantive component then the product goal will be achieved, task quality will be x, user costs will be y and computer costs will be z. If task quality x, user costs y and computer costs z are achieved, Then Pa = Pd. Therefore, Pd is a member of P-class for a worksystem comprising instances of U-class and C-class. This conception may be said to be coherent, as it is based on two relationships: the relationship between the task goal structure and Tq for some product goal, and the relationship between the worksystem structures and behaviours, sufficient to achieve this task goal structure, and Uc and Cc. These relationships may be said to be coherent, as performance is a function of the efficacy with which some task goal structures are achieved by some worksystem structures and behaviours. The GEDP conception may be considered complete, as the concepts of the conception of the engineering discipline of HCI, which inform its development, appear within it. The issue of fitness-for-purpose will be addressed via operationalisation of the conception of the GEDP to inform the development of CEDPs, which may then be tested and generalised.

Validation and Ascription of Guarantees to EDPs

Operationalisation of the GEDP as CEDPs supports empirical testing of the class-level design solutions prescribed. This testing establishes whether the GEDP is fit-for-purpose, that is, it supports the specification then implementation of a design solution which achieves the desired level of performance stated in the design problem. The fourth stage of validation, generalisation, involves establishing the generality of the CEDP. These four stages of validation support the ascription of a guarantee that a worksystem, which performs the task goal structure specified in the methodological component of the EDP, achieves a level of Tq within the P-class stated in the EDP. A second guarantee, that the substantive component supports the specification of a worksystem, which exhibits the structures and behaviours sufficient to achieve the task goal structure, specified in the methodological component, whilst incurring a level of costs within the P-class stated in the EDP, may then be ascribed. A third guarantee, that correct application of the EDP to a design problem, within its scope supports the specification then implementation of a design solution which achieves Pd, is then ascribed on the basis of the former guarantees and further empirical testing. EDPs thus support the specification then implementation of a design solution which achieves the desired performance, if the design problem is within the scope of the EDP.

STRATEGY FOR CEDP DEVELOPMENT

Stork & Long (1994) have applied the conception of HCI to establish a basis for developing EDPs. They have operationalised the general HCI design problem by metricating the concepts, of which it is comprised, to express a specific design problem (SDP) in the domain of domestic energy management. Metrication provides observable and measurable criteria against which to assess performance. The design solutions (SDSs) of such specific design problems and the abstraction of prescriptive knowledge would constitute an EDP – the goal of Stork and Long. However, the operationalisation of an SDP per se does not ensure that a CDP, of which the SDP is an instance, will be found, other than by the assumption of a single domain. This strategy may be termed the ‘initial instance’ strategy, as it seeks to develop CEDPs by specifying design knowledge for SDPs, by means of SDSs, and then generalising across instances. This approach may be contrasted with an alternative ‘initial class’ strategy for CEDP development, as proposed here. The ‘initial class’ strategy supports CEDP development by constructing solutions to CDPs and then construing relevant design knowledge. Because this knowledge is construed at the class level, it is promising for CEDP development. The development of CDPs prior to operationalisation is therefore desirable, as this development constrains the DPs operationalised to those which offer promise in supporting the identification of class-level knowledge. A class may be considered promising for development, if an SDS exists and there are SDPs which share features of the solved SDP. Once such an initial class hypothesis has been formulated, the viability of the class may be assessed by examination of the work performed and the worksystem structures and behaviours sufficient to achieve Pd. If the performance achieved by the worksystem (Pa), specified in the SDS, is similar to the Pd of other SDPs (i.e. Pa = Pd), then the SDPs show promise for CDP development. Cummaford and Long 84

Method for CDP development

The first phase of the ‘initial class’ strategy for CDP development involves identification of an SDP and a corresponding SDS. The second stage involves identifying further SDPs which require a similar Pd, which supports specification of P-class. The user(s) and computer(s) which are to achieve P-class are then assessed for similarities. They may be considered similar, if the user(s) and computer(s), specified in each SDP, comprise sufficient structures supporting sufficient behaviours to achieve P-class. If this sufficiency holds, these user(s) and computer(s) form U-class and C-class of the CDP respectively. In practice, once P-class has been specified, developing CDPs involves identifying U-class and testing instances (members) of this class interacting with instances of C-class. These instances of U-class and C-class are then used to inform the development of a CDS, which achieves P-class. The level of generality should be considered prior to development. Classes which contain very few instances low in the hierarchy contain design knowledge which is very specific. The costs of developing a class at a given level of generality should be balanced against the number of instances to which it may be applied successfully.

Candidate CDPs: Internet-Based Transaction Systems

A class of transaction system design problems has been identified and is presented to illustrate the concept of CDPs. This parent class has three instances (subclasses), each of which is also a class. Each subclass is characterised by P-class, to be achieved by U-class interacting with C-class with respect to some domain. The general characteristics of each of these CDPs are inherited from the parent class. The subclasses of transaction system CDP are: (homogeneous) physical goods (e.g. books); information (e.g. online newspapers); and banking and finance (e.g. loans). These subclasses are abstractions over SDPs, e.g. a design problem concerning a transaction system to support the effective exchange of books for currency in an internet bookshop is an instance of the class of (homogeneous) physical goods. Subclasses may be distinguished by P-class. Differences in the product goal required for each of these subclasses have been identified for physical goods and information sub-classes (Hallam-Baker, 1997). In addition to these two classes, a banking CDP has been developed. The classes differ in the nature of the resources exchanged, the immediacy of the exchange, the possibility for reversing the transaction and the potential loss to the vendor. These differences in product goal indicate that the CDSs for these subclasses will specify classes of worksystem with different task goal structures, as the respective worksystems perform different work. It should be noted that candidate CDPs are identified on the basis of differences in the domain objects transformed by the respective worksystems. Operationalisation of CDPs will support the assessment of whether such differences will result in different worksystem specifications. Each CDS specified is then assessed to establish the task goal structure, sufficient to achieve a level of Tq within P-class. This sufficiency informs the development of the methodological component of the corresponding CEDP. The worksystem structures and behaviours sufficient to achieve the task goal structure, whilst incurring a level of Uc and Cc within P-class, are then construed. These structures and behaviours inform the development of the substantive component of the CEDP. Validation and the ascription of guarantees are based on subsequent operationalisations of the conceptualised CEDPs.

FUTURE RESEARCH

The ‘initial class’ strategy will be operationalised, resulting in CDPs and CDSs for the three transaction system subclasses identified. These CDPs and CDSs will be used to inform CEDP development. The resulting CEDPs will be operationalised and tested to inform the development of guarantees. If these stages of validation are successful, the CEDPs will be generalised and the CEDPs may be considered valid. Abstraction of these subclass CEDPs will be used to produce the parent CEDP, which will then be operationalised, tested and generalised.

ACKNOWLEDGEMENTS

This research associated with this paper was carried out under an EPSRC studentship.

REFERENCES

Dowell, J. and Long, J. B. (1989) Towards a conception for an engineering discipline of human factors. Ergonomics 32, 1513-1536.

Dowell, J. and Long, J. B. (1998) Conception of the cognitive engineering design problem. Ergonomics 41, 2, 126-139.

Hallam-Baker, P. M. (1997) User Interface Requirements for Sale of Goods. Available from: http://www.w3.org/ECommerce/interface.html

Kalakota, R. and Whinston, A. B. (1996) Frontiers of Electronic Commerce. Reading, Mass: Addison-Wesley.

Long, J. B. (1996) Specifying relations between research and the design of human-computer interactions. International Journal of Human Computer Studies, 44, 6, 875-920.

Long, J. B. and Dowell, J. (1989) Conceptions of the discipline of HCI: craft, applied science, and engineering. in Sutcliffe A. and Macaulay L., (Eds.) Proceedings of the Fifth Conference of the BCS HCI SG. Cambridge: Cambridge University Press.

Stork, A. and Long, J. B. (1994) A specific planning and control design problem in the home: rationale and a case study. in Proceedings of the International Working Conference on Home- Oriented Informatics, Telematics and Automation. University of Copenhagen, Denmark. 419-428.

Stephen Cummaford

Ergonomics & HCI Unit, University College London, 26 Bedford Way, London, WC1H OAP

ABSTRACT

There is a need for more formal HCI design knowledge, such that effective design knowledge may be specified in a format which facilitates re-use. A conception of Engineering Design Principles (EDPs) is presented, as a framework within which to systematically relate design knowledge to performance. It is argued that the specification of these relations supports validation, leading to a higher likelihood that application of an EDP to an appropriate design problem will result in a satisfactory design solution. A hierarchy of classes of design problem is presented, and discussed in context of the ongoing research project.

Keywords Cognitive engineering, design, knowledge re-use, performance, principles, validation.

INTRODUCTION

Human-computer interaction practitioners have designed many effective technologies. However, the knowledge applied during development of a successful design solution is not often stated explicitly. Such ‘craft’ skills are difficult to represent explicitly, and as such, cannot easily be tested, and so validated [4]. Such under-specification limits the possibility of successful communication, and so re-use, of effective design knowledge. Engineering Design Principles (EDPs) benefit designers by facilitating more complete communication of designers’ knowledge. Design knowledge applied to produce a successful solution to a design problem can be carried forward to solve similar design problems. Validated design knowledge allows designers to generate new solutions with less prototyping and testing, thus reducing system development costs.

ENGINEERING DESIGN PRINCIPLES

The concept of specifying a complete, class-level, design solution was presented in the Engineering Conception of HCI (HCIe), which was developed within the Ergonomics & HCI Unit, UCL. [3]. The HCIe conception characterises the worksystem, tasks, and interaction structure in terms which support expression of performance, in terms of task quality achieved, and costs incurred whilst performing the work. The focus of my thesis is to develop coherent representations of the relationships between worksystem, tasks, interaction structure, and performance. Such representations support validation of the relationships between a design solution and performance. This conception of principles, as relating to specific classes of design problem, allows design knowledge to be represented at varying levels of generality, as classes may contain instances which are themselves classes.

EDP components

The EDP conception relates the elements of the HCIe conception, by specifying design representations which allow the relationship between the design solution and performance to be measured. An EDP consists of related representations which support coherent specification of a class-level effective design solution. Examples for each concept are provided, from a recent e-store transaction systems case-study, in { } brackets.

Scope: identifies the class of design problems for which this EDP has been validated {physical goods transaction systems supporting order collation and payment, for use by general public without specific training}.
Product goal: the work which is to be done {online order collation and payment }.
Domain model: contains objects which are transformed by the worksystem. Objects have attributes {book, has an owner} and the product goal is expressed in terms of object-attribute value transformations {change book owner from ‘vendor’ to ‘customer’ }.
Task-goal structure: specifies the interactive behaviours to be performed by the worksystem as a hierarchical box diagram, which decomposes the work to be done into behaviour primitives. These behaviours achieve the product goal.
User and computer models: specify the user and computer structures and behaviours which are sufficient to perform the task-goal structure {knowledge of credit card use, shipping address; realtime credit card transaction processing}. These models are used to determine costs incurred by the worksystem, an aspect of performance.
Achievable task quality: statement of how well the work was achieved by previous artefacts designed with this EDP, an aspect of performance { in empirical test, 100% of users completed transaction, no set-up or training required for users with 6+ months internet experience }. Cost matrix an expression of the costs incurred by the worksystem whilst performing the task-goal structure. It is constructed by listing the elements of the user model on the y axis {read, calculate, working memory} and the subtasks of the product goal on the x axis { order item, enter payment details}. The cost matrix is used to record the number of times each user model element is activated during each task. This is assessed empirically, by observation of user trials, and so measures actual user costs, rather than ideal performance. The cost matrix has proved useful in systematic comparison of competing design solutions [2].
Validation:  The EDP conception supports expression of design knowledge by specifying an effective design solution at some level of generality. Performance is measured empirically, by testing with implemented design solutions specified by an EDP. Establishing performance supports assessment of whether the design solution is actually effective, rather the extent of its functionality [5]. The concepts contained in the EDP conception, it is argued [1], support validation by evaluation of the relations between performance and worksystem components performing a task-goal structure.

CLASSES OF DESIGN PROBLEM

A class of transaction system design problems has been identified to inform the development of EDPs. A transaction system is the order collation and payment component of an internet store. The parent class has three instances (subclasses), each of which is also a class. The subclasses of transaction system design problem are: physical goods (e.g. books); online services (e.g. video on demand); and financial (e.g. loans). A case study, addressing the design of physical goods transaction systems, has been conducted. This case study involved specification of a class-level design problem, by abstraction over the requirements for several e-stores, and specification of a design solution, which was then implemented and tested. There were particularly high costs associated with finding the total price for goods, including shipping, in the existing system. This was because the user must enter address and credit card details before the price was calculated. The re-designed system featured two popup menus to select a country and a shipping option, after which the total price could be displayed throughout the interaction, thus reducing user costs. Further details from this case-study are shown on the accompanying poster.

FUTURE RESEARCH

The next stage of the research project is to develop class-based knowledge for the subclass of online services transaction systems. The class hierarchy hypothesised earlier will then be evaluated. The hypothesised class hierarchy will be partially validated, if commonalities are found between the first and EDPs, such that design knowledge may be abstracted and represented at the general class level. The abstracted, class-level knowledge will then be validated by application to further design problems which are instances of the third subclass. The final stage of the project will be to address the usability of the design representations, such that the benefits of using the EDP framework are realised, whilst incurring an acceptable cost of use to designers.

ACKNOWLEDGEMENTS

I would like to thank my supervisor, Professor John Long, for his enthusiastic and insightful input throughout this project. This work was funded by the UK Engineering and Physical Research Council.

REFERENCES

[1] Cumrnaford and Long (1998) Towards a conception of HCI engineering design principles, in T.R.G. Green, L. Bannon, C.P. Warren & J. Buckley (eds.) Proceedings of ECCE-9, the Ninth European Conference on Cognitive Ergonomics. EACE, pp. 79-84.

Steve Cummaford and John Long

Ergonomics & HCI Unit, University College London, 26 Bedford Way, London WC1H 0AP