Steve Cummaford and John Long

Ergonomics & HCI Unit, University College London, 26 Bedford Way, London WC1H 0AP

Stephen Cummaford

Ergonomics & HCI Unit, University College London, 26 Bedford Way, London, WC1H OAP

ABSTRACT

There is a need for more formal HCI design knowledge, such that effective design knowledge may be specified in a format which facilitates re-use. A conception of Engineering Design Principles (EDPs) is presented, as a framework within which to systematically relate design knowledge to performance. It is argued that the specification of these relations supports validation, leading to a higher likelihood that application of an EDP to an appropriate design problem will result in a satisfactory design solution. A hierarchy of classes of design problem is presented, and discussed in context of the ongoing research project.

Keywords Cognitive engineering, design, knowledge re-use, performance, principles, validation.

INTRODUCTION

Human-computer interaction practitioners have designed many effective technologies. However, the knowledge applied during development of a successful design solution is not often stated explicitly. Such ‘craft’ skills are difficult to represent explicitly, and as such, cannot easily be tested, and so validated [4]. Such under-specification limits the possibility of successful communication, and so re-use, of effective design knowledge. Engineering Design Principles (EDPs) benefit designers by facilitating more complete communication of designers’ knowledge. Design knowledge applied to produce a successful solution to a design problem can be carried forward to solve similar design problems. Validated design knowledge allows designers to generate new solutions with less prototyping and testing, thus reducing system development costs.

ENGINEERING DESIGN PRINCIPLES

The concept of specifying a complete, class-level, design solution was presented in the Engineering Conception of HCI (HCIe), which was developed within the Ergonomics & HCI Unit, UCL. [3]. The HCIe conception characterises the worksystem, tasks, and interaction structure in terms which support expression of performance, in terms of task quality achieved, and costs incurred whilst performing the work. The focus of my thesis is to develop coherent representations of the relationships between worksystem, tasks, interaction structure, and performance. Such representations support validation of the relationships between a design solution and performance. This conception of principles, as relating to specific classes of design problem, allows design knowledge to be represented at varying levels of generality, as classes may contain instances which are themselves classes.

EDP components

The EDP conception relates the elements of the HCIe conception, by specifying design representations which allow the relationship between the design solution and performance to be measured. An EDP consists of related representations which support coherent specification of a class-level effective design solution. Examples for each concept are provided, from a recent e-store transaction systems case-study, in { } brackets.

Scope: identifies the class of design problems for which this EDP has been validated {physical goods transaction systems supporting order collation and payment, for use by general public without specific training}.
Product goal: the work which is to be done {online order collation and payment }.
Domain model: contains objects which are transformed by the worksystem. Objects have attributes {book, has an owner} and the product goal is expressed in terms of object-attribute value transformations {change book owner from ‘vendor’ to ‘customer’ }.
Task-goal structure: specifies the interactive behaviours to be performed by the worksystem as a hierarchical box diagram, which decomposes the work to be done into behaviour primitives. These behaviours achieve the product goal.
User and computer models: specify the user and computer structures and behaviours which are sufficient to perform the task-goal structure {knowledge of credit card use, shipping address; realtime credit card transaction processing}. These models are used to determine costs incurred by the worksystem, an aspect of performance.
Achievable task quality: statement of how well the work was achieved by previous artefacts designed with this EDP, an aspect of performance { in empirical test, 100% of users completed transaction, no set-up or training required for users with 6+ months internet experience }. Cost matrix an expression of the costs incurred by the worksystem whilst performing the task-goal structure. It is constructed by listing the elements of the user model on the y axis {read, calculate, working memory} and the subtasks of the product goal on the x axis { order item, enter payment details}. The cost matrix is used to record the number of times each user model element is activated during each task. This is assessed empirically, by observation of user trials, and so measures actual user costs, rather than ideal performance. The cost matrix has proved useful in systematic comparison of competing design solutions [2].
Validation:  The EDP conception supports expression of design knowledge by specifying an effective design solution at some level of generality. Performance is measured empirically, by testing with implemented design solutions specified by an EDP. Establishing performance supports assessment of whether the design solution is actually effective, rather the extent of its functionality [5]. The concepts contained in the EDP conception, it is argued [1], support validation by evaluation of the relations between performance and worksystem components performing a task-goal structure.

CLASSES OF DESIGN PROBLEM

A class of transaction system design problems has been identified to inform the development of EDPs. A transaction system is the order collation and payment component of an internet store. The parent class has three instances (subclasses), each of which is also a class. The subclasses of transaction system design problem are: physical goods (e.g. books); online services (e.g. video on demand); and financial (e.g. loans). A case study, addressing the design of physical goods transaction systems, has been conducted. This case study involved specification of a class-level design problem, by abstraction over the requirements for several e-stores, and specification of a design solution, which was then implemented and tested. There were particularly high costs associated with finding the total price for goods, including shipping, in the existing system. This was because the user must enter address and credit card details before the price was calculated. The re-designed system featured two popup menus to select a country and a shipping option, after which the total price could be displayed throughout the interaction, thus reducing user costs. Further details from this case-study are shown on the accompanying poster.

FUTURE RESEARCH

The next stage of the research project is to develop class-based knowledge for the subclass of online services transaction systems. The class hierarchy hypothesised earlier will then be evaluated. The hypothesised class hierarchy will be partially validated, if commonalities are found between the first and EDPs, such that design knowledge may be abstracted and represented at the general class level. The abstracted, class-level knowledge will then be validated by application to further design problems which are instances of the third subclass. The final stage of the project will be to address the usability of the design representations, such that the benefits of using the EDP framework are realised, whilst incurring an acceptable cost of use to designers.

ACKNOWLEDGEMENTS

I would like to thank my supervisor, Professor John Long, for his enthusiastic and insightful input throughout this project. This work was funded by the UK Engineering and Physical Research Council.

REFERENCES

[1] Cumrnaford and Long (1998) Towards a conception of HCI engineering design principles, in T.R.G. Green, L. Bannon, C.P. Warren & J. Buckley (eds.) Proceedings of ECCE-9, the Ninth European Conference on Cognitive Ergonomics. EACE, pp. 79-84.

S. Cummaford and J. Long

Ergonomics & HCI Unit, University College London, 26 Bedford Way,
London, WC1H 0AP

ABSTRACT

Current HCI design knowledge is generally not well specified and thus not validatable. There is a need for more formal design knowledge, which can be validated, such that guarantees may be developed. This need would be met by Engineering Design Principles (EDPs). EDPs support the specification then implementation of a class of design solution for a class of design problem within the scope of the EDP. A conception of the general EDP (GEDP) is proposed here, illustrated with reference to internet-based transaction systems. The GEDP is derived from the conception of the general design problem of an engineering discipline of HCI, and the general design solution, as conceptualised here. This conception of the GEDP, it is argued, is sufficiently formal to support the initial operationalisation of class-level design problems to support the development of class-level EDPs. A strategy for developing class-level design problems is proposed and illustrated with reference to transaction systems. This strategy appears promising for the development of class-level EDPs, supported by empirical guarantees.

Keywords Engineering, design principles, conception, human-computer interaction, performance guarantees, internet transaction systems.

NEED FOR HCI ENGINEERING DESIGN PRINCIPLES

Current best practice in HCI design has produced many technologies that interact with the user to perform effective work. However, the knowledge applied in the design of these technologies is all-toooften not explicitly stated and so not formally conceptualised, although it may be successfully operationalised by designers. Reliance on such ‘craft’ skills militates against the identification, and so the validation, of successful design knowledge and, as a result, its take-up and re-use. The lack of validation and the consequent ineffective development of design knowledge thus leads to slow and inefficient HCI discipline progress (Long 1996).

There is a need for more formal HCI design knowledge, that is, whose conception is coherent, complete and fit-for-purpose, such that guarantees may be developed and ascribed. HCI Engineering Design Principles (EDPs) would meet this need by establishing these guarantees on the basis of analytic and empirical testing, leading to their validation. EDPs are explicit, and so validatable, prescriptions of substantive and methodological design knowledge which, when applied to a design problem within the scope of the principle, would support the specification then implementation of a design solution with guarantee (Dowell & Long, 1989). The development of such knowledge would thus increment the knowledge of an engineering discipline of HCI and would be fit-for-purpose, by providing support with a better guarantee for the practices of solving HCI design problems.

The benefits of such EDPs would be numerous. By employing design knowledge, which has already been shown to support the development of successful design solutions to design problems of a similar type, the need for iterative system development would be reduced. The first iteration would benefit from previous solutions. The re-use of such design knowledge would thus reduce the development time for a technology for which a principle had been formulated. Consequently, the cost of iterative usability testing would be reduced. Furthermore, the structuring of EDPs, at varying levels of generality, would support the re-use of successful design knowledge in new design problems, providing these problems could be characterised similarly at a general level. EDPs would also facilitate design knowledge organisation, by offering a structure with which to taxonomise acquired design knowledge, relating to classes of design problem.

This paper seeks to inform EDP development by conceptualising design knowledge sufficient to prescribe a general design solution (GDS) for a general design problem (GDP). Conceptualisation of such design knowledge, once the general EDP (GEDP) has been established as applicable, is required by EDP development to make explicit the concepts, which need to be instantiated as class-level EDPs (CEDPs). The process of EDP validation comprises four stages: conceptualisation; operationalisation; test; and generalisation (Long, 1996). These four stages support the development of formal, and so validatable, HCI engineering design knowledge. Conceptualisation supports the identification of promising knowledge, which guides instantiation of the GEDP, to produce a CEDP. Instantiated CEDPs may then be operationalised, tested and generalised. These four stages of validation support the ascription of performance guarantees.

This paper begins with an expression of the GDP of an engineering discipline of HCI, as proposed by Dowell and Long (1989; 1998), which is then used to inform the conceptualisation of the GDS, the components of the GEDP and the relationships between the GDP, the GEDP and the GDS. GEDP concepts are illustrated with respect to internet-based transaction processing systems (transaction systems). These transaction systems are in widespread use in electronic commerce, e.g. the order collation and payment system in an internet bookshop. The second section addresses CEDP development issues. Two contrastive strategies are presented, the ‘initial instance’ strategy (Stork & Long, 1994) and the ‘initial class’ strategy, as proposed here. These strategies are assessed and the ‘initial class’ strategy is developed by consideration of class-level design problems (CDPs) and an approach to developing CDPs to support the development of CEDPs. CDPs are illustrated with respect to transaction systems.

PRACTICES SUPPORTED BY EDPS

Long and Dowell (1989) characterise disciplines as comprising: a general problem with a particular scope; practices which provide solutions to the general problem; and knowledge, which supports those practices. EDPs would be the knowledge of an engineering discipline of HCI. They argue that disciplines, and so knowledge, may be characterised by the completeness with which solutions are specified, supporting the practices of implement and test, if incomplete; or specify then implement, if complete. EDPs seek to support the practices of specify then implement by employing formal, and so validatable, design knowledge, which offers complete, coherent, prescriptive design support. The efficacy of prescriptive design knowledge may be seen in more mature engineering disciplines (such as electrical engineering), in which discipline knowledge supports the complete and coherent specification of design solutions prior to implementation.

CONCEPTION OF THE RELATIONS BETWEEN GDP, GEDP AND GDS

To support the development of such EDPs, Dowell & Long (1989) proposed a conception of the GDP of an engineering discipline of HCI in terms of: work; the interactive worksystem; and performance, as task quality and worksystem costs. This conception of the GDP is summarised below to inform the conception of the GDS and GEDP as proposed here. By conceptualising the relations between the GDP, the GEDP and the GDS, coherence and completeness may be assessed. The conception of the GDS is proposed here in terms of: work; the interactive worksystem; and performance, as task quality and worksystem costs. The concepts of the GDP are thus recruited into the conception of the GDS. The concepts of the GDP form criteria for the success of the GDS; use of the same concepts supports assessment of the success of the GDS in satisfying the criteria in the GDP. The conception of the GEDP of an engineering discipline of HCI is proposed here in terms of: scope, comprising a class of users, a class of computers and a class of achievable performances; substantive component; methodological component; and guarantees. These relationships between the concepts of the GEDP and their relationship to concepts contained in the GDP and GDS are discussed later. The discussion of these relationships supports the conceptualisation of complete and coherent relations between the conceptions of the GDP, GEDP and GDS of an engineering discipline of HCI.

GENERAL DESIGN PROBLEM

A design problem 1 expresses an inequality between actual performance (Pa) and desired performance (Pd) of some interactive worksystem (i.e. Pa >Pd with respect to some domain; a successful design solution specifies some interactive worksystem (hereafter worksystem) which achieves the desired performance (Pa = Pd) with respect to some domain. Worksystems comprise users and computers, both of which have structures supporting behaviours. The desired performance is expressed as work, achieved to a desired level of task quality, whilst incurring an acceptable level of costs to the worksystem. Work is expressed as transformations of the attribute values of objects in the domain of the worksystem. These domain transformations (Dowell & Long) inform the conception of the GDS and GEDP and  are in italics on first exposition. Engineering Design Principles are achieved at some desired level of task quality (Tq), whilst incurring some acceptable level of costs to the user (Uc) and the computer (Cc). Attributes are features of domain objects, which afford transformation by the worksystem. The goals of the worksystem are defined as a product goal, which is a transformation of object attribute values. Realisation of a product goal may involve the transformation of many attributes and their values, these transformations being termed task goals. Thus, a product goal may be re-expressed as a task goal structure, which specifies the order and relations between a number of task goals, sufficient to achieve the product goal. As more than one task goal structure may be sufficient to achieve a product goal, it is necessary to distinguish between alternative task goal structures in terms of task quality. Task quality describes the difference between the product goal and the actual transformation specified by a task goal structure. This concept supports evaluation of alternative structures of this type (Dowell & Long, 1989). The worksystem comprises one or more users interacting with one or more computers, each of which is characterised by structures which support behaviours. Desired performance is thus effected by a particular class of user and computer structures, supporting behaviours, which achieve domain transformations, whilst incurring some acceptable level of costs. Worksystem structures are necessary to support behaviour, e.g. knowledge of financial transactions is necessary to support transacting behaviours. Worksystem behaviours involve the transformation of object attributes and their values, e.g. transferring ownership of goods from the vendor to the customer in transaction processing may be expressed as transforming the attribute ‘owner’ from value ‘vendor’ to value ‘customer’ for domain object ‘book x’.

SUBSTANTIVE AND METHODOLOGICAL EDPS

Dowell and Long assert that EDPs may be either substantive or methodological. They state that substantive EDPs “prescribe the features and properties of artefacts, or systems that will, constitute an optimal design solution to a general design problem.” Methodological EDPs “prescribe the methods for solving a general design problem optimally.” (Dowell & Long, 1989). In the conception of EDPs proposed here, substantive and methodological knowledge is assumed to be unitary. The issue of whether optimal solutions are commensurate with EDPs is not addressed, as the guarantees ascribed here would be derived from empirical testing. Thus, these guarantees cannot be claimed optimal, but empirically established.

NEED FOR CONCEPTION OF GDS AND GEDP

The Dowell and Long conception supports the development of EDPs by offering a coherent and complete expression of the GDP to be addressed by the GEDP. However, the relationship between the concepts of the GDP and the concepts of the GEDP are not formally conceptualised. Furthermore, the conception of the GDS is implicit. Thus, the relationship between the GEDP and the GDS is not formally conceptualised. The GDS is conceptualised below, as required to inform the development of EDPs.

CONCEPTION OF GDS

A design solution 2 contains the specification of a worksystem for which the actual performance equals the desired performance (i.e. Pa = Pd), as stated in the design problem. Worksystems comprising users and computers are conceptualised as structures, which support behaviours, which interact to perform work in a domain. Work is expressed as transformations of object attribute values to achieve task goals, which comprise a task goal structure. The quality with which the task goal structure achieves the product goal specified in the GDP is expressed as Tq and the costs incurred by the users and computers are expressed as Uc and Cc.

CONCEPTION OF GEDP

A conception of the GEDP may be considered complete, if its expression is sufficient to identify the applicability of the GEDP, in terms of the GDP. It may be considered coherent, if its expression is sufficient to prescribe design knowledge for specifying the GDS. The ascription of performance guarantees must also be explicitly conceptualised for coherence. The conception of EDPs proposed here includes the concepts of: scope; comprising a class of users, a class of computers and a class of achievable performances; substantive component; methodological component; and performance guarantees. This conception of the GEDP 3, it is argued, is sufficiently coherent and complete to support the initial operationalisation, test and generalisation of EDPs, and so is potentially fit-forpurpose. The concepts of the GEDP are formally conceptualised at a level commensurate with the conception of the GDP. This conception of the GEDP supports carry-forward of coherent and complete design knowledge by supporting the expression of design knowledge, at the appropriate level of generality. This knowledge supports the operationalisation of CEDPs, and its success determines whether it is fit-for-purpose. CEDPs formally specify the relationship between a CDP and a corresponding CDS. The concept of classes supports the representation of design knowledge at different levels of generality. These classes are 2 Concepts from Dowell & Long which have been recruited into the conception of the GDS and the GEDP are in bold italics on first exposition.

Concepts which are novel to this paper are in bold on first exposition. Cummaford and Long 82 identified by reference to the scope of CEDPs – class hierarchies are not intended to constitute a taxonomy of all possible CDPs, but rather only of those CDPs for which a CDS exists. The ultimate success of a CEDP is measured by the performance achieved by the specific technologies supported by its application. The associated guarantees are based on the empirical testing of a series of instances of CEDP application. The coherent and complete conceptualisation which guides this operationalisation may thus be assessed for fitnessfor- purpose. The concepts of the GEDP are informed by the concepts of the GDP and GDS, as the purpose of the GEDP is to identify its applicability to the GDP and prescription of the GDS, if applicable. The GEDP supports the prescription of a GDS, which achieves the desired performance stated in the GDP, if identified as applicable. Scope of the GEDP Specifying criteria for identifying design problems, to which an EDP may be applied, ensures that the knowledge is applied only to those design problems for which it supports the specification of a design solution. Design problems contain not less than one or more users, interacting with not less than one or more computers, and some desired performance. The scope of the GEDP thus comprises a class of users (U-class), a class of computers (C-class) and a class of achievable performances (P-class). If the user and computer in the design problem are members of U-class and C-class respectively, and the desired performance stated in the design problem is a member of P-class, then a design solution would be produced and the actual performance of the solution would equal the desired performance stated in the design problem. The relationship between Uclass, C-class and P-class is developed by empirical testing of the implemented design solutions, produced by CEDP operationalisation. If the user, computer or the desired performance are outside the scope of the principle, then there is no guarantee that the design solution may be specified then implemented. For transaction systems, the criteria for establishing U-class membership would establish the minimum structures and behaviours required for some user, in conjunction with some member of C-class, to achieve a performance which is a member of P-class. Such structures might include knowledge of financial transactions with card-based payment technologies. Supported behaviours might include matching goods descriptions to their shopping goals. The criteria for C-class membership might include structures such as a Virtual Shopping Cart, and supported behaviours, such as real-time processing of payments via the internet. P-class would specify the product goal, e.g. support the exchange of resources for currency, which could be achieved by members of U-class and C-class, to a desired level of task quality, whilst incurring an acceptable level of costs.

Substantive and Methodological Components of GEDP

EDPs contain substantive and methodological design knowledge which may be applied to any design problem within the scope of the EDP. The substantive component is characterised by the conceptualisation of user and computer structures and behaviours, comprising the worksystem, which are present in some instance of the class of users (Uclass) or class of computers (C-class) respectively. The methodological component supports the conceptualisation of a task goal structure, comprising task goals, to be effected by the worksystem, which achieves the product goal stated in P-class. The product goal specifies the work to be effected in the domain by the worksystem, in terms of object attribute value transformations. The structures and behaviours specified in the substantive component are sufficient to achieve the task goal structure specified in the methodological component to an acceptable level of task quality, whilst incurring some acceptable level of costs; where task quality and worksystem costs are members of P-class. This sufficiency is supported by empirical testing of a CEDP, which indicates whether the GEDP is fit-for-purpose. Support for the conceptualisation of user and computer structures and behaviours may take the form of models of interaction between user and computer, expressed as structures supporting behaviours. In the case of transaction systems, a mercantile model of the stages of a transaction to support the specification of behaviours, sufficient to achieve the required domain transformations, would constitute candidate substantive knowledge. One such model (Kalakota & Whinston, 1996) characterises a transaction as: prepurchase determination (information seeking); purchase (agreement of a contract for exchange); and postpurchase interaction (exchange, and evaluation of the product). This model might indicate that information seeking behaviours are necessary to complete a transaction, these behaviours being supported by structures, e.g., purchasing goals, hands. Summary of GEDP Conception This paper proposes a conception of EDPs within which guarantees may be developed for formal HCI engineering design knowledge. The conception proposed thus far comprises the following concepts and relationships: For any design problem {user, computer, Pd} and any EDP {U-class, C-class, P-class, substantive component, methodological component} If user is a member of U-class and computer is a member of C-class,  then user structures and behaviours, and computer structures and behaviours, stated in the substantive component are present. If user structures and behaviours and computer structures and behaviours specified by the substantive component are present then the task goal structure specified by the methodological component is achievable. If the task goal structure specified in the methodological component is effected by a worksystem comprising the structures and behaviours specified in the substantive component then the product goal will be achieved, task quality will be x, user costs will be y and computer costs will be z. If task quality x, user costs y and computer costs z are achieved, Then Pa = Pd. Therefore, Pd is a member of P-class for a worksystem comprising instances of U-class and C-class. This conception may be said to be coherent, as it is based on two relationships: the relationship between the task goal structure and Tq for some product goal, and the relationship between the worksystem structures and behaviours, sufficient to achieve this task goal structure, and Uc and Cc. These relationships may be said to be coherent, as performance is a function of the efficacy with which some task goal structures are achieved by some worksystem structures and behaviours. The GEDP conception may be considered complete, as the concepts of the conception of the engineering discipline of HCI, which inform its development, appear within it. The issue of fitness-for-purpose will be addressed via operationalisation of the conception of the GEDP to inform the development of CEDPs, which may then be tested and generalised.

Validation and Ascription of Guarantees to EDPs

Operationalisation of the GEDP as CEDPs supports empirical testing of the class-level design solutions prescribed. This testing establishes whether the GEDP is fit-for-purpose, that is, it supports the specification then implementation of a design solution which achieves the desired level of performance stated in the design problem. The fourth stage of validation, generalisation, involves establishing the generality of the CEDP. These four stages of validation support the ascription of a guarantee that a worksystem, which performs the task goal structure specified in the methodological component of the EDP, achieves a level of Tq within the P-class stated in the EDP. A second guarantee, that the substantive component supports the specification of a worksystem, which exhibits the structures and behaviours sufficient to achieve the task goal structure, specified in the methodological component, whilst incurring a level of costs within the P-class stated in the EDP, may then be ascribed. A third guarantee, that correct application of the EDP to a design problem, within its scope supports the specification then implementation of a design solution which achieves Pd, is then ascribed on the basis of the former guarantees and further empirical testing. EDPs thus support the specification then implementation of a design solution which achieves the desired performance, if the design problem is within the scope of the EDP.

STRATEGY FOR CEDP DEVELOPMENT

Stork & Long (1994) have applied the conception of HCI to establish a basis for developing EDPs. They have operationalised the general HCI design problem by metricating the concepts, of which it is comprised, to express a specific design problem (SDP) in the domain of domestic energy management. Metrication provides observable and measurable criteria against which to assess performance. The design solutions (SDSs) of such specific design problems and the abstraction of prescriptive knowledge would constitute an EDP – the goal of Stork and Long. However, the operationalisation of an SDP per se does not ensure that a CDP, of which the SDP is an instance, will be found, other than by the assumption of a single domain. This strategy may be termed the ‘initial instance’ strategy, as it seeks to develop CEDPs by specifying design knowledge for SDPs, by means of SDSs, and then generalising across instances. This approach may be contrasted with an alternative ‘initial class’ strategy for CEDP development, as proposed here. The ‘initial class’ strategy supports CEDP development by constructing solutions to CDPs and then construing relevant design knowledge. Because this knowledge is construed at the class level, it is promising for CEDP development. The development of CDPs prior to operationalisation is therefore desirable, as this development constrains the DPs operationalised to those which offer promise in supporting the identification of class-level knowledge. A class may be considered promising for development, if an SDS exists and there are SDPs which share features of the solved SDP. Once such an initial class hypothesis has been formulated, the viability of the class may be assessed by examination of the work performed and the worksystem structures and behaviours sufficient to achieve Pd. If the performance achieved by the worksystem (Pa), specified in the SDS, is similar to the Pd of other SDPs (i.e. Pa = Pd), then the SDPs show promise for CDP development. Cummaford and Long 84

Method for CDP development

The first phase of the ‘initial class’ strategy for CDP development involves identification of an SDP and a corresponding SDS. The second stage involves identifying further SDPs which require a similar Pd, which supports specification of P-class. The user(s) and computer(s) which are to achieve P-class are then assessed for similarities. They may be considered similar, if the user(s) and computer(s), specified in each SDP, comprise sufficient structures supporting sufficient behaviours to achieve P-class. If this sufficiency holds, these user(s) and computer(s) form U-class and C-class of the CDP respectively. In practice, once P-class has been specified, developing CDPs involves identifying U-class and testing instances (members) of this class interacting with instances of C-class. These instances of U-class and C-class are then used to inform the development of a CDS, which achieves P-class. The level of generality should be considered prior to development. Classes which contain very few instances low in the hierarchy contain design knowledge which is very specific. The costs of developing a class at a given level of generality should be balanced against the number of instances to which it may be applied successfully.

Candidate CDPs: Internet-Based Transaction Systems

A class of transaction system design problems has been identified and is presented to illustrate the concept of CDPs. This parent class has three instances (subclasses), each of which is also a class. Each subclass is characterised by P-class, to be achieved by U-class interacting with C-class with respect to some domain. The general characteristics of each of these CDPs are inherited from the parent class. The subclasses of transaction system CDP are: (homogeneous) physical goods (e.g. books); information (e.g. online newspapers); and banking and finance (e.g. loans). These subclasses are abstractions over SDPs, e.g. a design problem concerning a transaction system to support the effective exchange of books for currency in an internet bookshop is an instance of the class of (homogeneous) physical goods. Subclasses may be distinguished by P-class. Differences in the product goal required for each of these subclasses have been identified for physical goods and information sub-classes (Hallam-Baker, 1997). In addition to these two classes, a banking CDP has been developed. The classes differ in the nature of the resources exchanged, the immediacy of the exchange, the possibility for reversing the transaction and the potential loss to the vendor. These differences in product goal indicate that the CDSs for these subclasses will specify classes of worksystem with different task goal structures, as the respective worksystems perform different work. It should be noted that candidate CDPs are identified on the basis of differences in the domain objects transformed by the respective worksystems. Operationalisation of CDPs will support the assessment of whether such differences will result in different worksystem specifications. Each CDS specified is then assessed to establish the task goal structure, sufficient to achieve a level of Tq within P-class. This sufficiency informs the development of the methodological component of the corresponding CEDP. The worksystem structures and behaviours sufficient to achieve the task goal structure, whilst incurring a level of Uc and Cc within P-class, are then construed. These structures and behaviours inform the development of the substantive component of the CEDP. Validation and the ascription of guarantees are based on subsequent operationalisations of the conceptualised CEDPs.

FUTURE RESEARCH

The ‘initial class’ strategy will be operationalised, resulting in CDPs and CDSs for the three transaction system subclasses identified. These CDPs and CDSs will be used to inform CEDP development. The resulting CEDPs will be operationalised and tested to inform the development of guarantees. If these stages of validation are successful, the CEDPs will be generalised and the CEDPs may be considered valid. Abstraction of these subclass CEDPs will be used to produce the parent CEDP, which will then be operationalised, tested and generalised.

ACKNOWLEDGEMENTS

This research associated with this paper was carried out under an EPSRC studentship.

REFERENCES

Dowell, J. and Long, J. B. (1989) Towards a conception for an engineering discipline of human factors. Ergonomics 32, 1513-1536.

Dowell, J. and Long, J. B. (1998) Conception of the cognitive engineering design problem. Ergonomics 41, 2, 126-139.

Hallam-Baker, P. M. (1997) User Interface Requirements for Sale of Goods. Available from: http://www.w3.org/ECommerce/interface.html

Kalakota, R. and Whinston, A. B. (1996) Frontiers of Electronic Commerce. Reading, Mass: Addison-Wesley.

Long, J. B. (1996) Specifying relations between research and the design of human-computer interactions. International Journal of Human Computer Studies, 44, 6, 875-920.

Long, J. B. and Dowell, J. (1989) Conceptions of the discipline of HCI: craft, applied science, and engineering. in Sutcliffe A. and Macaulay L., (Eds.) Proceedings of the Fifth Conference of the BCS HCI SG. Cambridge: Cambridge University Press.

Stork, A. and Long, J. B. (1994) A specific planning and control design problem in the home: rationale and a case study. in Proceedings of the International Working Conference on Home- Oriented Informatics, Telematics and Automation. University of Copenhagen, Denmark. 419-428.

John Long

Emeritus Professor, Ergonomics and HCI Unit, University College London, 26, Bedford Way, London, WC1H OAP, UK ( Now at UCL Interaction Centre)

Keywords: HCI Engineering, Domain of work, Design problems

Abstract

Usability continues to be central to Human-Computer Interaction (HCI), in spite of recent developments associated with user experience, user pleasure, etc. The Internet continues to dominate the introduction of a wide range of new technologies and rightly has become central to HCI. One current view of HCI, then, is in terms of the usability of Web pages. This view is appealing and there can be no doubt that usable Web pages would be a laudable achievement for HCI This paper supports such a view, but argues that it is too limited. HCI, as an engineering discipline, supports user interaction design, but for effectiveness. The latter necessarily includes usability – how easy it is to use a computer, but also task quality – how well the work is performed by the interactive user-computer worksystem. Easy-to-use computers may produce poor work, hard-to-use computers may produce good work. Usability alone fails to distinguish between these two cases, but differentiation is essential for HCI design and evaluation, because both usability and task quality need to be as desired. This paper proposes a domain approach to make good the limitation of HCI as the usability of Web pages. The domain of work is represented as object, attributes, values and their desired changes – task quality. Implicit domain models (medical reception; military command and control; and domestic energy management) and explicit domain models (off-load planning; emergency management; and air traffic management) are described. Each model is illustrated in terms of its potential support for design as the diagnosis of design problems and the prescription of design solutions, both as they relate to the task quality of the associated domains of work. It is concluded that HCI is indeed more than the usability of Web pages and that the domain approach to making good the limitation shows promise. In particular, implicit domain models are considered to support design in the short-term, while explicit domain models are considered to support design and research in the long-term.

1. Introduction

Usability and Web pages continue to be central to HCI. However, the view that HCI is no more than the usability of Web pages is rejected as too limited. Nevertheless, usable Web pages constitutes a laudable goal for HCI. This paper proposes a domain approach to HCI to make good this limitation.

1.1 Usability and HCI

HCI has evolved considerably since those early days. A number of developments have extended its scope. For example, computer-supported co-operative work (CSCW) emphasises the interdependance between computer users. Ubiquitous computing (Ubicomp) emphasises the variety of computing devices and their locations. Virtual reality (VR) emphasises the importance of ‘presence’ in its simulations. However, in spite of these developments, HCI continues to retain usability as a critical expression of performance.

The concept of usability has also evolved since the early days of HCI. Pleasure has been proposed as a concept, relating people to computing products, which goes beyond usability (Green and Jordan, 2002). User experience claims to be a more complete expression of the same relationship (Light & Wakeman, 2001). However, these new concepts do not claim that usability is unimportant, or even unnecessary, only that there are additional people-product relations. Usability, then, remains central to HCI. It is concluded that in spite of extensions to both the scope of HCI and usability, over the years since their inception, the latter remains central to the former.

1.2 Web Pages and HCI

New technologies continue to characterise and to extend the scope of HCI. Personal computers soon became networked. Broadband communication integrated data processing and communications. Multi-media synthesised text and graphics. Multi-modality combined user inputs from speech, gesture and keyboard. Other new technologies include: distributed networks; mobile computing and telephony; smart buildings; wearable computing; augmented reality, etc. However, some of these new technologies are only in their infancy. Other technologies have proved less successful than originally expected. Yet other technologies have achieved only modest market penetration. In contrast, the new technology, that is the Internet, has continued to develop, offers a number of successful services (although not all are succesful) and has achieved deep market penetration. It is unsurprising, then, that Web pages have rightly become central to HCI.

1.3 Usability, Web Pages and HCI

Given the centrality of both usability (Section 1.1) and Web pages (Section 1.2) to HCI, one might be forgiven for thinking that HCI is essentially the usability of Web pages. This view is appealing and there can be no doubt that usable Web pages would be a laudable and notable achievement for HCI. The notability would derive both from the importance of the achievement for users and from the increased competition of other professionals, such as graphic designers, software engineers, marketeers and financiers to influence the design and evaluation of interactive computer systems. The view that HCI is the usability of Web pages is shared here. However, the proposition is understood as ‘HCI is not less than the usability of Web pages’, but not ‘HCI is only the usability of Web pages’. The scope of HCI must include usability, a primary expression of performance and Web pages, the dominant New Technology. HCI, then, is not limited to, but more than the usability of Web pages. The aim of this paper is to propose an approach – that of the domain – for HCI to go beyond the usability of Web pages.

2. General Domain Approach to HCI

The limitations of the view that HCI is no more than the usability of Web pages are analysed in terms of: completeness; coherence; and fitness-for- purpose. A domain approach is proposed, which makes good these limitations in terms of the three criteria.

2.1 Limited View of HCI

It has been argued earlier that the view of HCI as only the usability of Web pages is too limited. It is worth setting out these limitations in more detail. The latter both provide support for the argument and set up criteria against which to assess alternative approaches to HCI -such as that of the domain, as proposed here. The criteria are: completeness; coherence; and fitness-for-purpose. These criteria derive from the concept of a discipline, whose knowledge, acquired by research, supports its practices (Long, 1996).

First, the usability of Web pages is an incomplete expression of HCI performance. Ideally, usable interactive user-computer systems also achieve their goals. However, HCI evaluations routinely identify usable systems, which fail to achieve their goals and hard-to-use systems, which do achieve their goals. Usability, then, can only constitute one aspect of performance. There also needs to be some expression of the work an interactive system performs and how well it performs that work. The usability of Web pages is too limited a view of performance. If the Web pages provide a service, such as e-commerce, performance needs to express how well the service is effected, for example, goods purchased.

Usability also fails to reflect the computer’s contribution to performing work. For example, a hard-to-use system might be made more easy to use by simplification of the interface. Simplification might be achieved by reducing inappropriate computer behaviours (and so the code supporting them) or increasing appropriate behaviours (and so the code supporting them). In both cases, usability is the same, but the computer contribution differs. This difference should be reflected in a more complete expression of the interactive worksystem’s performance.

Second, the usability of Web pages is an incoherent expression of HCI performance. Usability appears to be a property of the computer. For example, a hard-to-use computer can be re-designed to be easier to use. But computers can also become easier to use by training, learning and practice with no change in the computer itself. Training, learning and practice all modify properties of the user and not the computer. So, usability would be more coherently expressed, if it referred to the user contribution to the interaction. For example, if usability were expressed as effort, or somesuch, the latter might vary as a function of re-design, training, etc. Third, and last, the usability of Web pages is not fit-for-purpose for expressing HCI performance. HCI, as an engineering discipline (Long and Dowell, 1989), acquires and validates knowledge by research to support HCI design. HCI design (and evaluation) can be understood as the diagnosis of design problems and the prescription of design solutions. HCI, then, attempts to ‘design for effectiveness’, indeed to  ‘design users interacting with computers to perform effective work’, (Dowell and Long, 1989 and 1998). The usability of Web pages fails to express such effectiveness and so is not fit-for-purpose for supporting HCI performance.

In summary, then, the usability of Web pages is neither complete, coherent, nor fit-for-purpose for expressing HCI performance for an HCI discipline intent on designing for effectiveness. Those three criteria need to be met by any alternative approach to HCI, including the domain approach, proposed here.

2.2 Particular Domain Approach to HCI

The domain approach to HCI originates in proposals made by Dowell and Long (1989 and 1998). According to the latter, HCI is an engineering discipline, whose research acquires knowledge to support the solution of the general problem of design, having the particular scope of users interacting with computers to perform effective work. Following Dowell and Long, users interacting with computers constitute an interactive worksystem, composed of at least two separate, but interacting, sub-systems, namely users and computers. Such interactive systems have a domain of application. The domain of work is where work originates, is performed and has its consequences. Goals are allocated to worksystems by organisations. A domain is distinct from, and delimits, a worksystem. Worksystems are designed to perform effective work.

Work is conceived as comprising one or more objects, constituted of attributes, which have values. Goals express a requirement for change in the values of these attributes. Interactive subsystems consist of user behaviours interacting with computer behaviours. These behaviours are supported by mutually exclusive user structures and computer structures and are executed to perform work effectively. Effectiveness is expressed by the concept of performance, that is, how well a system achieves its goals – task quality, and the system costs that are incurred in so doing. Costs are incurred by both the user and the computer, may be physical and mental/cognitive/abstract and can be thought of as workload – for present purposes. The domain approach thus adds the separate concept of work to the system effecting the work. In addition, it applies the concept of work as object/attribute/value transformation (by the worksystem) to performance. It further distinguishes user and computer workloads. The conception is entirely motivated and rationalised by the requirements of an HCI engineering discipline to support design for performance.

This view of HCI is considered to meet the three criteria of completeness, coherence and fitnessfor- purpose, identified earlier and so make good the limitations of the view of HCI as only the usability of Web pages.

First, the view is complete. Performance of an interactive work-system is expressed in terms of task quality – how well the work is carried out and worksystem costs, the workload involved in carrying out the work that well. System workload is composed of user workload and computer workload. If usability is considered to be expressed by user workload, then task quality and computer workload are the additional concepts, required for a complete view of HCI.

Second, the view is coherent. User workload is a property of the user; computer workload is a property of the computer. An increase or decrease in the one can be accompanied by an increase or decrease in the other. In general, the trend in current design is to increase computer workload for a decrease in user workload (given the decreasing cost of computer hardware and software). This current trend in design is well expressed by differentiated worksystem costs, but not by usability, either alone or as a presumed property of the computer. Further, design, training or user selection can all be expected to change user workload, but only design can change computer workload.

Third, and last, task quality and differentiated system workload are fit-for-purpose for expressing HCI performance, because design problems can be diagnosed as ineffective performance, and thus design solutions can be prescribed as effective performance. HCI, as an engineering discipline, can, be supported in its intent ‘to design for effectiveness’. Performance-driven design is taken to be the hall-mark of an engineering discipline (Newman, 1994) In conclusion, then, the domain approach to HCI proposed here meets the three criteria of completeness, coherence and fitness-for-purpose that were used to identify the limitations of the view of HCI as only the usability of Web pages. This domain approach to HCI is now illustrated.

3. Domain Approach Illustrations

Each illustration of the domain approach to HCI describes a domain model and its potential to support design as the diagnosis of design problems and the prescription of design solutions. Only task quality is illustrated as an expression of performance. User and computer costs as workload refer to the worksystem and not the work. User workload is a more coherent expression of usability. Only task quality derives from the domain and so goes beyond usability. Hence, the focus of the illustrations on task quality.

3.1 Implicit Domain Model Illustrations

Modelling the domain of work, following the domain conception of Dowell and Long (1989), necessarily produces an explicit domain model. However, the domain conception can also be used informally to identify domain aspects, implicit in descriptions of interactive worksystems. Such descriptions derive from domain expertise and may be provided by domain experts or worksystem documentation, such as operational procedures, training manuals, etc. Implicit domain models are important for three reasons. First, they are readily available, given access to domain experts and/or worksystem documentation. Second, implicit domain models can be used to support design and evaluation as illustrated here. Third, such models provide the starting point for developing explicit domain models. Implicit domain models use a wide range of representations, including text, diagrams, etc. Note that because the domain models are implicit, these representations include descriptions of the worksystem and other aspects, excluded from the domain conception of Dowell and Long.

3.1.1 Domain of Medical Reception

Domain models of Medical Reception (MR) are implicit in the documentation of medical reception itself (for example, Drury, 1981; Jeffreys and Sachs, 1983, etc.) MR involves combinations of people and office devices (including computers, at least in the UK) in the support of interactions between medical practitioners and their patients in medical general practices. Receptionists are central to the organisation of MR. For example, by 1981 over 70% of medical general practitioners in the UK employed receptionists to operate doctor patient appointment systems, in a group practice (comprising three or more doctors), during surgeries (that is, when the doctors see patients). Most of the receptionists’ time is spent dealing with: requests for surgery appointments and home visits, either by telephone or in person; patients, who turn up with or without appointments; telephone requests to speak to doctors and other medical health workers; registration of new patients; and other patient enquiries and complaints. When making an appointment, the receptionist tries to satisfy the patients’ request for a particular doctor, without keeping them waiting for longer than is acceptable. (At busy times, there may be ten or more patients queuing for an appointment, either on the telephone or in person.) It is difficult, under such conditions, to keep track of the order in which patients present themselves, by telephone or in person, so receptionists can become confused and either have to interrupt and recommence appointment-booking, keeping patients waiting longer than is acceptable or fail to respect the ‘first-come-first-served’ principle underlying ‘fair’ queuing.

Domain models can also be implicit in the transcribed videotapes of MR observational studies (Hill, Long, Smith and Whitefield, 1995). Consider this example of protocol data from the latter study:

Telephone 1    : flash

Receptionist 1 : pick up receiver Telephone
: (over telephone) say: “Can I help”
: pick up green card (?baby registration card) from hatch
: read green card : put green card on desk top
: (over telephone) say: “No, I’m sorry I’ve got nothing”.

Nurse                 : look at nurse appointment book.

Receptionist 1 : (over telephone) say: “Did you say it’s an eye infection?”
: (over telephone) say: “I’ve got literally nothing”.
: (over telephone) say: “All I can offer you is 11:45 this morning – an emergency                                            appointment”.

Receptionist 2 : search prescription – box

Receptionist 1 : (over telephone) say: “Or 10 past 10 with Dr. J tomorrow morning”.

Receptionist 2 : take out prescription

Receptionist 1 : (over telephone) say: “O.K., your name again?”
: write in appointment book
: (over telephone) say: “O.K., 11:45 Dr. I”.
: (over telephone) say: “Thank you. ‘Bye”.
: replace receiver Telephone 1.
: (to hatch) say: “Have you not got a card when you registered the baby

On the basis of MR documentation and the protocol data, the following set of user requirements might be expressed by a receptionist domain expert: “Booking appointments works well, if only one patient is waiting and there are no interruptions. If many patients are waiting, on the telephone and at the hatch (that is, in person), or following an interruption by patient, nurse, doctor or other receptionist, we are often confused as to where we are in the booking procedure. We often take patients out of turn or have to start booking again. This results in us having to work harder than is reasonable and to keep the patients waiting longer than is acceptable to them. Further, we often fail to meet the desires of patients as concerns the particular doctor, the time, or both booked for the appointment. We need some computer help to deal with waiting patients and interruptions, so that our work and the patient waiting times are reduced to acceptable levels”.

It should be noted that neither the MR documentation, the protocol data, nor the receptionist domain expert refer explicitly to the domain, as conceived by Dowell and Long. However, their conception can be used to make explicit the domain, implicit in the description of worksystem behaviours.

For example, following Hill et al (1995), ‘suitability-of-appointment-for-patient’ could be conceived as one attribute of a ‘medical case object’, having values of ‘suitable’ or ‘unsuitable’. For the protocol data, the patient would obviously have preferred an appointment with Dr. J. rather than Dr. I., although an earlier appointment with the latter (for an eye infection) was preferable in this instance to a later appointment with the former. Likewise, an expedition-of-appointment-for-patient’ attribute might have the values ‘timely’ or ‘untimely’. The interruption of a nurse or doctor would have delayed the booking of the appointment described by the protocol data, so rendering it untimely.

Such an informal interpretation of the implicit MR model could be used to support design (and evaluation) in terms of design problem diagnosis and design solution prescription. The design problem and solution would be expressed in terms of the task quality of the MR domain of work.

As concerns design problem diagnosis, the earlier receptionist user requirements identify both acceptable user waiting times and meeting patients’ doctor preferences as candidate problems. In terms of the implicit MR domain model, the design problem could be expressed as the undesired MR task quality of (doctor) suitability and appointment expedition. The problem could be quantified, for example, appointment-booking achieving 90 per cent (doctor) suitability and 90 per cent expedition timelines.

As concerns design problem prescription, computer-based decision support might be provided for the receptionist. The support could record the order of patient arrival and store patients’ doctor preferences, using the latter to prompt preferred appointments. The additional computer support might be expected to increase MR task quality of (doctor) suitability and expedition to a desired level.

The illustration shows how the domain approach goes beyond usability (workload). The latter is implicated in the receptionist user requirements (“This results in us having to work too hard…”). It is also implicated in the additional computer decision support. However, in neither case does usability express MR task quality. (Doctor) suitability and appointment expedition express how well the MR work is performed and not the usability/workload of the MR system performing the work that well. The two concepts, of course, may be intimately related.

3.1.2 Domain of Military Command and Control

The domain of military command and control (C2) carries out two types of work – planning for armed conflict and operating for armed conflict. Nation states pursue their interests by the use of their resources, both human and nonhuman. Users may be political, military, diplomatic, etc. Resources include land, sea, air, space, installations, etc. Domain models of C2 are implicit in military history, doctrine, manuals, domain expert descriptions and military operating procedures. This illustration of military C2 uses the Vincennes incident as described in the official report of the associated enquiry (USDOD, 1988).

The incident took place during the Iran/Iraq war, initially a land battle. Iraq attempted to disrupt Iran’s oil trade, launching air attacks against its oil installations. In response, Iran disrupted oil transport in the Persian Gulf. The US response was to send naval forces to ensure oil supplies. The Iraqi Air Force successfully attacked Iranian forces near the North Persian Gulf. Iranian retaliation of small boat attacks on commercial shipping was expected. The incident occurred when the USS Vincennes – a cruiser – mistakenly shot down civilian Iran Air Flight 655, with total loss of life, while simultaneously engaging a group of Iranian small boats.

The domain model of military C2 is implicit in the expert military descriptions, which appear in the official report. Colbert and Long (1995) summarised the events of the incident as they occur in the report descriptions. Here is a selective extract (the time of the event precedes the events):

0620 The small boats and the Vincennes begin to close ———-.

0647 Flight 655 takes off and is detected by the Vincennes as an ‘unknown, presumed enemy’.

0649 Flight 655 adopts its flight path, which is towards the Vincennes. The Vincennes challenges its air contact (actually Flight 655), but receives no reply. For a moment, Vincennes’ air contact appears electronically to identify itself as a military aircraft (due to freak weather conditions?).

0651 One of the Vincennes’ guns jams when one of the small boats is about to adopt a dangerous position ———. Further challenges to the air contact receive no reply —–. Flight 655’s altitude is misread. The air contact is perceived as diving towards the Vincennes; in fact, it is climbing away from it.

0654 Two surface to air missiles are launched by the Vincennes ——. The missiles destroy Flight 655. All passengers and crew are killed.

The summary does not refer explicitly to the domain of C2, as conceived by Dowell and Long. However, their conception can be used to make the implicit domain model explicit. For example, following Colbert and Long (1995), armed conflict objects could be conceived as constituted of: friends; hostiles; and neutrals objects. All three objects have attributes of vulnerability and involvement. In addition, the friends object has the attribute of power (to realise its interest) and the hostiles object has the attribute of threat (to frustrate the friend’s interest). Attributes could have the values of low, medium and high (or indeed any finer gradations, as required). The C2 expert descriptions of the report can be re-expressed to render the domain model explicit as follows:

0620 The involvement, power and vulnerability of the friend (Vincennes) and the hostile increase again.

0647 The involvement of the neutral (Flight 655) and the friend begins to increase.

0649 The involvement and vulnerability of the neutral, and the friend’s power with respect to it, increases rapidly.

0651 The power of the friend with respect to the neutral temporarily falls sharply——. The involvement and vulnerability of the neutral and the friend’s power with respect to it continue to increase rapidly.

0654 The friend’s power with respect to the neutral is realised, with catastrophic results.

The design problem, as concerns task quality, can be diagnosed as the involvement of the neutral (Flight 655) and the realisation of the friend’s (Vincennes’) power with respect to the neutral. Both the latter’s involvement and the friend’s power with respect to the neutral have undesired high values. As a result, task quality is not as desired, and so constitutes a design problem.

As part of the prescription of a possible design solution, air contact altitude might be more clearly presented, for example, using colour or three-dimensional coding. The Vincennes would then have perceived Flight 655 as climbing away from it and not diving towards it. As a result, the neutral’s involvement would have decreased and the friend’s power would not have been realised. Neutral involvement and friend’s power values, and so task quality, would be as desired.

The illustration shows how the domain approach goes beyond usability. The latter might well be implicated in the Vincennes’ misreading of Flight 655’s altitude, but altitude is co-extensive with neither neutral’s involvement nor friend’s power realisation. Usability tells us how easy or difficult it is to perform work, not how well the work is performed.

3.1.3 Domain of Domestic Energy Management

Energy has multiple uses in the home – heating, cooking, lighting, etc. A major use – in the UK at least – is the heating of the house to maintain the comfort of the people living there. Management consists of the planning and control of the heating to ensure people’s comfort. Management is typically effected by ‘central’ heating systems, consisting of a boiler, which heats water, a tank, which stores it and pipes, which conduct the water to the radiators. Users interact with a controller to set and to programme the heating. The hot water heats the radiators, which in turn heat the house, which determines the comfort of the occupants.

Elsewhere (Stork, Middlemass and Long, 1995), a set of user requirements was elicited for such a central heating system. They are as follows: “The domestic routine of A occasionally requires him to remain at home to work in the mornings, rather than at his office. However, if A leaves after 8 o’clock in the morning, or stays at home to work, then the house is too cold until he turns the gas-powered central heating back on. If he expects to be at home a short time after 8 o’clock, then he often uses the one-hour boost facility on the heating controller to turn the heating back on, which can result in him being too cold, if he is at home longer than expected. A’s ability to work is adversely affected by being cold and having to control the heating. A finds it difficult to plan much in advance, either whether he is staying at home to work, or if he stays, how long he will stay to work. The current gas bill is acceptable, and an increase could be tolerated, although a decrease would be desirable”.

The user requirements make some explicit reference to the domain, for example, “the house is too cold” and “him being too cold”. The references are, however, incomplete. Following Stork and Long (1994), a more complete and explicit domain model might have two main physical objects: A and the house. A has a physical attribute of temperature and an abstract attribute of comfort. The attribute of comfort is related to the attribute of temperature, having a range of acceptable temperatures (for example, 35.75c to 37.5c).

The second physical object is the house, which has physical objects that are the rooms. The rooms have a physical attribute of their temperature and physical objects as radiators. The radiators have a physical attribute of their temperatures. The temperature is related to the temperature of A (an approximately linear relationship) and the temperature of the radiators.

In terms of this explicit domain model, a design problem exists, because the current values of the temperatures of the radiators result in the value of the comfort attribute of A being ‘not comfortable’, at some times, that is, task quality is too low. A design solution prescribed by Stork et al, (1995) involved programming the heating to be on for weekday mornings with an additional remote-heating controller, having an advance button and a bright status light, installed by the front door. The temperatures of the radiators and rooms, and so the house and A, were maintained, such that A’s value remained comfortable, that is, task quality was as desired.

The illustration shows how the domain approach goes beyond usability. Usability/workload is implicated in the user requirements. For example, “A finds it difficult to plan much in advance” and “A’s ability is affected by —— having to control the heating”. However, these aspects do not refer to A’s comfort, the maintenance of which, as desired, constitutes the work of the domestic energy work-system. The design solution likewise. However, the two aspects of performance, and so effectiveness, although related, are differentiated in the domain approach. The rationale is that any design change, such as the addition of a front door controller might affect usability/workload and task quality differentially.

The illustration of how implicit domain models, such as medical reception, military command and control and domestic energy management, when informed by the explicit domain approach of Dowell and Long, can inform design as the diagnosis of design problems of task quality and the prescription of design solutions for task quality, are now complete.

In all cases, the illustrations show how the domain approach goes beyond usability, and given the range of applications, beyond the usability of Web pages. Usability/workload expresses the performance of the work-system; task quality the performance of the work. Together they constitute effectiveness. We turn next to consider how explicit domain models go beyond usability.

3.2 Explicit Domain Model Illustrations

As indicated earlier, applying the domain conception of Dowell and Long necessarily produces an explicit model of the domain of work. Explicit models have also been referenced (Hill et al., 1995; Colbert and Long, 1995; and Stork et al, 1995). The illustrations, which follow show how explicit models support design and in so doing, go beyond usability.

3.2.1 Domain of Amphibious Landing Off-load Planning

As indicated earlier, military C2 carries out planning for armed conflict. Amphibious landing off-load planning is one such type of planning. An amphibious landing may here be considered an attack (that is, armed conflict) against a potentially hostile shore, launched from the sea, and involving air, sea and land forces. It includes the movement ashore of a landing force, embarked on transport ships and naval vessels, by means of amphibious vehicles, landing craft, and helicopters. The landing force arrives ready for combat ashore, and at beaches and landing zones (rather than ports or airfields) (Evans; 1990). Critical for the landing are the off-load plans (see Figure 1), which represents a typical traditional off-load plan as a table (upper figure) and as a gantt chart (lower figure). These plans specify who is to go ashore (left-hand columns of the table) and where and when (right-hand columns of the table). They also specify who is to take the landing force ashore and how the force is to be grouped tactically  (middle columns) .

Figure 1. Typical traditional off-load plan
as a table (upper figure) and as a gantt chart (lower figure)

An explicit domain model of amphibious offload planning (following Colbert and Long, 1995 and Long, 2000) is shown in Figure 2. The plan has both physical and abstract aspects, the latter represented at different levels of description. Plan object attributes comprise: scope; content; and view. Scope has time and object values. Content has the values of conflict object goal states etc, view has the values of table, gantt chart etc.  
Figure 2 Domain model of amphibious off-load planning

Following Long (2000), decision problems, such as off-load planning, require three decision types: solution selection; solution construction; and problem elaboration. Traditional off-load plans (Figure 2) provide only limited support for these decision types. Colbert and Long (1995), using the explicit domain models of armed conflict and off-load planning, diagnosed a design problem, indicating the task quality of such traditional off-load plans are not as desired. The design problem identified, as undesired, aspects of both plan content and plan scope. Concerning plan content, there was a failure to achieve 100% availability by the specified deadlines and rates of lift in terms of man/hours. Concerning plan scope, there were too many errors, related to time and object values.

Figure 3 Re-designed interface for amphibious off-load planning

Again, using the domain models of armed conflict and off-load planning, Colbert and Long proposed a re-designed interface, constituting the prescription of a design solution (shown in Figure 3). The interface not only shows the off-load plan to date, but also: next load pending; next load options; and next load assessments. The next load option exploits the domain model of off-load planning, for example, contents. The next load assessment exploits the domain model of armed conflict, for example, power, safety, cohesion and fatigue and the domain model of off-load planning, for example, lift. The redesign was intended to provide improved support for plan solution selection, solution construction and problem elaboration and so to achieve desired content deadlines, and lift and a reduction in scope errors.

Colbert and Long conducted an evaluation of the re-designed off-load planning interface. The results were as follows. Content deadlines were not met as desired, 91.5% not 100% of content was made available by the deadline. The planned rate of lift, however, was as desired – the 267 men/hour achieved fell between the desired criteria of 255 -275 men/hour. Likewise, scope errors of 0.4 fell below the desired criteria of 1.8. We can conclude on this basis that off-load plan quality was as desired, except as concerns content availability by specified deadline. Further redesign would be required to meet this performance criterion.

However, what this illustration is intended to show is not the success of the redesign itself, but that the domain approach goes beyond usability and that explicit domain models have the potential to support design. First, off-load task quality was expressed as desired content availability and rate of lift, together with scope errors. Elsewhere, Colbert and Long evaluated usability/workload in addition to task quality. Users’ rating of workload was an average 3.0,  which was less than the acceptable criterion of 3.3, and so as desired. Thus, the redesign affected task quality and user costs differentially, so illustrating how the domain approach goes beyond usability. Both are required to express work-system performance and so to support design. Second, both domain models of armed conflict and of off-load planning contributed to the design by suggesting object/attribute/values, such as plan content and scope. Such aspects undoubtedly also contributed to the effects on the performance of task quality and user costs/workload.

3.2.2 Domain of Emergency Management

As in many countries, the UK has a system for the co-ordination of the emergency services in response to disasters, such as explosions, airplane crashes, etc. This system is called the Emergency Management Combined Response System (EMCRS). This system manages, that is, plans and controls, agencies, such as Fire and Police, when they respond to disasters. The UK EMCRS was set up to support better coordination between agencies.

The EMCRS is a command and control system with a three-level structure – operational, tactical and strategic. The system has objectives (embodied in plans), common to all agencies. These objectives are (in descending order of priority): to save life; to prevent escalation of the disaster; to relieve suffering; to safeguard the environment; to protect property; to facilitate criminal investigation and judicial, public, technical or other enquiries; and to restore normality as soon as possible (Home Office, 1994). The individual agencies relate their own individual plans by means of the shared EMCRS plans to interact effectively. Each agency plan specifies a set of functions, for example: Fire Service – rescuing trapped casualties; preventing escalation of the disaster, etc.

An explicit domain model of EMCR (following Hill and Long, 1996 and 2001) is shown in Figure 4. The model comprises physical and abstract domain objects, having attributes and values. For example, the lives object has a survival/evacuee status attribute, having the values of: rescued and not rescued. The disaster scene object has a site attribute, having the values of preserved and unpreserved. And the disaster character object has a fire-type status attribute, having the values of controlled and uncontrolled. The transformation of the object/attribute/values by the EMCRS determines the values of the stability and normality attributes of the disaster object. These values constitute EMCR task quality.

Hill and Long conducted an observational study, involving two stagings of an inter-agency combined response training scenario, which took place at the Home Office Emergency Planning College. The 60 trainees were members of the emergency services and local authority emergency planning officers. The training scenario concerned the derailment of an oil-tanker train on a bridge over a busy main road and a market. The data from the study were used to construct the explicit EMCR domain model. The data can also be used to diagnose a design problem and to prescribe a design solution to illustrate the contribution of the domain model to design.

Figure 4 Domain model of emergency management combined response system

One design problem derived from a behaviour conflict between the Police, Fire and Ambulance Services. The conflict concerned the relations between the Services’ trampling of the disaster site, and the preserved/unpreserved values of the site attribute of the disaster scene object (see earlier and Figure 4). In the training scenario, the Police declare the site a crime scene as vandalism is suspected to be the cause of the tanker-train derailment. As a result of the declaration, the Fire and Ambulance Services are required, by the EMCRS plan, not to trample the scene in the course of their functions (see earlier), as any evidence of a crime must be preserved “to facilitate criminal investigation” (see EMCRS objectives earlier). However, not trampling the site, that is, moving more carefully and so more slowly, delays the rescue of casualties by the Ambulance Service (a primary function) and hinders prevention of escalation of fires by the Fire Service (also a primary function). The Police Service behaviours of preserving the site of the disaster scene conflict with the Ambulance Service behaviours of casualty rescue, and the Fire Service behaviours of fire containment. In terms of EMCR task quality, there is an undesired survivor/evacuee attribute value of not rescued and an undesired fire-type attribute of uncontrolled. Task quality is not as desired and so constitutes a design problem.

The training data do not make clear, whether this design problem is related to planning or control, training or selection of service personnel and so do not suggest a specific design solution prescription. However, it is plausible to conceive that, in any re-design of the EMCRS plans, the disaster scene object site attribute could take on values of high, medium and low preservation. In the training scenario, the train and the track might be adjudged high preservation, the rest of the bridge medium preservation and the main road low preservation value. The additional trampling gradation would be expected to increase the survivor/evacuee attribute value of not rescued for the Ambulance Service and the fire-type attribute of uncontrolled for the Fire Service. The gradation, however, would also be expected to reduce the facilitation of criminal investigation. In other words, the original desired task quality would not be achieved by the re-design, but the task quality would be superior to the actual task quality, observed in the training scenario.

The additional gradation of trampling might make the EMCRS plan more or less usable, implicating more or less workload for the Police, Ambulance and Fire Services. Be that as it may, the illustration makes clear how the EMCR task quality of the domain approach, expressed as survivor/evacuee rescued and the management combined response system fire-type controlled, goes beyond usability and EMCR systems go beyond web pages.

Figure 5. Domain model of reconstructed air traffic management

3.2.3 Domain of Air Traffic Management

Air traffic management (ATM) is here understood as the planning and control of air traffic. Operational ATM manages air traffic, for example, Manchester Ringway Control Centre in the UK. The Centre manages a terminal manoeuvring area with 9 beacons, more than 2 airways, 1 stack, and 2 exits. Its traffic can be characterised as: departing; arriving, overflying; “low and slow”; high-level bunching and so forth. The management involves track and vertical separation rules (Dowell, 1993 and 1998). Planning is supported by “flight strips” and controlling by radar.

Dowell developed an explicit domain model of Manchester Ringway ATM and a simulation thereof – reconstructed air traffic management (RATM) (Dowell, 1993 and 1998). The model, following Long and Timmer (2001), appears in Figure 5. The model comprises physical and abstract objects. Airspace objects include beacons, for example, Alpha, Beta, etc. Aircraft objects include aircraft, for example, TAW, etc. The intersection of airspace objects and aircraft objects results in air traffic event objects with attributes of: position; altitude; speed; heading; and time. Transformation of the values of these attributes result in air traffic vector object attributes of safety, with values of flying time and vertical separation; and expedition, with values of flight progress; fuel use; manoeuvres; and exit variation. Safety and expedition express ATM task quality. Timmer and Long (1996 and 1997) conducted an observational study, using the RATM simulation. An extract from their data, involving the aircraft TAW is shown in Figure 6. The data can be used to diagnose design problems. For example, in the case of TAW, the data indicate its initial state to achieve desired task quality, the separation goal of false (that is, there is desired separation). However, following an operator intervention to improve fuel use, by changing TAW’s altitude (fuel use decreases with increases in altitude), TAW’s actual task quality becomes less than desired, the separation goal false being actually 1830 (that is, less than desired separation). This difference between actual and desired RATM task quality, constitutes the basis for an instance of a RATM design problem.

A further instance of a design problem, diagnosed by means of the domain model (in conjunction with a RATM worksystem model, proposed by Timmer and Long (2002)) appears in Figure 7. The RATM data have been collapsed to highlight the design problem diagnosis. The final state of the data shows actual performance to be less than desired Figure 5. Domain model of reconstructed air traffic management. performance. ZEN is safe, but unexpeditious (excessive progress and fuel use). The likely reason is that earlier, ZEN was expeditious, but unsafe (see domain model attribute values of Figure 4). The operator intervenes to speed up ZEN, so making it safe. However, the operator fails to update the flight progress slip, which continues to show ZEN at its cruising speed of 720. Exceeding the cruising speed results in excessive progress and fuel use. The operator’s earlier recognition of ZEN as an active, safe and unexpeditious (as concerns speed) aircraft, but subsequent failure to increase to slow down ZEN, is assumed to be associated with the decay of the unexpeditious category in memory and the flight progress strip showing (incorrectly) ZEN to be flying at cruising speed.

Figure 6 Reconstructed air traffic management data showing
Domain model attribute states and RATM Worksystem behaviours

Although Timmer and Long do not prescribe a design solution in detail, they do speculate about RATM re-design options. For example, a procedure, requiring immediate flight progress strip update, might solve the design problem. Alternatively, or in addition, an explicit display by the RATM interface of speed, progress and fuel use, which link position, altitude, speed, heading and time attributes to safety and expedition (see Figure 4) might also solve the design problem. If so, RATM actual performance, as concerns task quality, would equal desired performance.

In both cases, the new flight strips procedure and the re-designed interface would have implications for the usability/workload of the RATM system/operator. However, any such implications are not co-extensive with RATM task quality as expedition and safety. The illustration, thus, makes clear how the domain approach, expressed as task quality, goes beyond usability and how the explicit domain model can support design.

Figure 7 Reconstructed Air Traffic Management design problem (collapsed data)

4. Discussion and Conclusions

Both implicit and explicit domain models constitute HCI design knowledge. Such knowledge is intended to support both design and research. HCI design, as shown by the earlier illustrations, can be conceived as the diagnosis of design problems and the prescription of design solutions (Long, 2001). HCI research can be conceived as the acquisition and validation of design knowledge (Long, 2001). Implicit and explicit models are now assessed for their potential to support design and research.

The illustrations from the domains of medical reception, military command and control and domestic energy management all suggest implicit domain models have potential to support design as both diagnosis and prescription. In contrast, these implicit models would appear to have little potential for informing research, other than as a point of departure. Validation of the design knowledge requires its conceptualisation, operationalisation, test and generalisation (Long, 2001). Since these models are implicit, they are not (explicitly) conceptualised. Hence, they cannot be operationalised, tested and generalised, and so validated.

The illustrations from the domains of off-load planning, emergency management and air traffic management all suggest explicit models have potential to support design as diagnosis and prescription. Note, however, that these explicit domain models express only task quality, that is, performance. They do not describe the worksystem behaviours, neither those of the user nor of the computer, determining performance. Additional models are required to represent the latter. The explicit models would also appear to have the potential for informing research

As the domain concepts are explicit, the models meet the requirement of research for conceptualisation. Since conceptualised, the models offer the potential for operationalisation, test and generalisation, and so validation. It is concluded that both implicit and explicit domain models of work have the potential to support design practice as diagnosis and prescription. However, the nature of the support is different. Implicit models support (re-)design of the instance (that is, the particular work system, for example, the medical prescription system of Drs. I. and J. (see earlier) ). In contrast, explicit domain models express the instance as a member of the class (that is, the domain model of air traffic management of which Manchester Ringway is an example (see earlier) . Explicit model support for design is thus more than implicit model support. In contrast, explicit model support is more limited, as it expresses only performance (as task quality) and not the worksystem itself. As concerns research, only explicit domain models have the potential to provide support, because implicit models cannot be validated, in the absence of conceptualisation. Implicit domain models, however, can be the starting point for explicit domain models, developed by research.

It is concluded, generally, that implicit domain models are needed in the shorter term, to inform design of individual work systems. Explicit domain modelling is in its infancy. However, explicit domain models offer the possibility of validation by research and so a better guarantee, in the longer term, of support for both design and research. The importance of explicit domain models needs to be underlined and its further development encouraged. However, both implicit and explicit domain models can be considered as part of the domain approach to HCI. This approach accepts that usability and Web pages are rightly both central to HCI at this time. However, task quality – how well the work is done, derived from domain models, that is, performance, goes beyond usability, conceived either as a property of the computer or as the property of the user, that is, workload. The domain approach also goes beyond the new technology of Web pages. The Web is essentially an information provider with design problems and solutions, concerning access, navigation, etc. It also currently offers, in addition, a range of services, for example, ecommerce, insurance, banking, holiday and flight booking, etc. However, work such as air traffic management, emergency management and amphibious off-load planning are well beyond its current capability. It is concluded, then, that the domain approach goes beyond the usability of Web pages – the thesis of this paper. Finally, the arguments and the illustrations suggest that the domain approach shows promise in its support for an engineering HCI discipline, whose aim is design for effectiveness and whose design knowledge offers a better guarantee of its application.

Acknowledgements:

For conceptual foundations – John Dowell. For the research – all first authors of illustrations. For paper realisation – Doris; Natalie; Rhunette; Beverley; Minna and Jacky.

4. References

BEKKER, M.M. & LONG, J. User Involvement in the Design of Human-Computer Interactions: Some Similarities and Differences between Design Approaches. In Proceedings of the British Computer Society Human-Computer Interaction Specialist Group, 2000.

CARD, S.K., MORAN, T.P. & NEWELL, A., The Psychology of Human-Computer Interaction, New Jersey, Erlbaum, 1983.

COLBERT, M. & LONG, J. Towards the Development of Classes of Interaction: Initial Illustration with reference to Off-Load Planning. In Behaviour & Information Technology, 15 (3), 149-181, 1995.

Home Office. Dealing with Disaster, 2nd ed. Home Office Publication, London, HMSO, 1994.

DOWELL, J. Cognitive engineering and the rationalisation of the flight strip. Unpublished PhD Thesis, University of London, 1993.

DOWELL, J. Formulating the design problem of air traffic management. International Journal of Human-Computer Studies, 49, 743-766, 1998.

DOWELL, J. & LONG, J. Target Paper: Conception of the Cognitive Engineering Design Problem. Ergonomics, 41 (2), 126-139, 1998.

DOWELL, J. & LONG, J.B. Towards a conception for an engineering discipline of human factors. Ergonomics, 32, 1513-35, 1989.

DRURY, M. The medical secretary’s and receptionist’s handbook, 4th ed. London, Bailliere Tindall, 1981. EVANS, M.H.H. Amphibious operations. London, Brassey’s, 1990.

GREEN, W. S., & JORDAN, P. W. Pleasure with products: Beyond usability. London, Taylor & Francis, 2002.

HILL, B. & LONG, J. A Preliminary Model of the Planning and Control of the Combined Response to Disaster. In Proceedings of ECCE 8, 57-62, 1996.

HILL, B. & LONG, J. Performance Modelling in the Domain of Emergency Management. In M.A. Hanson, (Ed.). Contemporary Ergonomics 2001, 165 – 170. London, Taylor & Francis, 2001.

HILL, B., LONG, J., SMITH, W. & WHITEFIELD, A. A Model of Medical Reception – The Planning and Control of Multiple Task Work. Applied Cognitive Psychology, 9 (1), 81-114, 1995.

JEFFREYS, M. & SACHS, H. Rethinking General Practice: Dilemmas in Primary Health Care. London, Tavistock, 1983.

LIGHT, A. & WAKEMAN, J. Beyond the interface: users’ perceptions of interaction and audience on websites. Interacting with Computers, 13(3), 325-351, 2001.

LONG, J. Specifying Relations between Research and the Design of Human-Computer Interactions. International Journal of Human- Computer Studies, 44 (6), 875-920, 1996.

LONG, J. Cognitive Ergonomics: some lessons learned (some remaining). In M.A. Hanson, (Ed.), Contemporary Ergonomics 2001, 263 – 271. London, Taylor and Francis, 200.

LONG, J. & TIMMER, P. Design problems for cognitive ergonomics research: What we can learn from ATM-like micro-worlds. Le Travail Humain, 64(3), 197-222, 2001.

LONG, J.B. Domain Approach for Decision Support for Planning and Control: a Case-Study of Amphibious Landing Off-Load Planning. In Proceedings of APCHI 2000, 2000.

NEWMAN, W. A preliminary analysis of the products of HCI research, using pro forma abstracts. In Proceedings of CHI ’94, 278-284, 1994.

STORK, A. & LONG, J.B. A Specific Planning and Control Design Problem in the Home: Rationale and a Case Study. In Proceedings of the International Working Conference on Home-Oriented Informatics, Telematics and Automation, 1994.

STORK, A., MIDDLEMASS, J. & LONG, J. Applying a Structured Method for Usability Engineering to Domestic Energy Management User Requirements: a Successful Case Study. In Proceedings of HCI’95, 367-385, 1995.

TIMMER, P. & LONG, J. Expressing the effectiveness of planning horizons. Le Travail Humain, 65(2), 103-126, 2002.

TIMMER, P. & LONG, J. Integrating Domain and Worksystem Models: An Illustration from Air Traffic Management. In Proceedings of the Conference on Domain Knowledge in Interactive System Design, 194-207, 1996.

TIMMER, P. & LONG, J. Separating User Knowledge of Domain and Device: A Framework. In Proceedings of HCI’97, 379- 395, 1997.

P. TIMMER and J LONG

Ergonomics & HCI Unit, University College London, 26 Bedford Way, London, WC1H 0AP

Expression de l’efficacité des horizons de planification

RÉSUMÉ

L’évolution technologique est souvent motivée par des ‘problèmes’. Pourtant, l’expression de ces problèmes en termes de performance des systèmes de travail n’est souvent qu’anecdotique ou implicite. Cette recherche propose une méthode explicite pour exprimer l’efficacité d’un système de travail. La méthode est illustrée sur un système de travail de gestion du transport. L’intérêt particulier de ce domaine concerne la façon dont l’interaction opérateur-technologie soutient efficacement la planification à l’avance (sous la forme d’un horizon de planification). La méthode est composée de quatre étapes. Premièrement, le comportement de planification à l’avance est conceptualisé. Un aspect critique de la méthode est la ‘Théorie des Horizons de Planification de l’Opérateur’ ainsi que ‘l’extension’ et ‘l’opportunité’ d’horizons de planification particuliers. Deuxièmement,  le domaine de travail est modélisé, afin d’établir la qualité du travail effectué par le système de travail. Troisièmement, les comportements qui soutiennent la planification efficace sont modélisés. Finalement, une comparaison est faite entre la qualité réelle du travail effectué et la qualité désirée. Si la performance tombe en-dessous d’un niveau désiré, les comportements du système de travail contribuant à l’inefficacité sont analysés. Si une planification inefficace est identifiée (c’est-à-dire un problème), la méthode soutient la recherche des origines du problème, ainsi que la construction d’une théorie causale. Bien que l’illustration ne porte que sur la planification d’un système de travail de gestion du transport, les étapes de la méthode sont proposées pour soutenir plus généralement l’expression de l’efficacité des systèmes ou autres.

Mots-clés:            Horizon de Planification; Efficacité d’un système de travail;  Contrôle du trafic aérien.

I.         INTRODUCTION

I. 1.            MOTIVATION AND OVERVIEW

This paper describes a method that enables the expression of: a) the plans of a process operator, and how far into the future those plans extend; and b) an assessment of how adequate those plans are, for ensuring that work goals are attained. The method is illustrated using an Air Traffic Management microworld. The need to express operator plans, and their extension, arises when technologies are being developed to support the planning of interventions with a dynamic evolving process; domains where process evolution needs to be anticipated, and process interventions need to be planned to address anticipated process states. By associating each individual plan with an assessment of adequacy, design problems may be characterised, problems that may be alleviated by technological support. Where the expression of plans shows those plans to be inadequate for attaining management goals, then new technologies can be proposed that may result in more effective operator planning behaviour.

In general, the evolution of a human-technology worksystem may either be problem-driven or technology-driven (Woods & Roth, 1988). Problem-driven evolution arises when a specific problem is attributed to the design of a technology, and the technology is then redesigned (or replaced) to remedy that specific problem. Such problems are frequently expressed by operators in the form of a subjective, experience-based report or anecdote, which may or may not lead to some in-depth analysis of the problem. Alternatively, technology-driven evolution may arise when a technology is redeveloped (or replaced) simply because redevelopment (or replacement) is possible, that is to say, not in the light of a specific ‘problem’ (such as inadequate planning). As the aim of the method proposed here is to relate operator planning (and plans) to effective or ineffective performance (intervention outcomes), and given that ineffective performance is considered a problem in need of a (technological) solution, the method may be understood as a system development tool, to support problem-driven evolution at the early stage of ‘problem formulation’ (Rasmussen, 1992; Woods & Roth, 1988). The method therefore assists in the process of progressing from operator anecdotes about problems, to a structured and more formal analysis and expression of those problems. The need to express planning problems, and thereby evolve technological solutions, arises in traditional process control domains such as Air Traffic Management (ATM), Railway Signal Management (RSM), nuclear power generation, and so forth. Throughout the paper, method application will be illustrated with reference to an ATM-like microworld of sufficient complexity to demonstrate the phenomena of concern to the method.

Before continuing, a definition of the term ‘design problem’ is offered.  A design problem is considered to exist, and therefore acts as a motivator for problem-driven evolution, when a desired level of performance is not being achieved by a human-technology worksystem. That worksystem may then be termed ‘ineffective’, as desired performance is somehow compromised. Where a design problem is believed to exist, a method is needed for expressing that problem (its severity, frequency, behavioural causes, etc.) in a manner that will contribute towards its solution. Outlining such a method is the aim of this paper. Being able to express worksystem ineffectiveness is particularly valuable when problem-driven evolution occurs within safety critical domains. For domains where the consequences of ineffective human-technology interaction have serious potential outcomes for human life, being able to express whether or not particular interactions are effective, supports reasoning about the adequacy of the technology in question. Within the context of Cognitive Engineering, such expression and reasoning is termed ‘diagnosis’ (Dowell & Long, 1998; Rasmussen, 1986) and, as shall be illustrated in the paper, can support formulation of the design problem that a re-designed technology should solve (Dowell, 1998).

The emphasis on ‘design’ and ‘design problems (and solutions)’ characterises the present approach as one of engineering, that is, contributing to the design of effective worksystems (Amalberti & Deblon, 1992; Dowell & Long, 1998; Flach, 1998; Hollnagel, 1998; Rasmussen, 1986; Reason, 1998; Vincente, 1998; Woods, 1998), rather than as one of science, that is, understanding the phenomena associated with worksystems and their behaviours (Barnard, 1991; Meyer & Kieras, 1999). Within the design approach, the present can be more precisely characterised as ‘design for effectiveness’, seeking to use the design primitives of ‘work’, ‘worksystem’, and ‘performance’ to motivate the acquisition and validation of design knowledge, to diagnose and solve design problems (in contrast, for example, to ‘human performance’, expressed as some form of speed and errors (Reason, 1998)).

Problems of interaction arise with many different types of technology. Here, particular consideration is given to problems arising with technologies that support planning ahead. In presenting the method, at the first stage, a theory is presented, for use when modelling how far ahead an operator plans. In contrast to other work on planning (Amalberti & Deblon, 1992; Boudes & Cellier, 1997; O’Hara & Payne, 1999), this theory makes reference to a plan’s ‘extension’, and its ‘adequacy’ (how well the plan ensures work goals are achieved). A method for expressing the effectiveness (or otherwise) of a plan necessarily requires consideration of the plan’s extension (how far into the future the plan accounts for an intervention), and whether or not the plan is adequate to ensure goals are attained.

The aim of this paper is therefore to propose a method for expressing the adequacy of operator planning, with special interest in capturing instances of ineffective planning (diagnosing design problems). The presented method comprises four stages. In the first stage, planning behaviours of interest are conceptualised by a domain-independent Theory of the Operator Planning Horizon, and requirements for modelling an operator’s planning horizon are generated from that theory. During the second stage, the work carried out by the human-technology (planning) worksystem is considered, and captured by a model of the domain. Here, the ATM-like microworld is described. In the third stage, models of operator planning horizons are constructed. Finally, these two types of model, of the domain (work) and of planning horizons (plans and planning behaviour), are considered alongside each another, and diagnostic assessments are made as to whether or not the plans formed were effective (i.e., whether or not there is a design problem concerning planning). When a problem is identified, a causal theory is constructed to relate how operator-technology interactions contributed to the occurrence of that problem. The four stages together constitute a method that helps to establish an explicit link between planning behaviours and the quality (effectiveness) of work executed by the worksystem, a relation frequently addressed in only an implicit fashion (Boudes & Cellier, 1997). The method therefore addresses the construction and use of models during the design process, a tradition well established within HCI research (Blandford & Young (1993); Card, Moran & Newell (1983)). In the analysis of operator-technology interactions, ‘effectiveness’ is considered a primitive (ontological) entity, alongside: human (planning) behaviour; technological behaviour (and how it supports planning); and details of the work performed (Dowell & Long, 1998). The paper is structured to reflect the stages of the method outlined above, and concludes with a discussion of the method.

I.2            ATM-LIKE MICROWORLD

For the purposes of method illustration, a laboratory-based ATM-like microworld was used. The microworld was constructed on the basis of an observational field study at Ringway Control Centre in Manchester, and possesses selected characteristics of the operational system that make it suitable for illustrating the method (Dowell, 1998; Long & Timmer, 2001). The domain is dynamic and imposes a significant planning burden upon the operator, who must anticipate the future state of air traffic, establish goals, collect and integrate data from different sources, plan tactical interventions to two aircraft variables (altitude and speed), and finally intervene with aircraft. Aircraft response to worksystem intervention is time-lagged, and the quality of aircraft management, in terms of aircraft safety and expedition (fuel use, progress to plan, minimum interventions), is calculated by the simulation software, following an interactive scenario.

The managed domain, containing aircraft, beacons, airways and so forth (Dowell, 1998), is generated by the simulation software and displayed on a computer-based radar. Paper-based flight progress strips are used, as observed in the operational worksystem, and document aircraft entry and exit states, aircraft identity and route. The strips are annotated after each intervention with details of aircraft state changes. Interventions to aircraft altitude and speed can be made via menu selection on the radar, rather than by ground-to-air radio. Through interaction with these technologies, the operator’s task is to ensure the safe and expeditious management of aircraft across the sector. Aircraft traverse a sector along airways, moving from an entry beacon to an exit beacon, via an intermediate beacon (see Figure 1). Between eight and ten aircraft are managed during a scenario. Operators are naïve subjects, trained in the management procedures necessary to ensure aircraft safety and expedition, and with some practice in the management task.

Figure 1. Topographic representation of the simulated sector

Figure 1 Représentation topographique du secteur simulé

While the simulation constitutes a considerable simplification of operational ATM, it is sufficient for method illustration. Planning can be observed, of interventions to (process) object variables, and is available for analysis. Likewise, desired levels of worksystem performance can be specified, actual levels of performance measured, and effectiveness or ineffectiveness thereby expressed. Providing these criteria are met, it is proposed that the method may be applied to other domains (operational or microworld). Having considered the microworld of interest, the method is presented in four stages. First, consideration is given to conceptualising the operator planning horizon. Second, the work carried out by the microworld operator is considered (Stage 2), after which models of planning horizons are constructed (Stage 3). Finally, ineffectiveness is diagnosed (Stage 4).

II.         THEORY OF THE PLANNING HORIZON

II.1            PLANNING SCOPE

The term ‘planning’, within the Cognitive Ergonomics / Science / Psychology literature is used to refer to a wide range of human operator behaviours: problem-solving (O’Hara & Payne, 1999); task scheduling (Hayes-Roth & Hayes-Roth, 1979); decision-making (Pietras & Coury, 1994); or anticipation (Boudes & Cellier, 1997, 1998). Additionally, planning can refer to an immediate action (Shallice, 1982), or an action after some temporal delay (Amalberti & Deblon, 1992). In consequence, some clarification of the scope of the term, as used here, is required. Planning refers to the formation of plans for actions/interventions with aircraft at some point in the future. An important distinction here, therefore, is between pre-planned interventions and unplanned interventions (referred to in this paper as management by ‘instant execution’). Instant execution occurs where an intervention is specified, and then immediately executed by the operator. As such, the term refers to the formation of real-time control decisions, with no forward-looking time delay between the specification of an intervention and its execution. In contrast, planning is always for some future intervention (ahead), that is, not the next action undertaken by the operator, after specifying the intervention. The planning ahead of interventions is commonly: (i) viewed as more efficient (i.e., involving fewer interventions) than instant execution, when implementation costs are high (O’Hara & Payne, 1999); (ii) associated with expertise and strategic thinking (de Groot, 1978); and therefore (iii) considered a form of best practice. For example, de Groot showed that grandmasters plan ahead to a depth of six or seven chess interventions, in contrast to novices, who may only consider one or two interventions ahead. The benefits of such ability are clearly demonstrated by game outcome; the grandmaster is a ‘master’ for good reason.

The development of technology, for the management of dynamic processes, reflects the general desirability of planning interventions further into the future. The ATM-like microworld, used in this paper, serves only to illustrate the method. However, within operational Air Traffic Management in recent years, the concept of ‘gate-to-gate’ aircraft route planning has emerged. This concept requires that the planning of aircraft interventions no longer be devolved to the level of sector management, but rather be considered for all sectors traversed by aircraft during flight, by a separate team of specialist ‘multi-sector planners’. Again, this strategy suggests that planning further ahead is considered generally to be a desirable feature of dynamic process management, a feature for which technological support is sought (David, 1997; Miaillier, 1998).

II.2            PLANNING THEORIES

Models of human and machine planning have a long history (Hoc, 1988; Miller, Galanter, & Pribram, 1960; Sacerdoti, 1977), such models drawing to a greater or lesser extent, on planning theory  (Suchman, 1993, 1987; Vera & Simon, 1993). For the purpose of expressing planning effectiveness, at Stage Three of the method, a model is constructed that represents an operator’s plans for future interventions with aircraft. Model construction is informed by a Theory of the Operator Planning Horizon. The theory is largely a synthesis of parts of other planning theories, with some additions to make it suitable for expressing planning effectiveness. Such theory-driven modelling is possible because the simulation presents the operator with a well-defined problem space, and the task is completed by operator-technology interaction. There are no human-to-human collaborative processes influencing the planning task (Hughes, Somerville, Bentley, & Randall, 1993). As such, the method addresses planning as work at the level of micro-level mental processes, rather than as a macro-level community activity (Engeström, 2000). There is only one agent of management, who plans, anticipates, intervenes, and so forth.

From planning theory, a number of stable phenomena have been documented, of both planning behaviour, and of plans themselves, that may be assumed when modelling operator planning for dynamic process management.  For example, planning is most frequently characterised as being a form of goal-oriented behaviour (Newell & Simon, 1972). Having expressed the goals of management within the microworld as the maintenance of aircraft safety and expedition, it is anticipated that individual plans for interventions (to speed and altitude of individual aircraft) can be associated with these management goals.

During the planning process, a number of mental representations (knowledge sources) are known to be utilised, mental representations of (1) the domain state (Moray, 1992; Rasmussen, 1986), and of (2) how to interact with technologies/devices (Young, 1983) to bring about transformation of that domain (Payne, Squibb, & Howes, 1990). A mental representation of the state of the domain corresponds to what the operator knows, from moment to moment, about the state of the managed traffic – the work being undertaken (Dowell & Long, 1998). This representation is frequently referred to as the operator’s ‘picture’ (Cox, 1992; Whitfield & Jackson, 1982), and it is used to predict the future states of aircraft on the sector, and thereby plan.

Different types of plan are known to exist (Hoc, 1993), two possible types being plans for: (1) high-level process management; and (2) particular interventions to particular process objects. Here, the plans of concern are scoped to the latter class, plans for interventions with simulated aircraft (worksystem actions that transform the domain). In addition to mental representations of the domain (used in planning), the operator requires representations concerning how to interact with worksystem devices (Young, 1983), so that the detailed specification of actions, which bring about interventions, can be constructed.

Hayes-Roth and Hayes-Roth (1979) have shown that human planning is opportunistic, in that individuals plan when opportunity and necessity demand. Operators may not maintain a complete, coherent and well integrated set of plans for all aircraft managed within the sector. Rather, plans may exist for some aircraft, known to be particularly problematic (top-down planning), but as domain events arise (or are predicted to arise) that affect management goals, plans may be constructed in a reactive manner (bottom-up). It is therefore not the case that what is planned (at ‘planning time’) is always executed (at specified ‘execution time’). Rather a range of outcomes may be observed as plans may be discarded or repaired (Hoc 1988; Woods, 1988), due to failures in information acquisition and subsequent anticipation, or decay (i.e., be forgotten (Timmer & Long, 2000)). Having considered some of the relevant planning literature, for the management of dynamic processes, the Theory of the Operator Planning Horizon can now be presented.

II.3            THEORY OF THE OPERATOR PLANNING HORIZON

The first stage of the method, for expressing the effectiveness of a planning horizon, involves conceptualising the planning phenomena of interest. For this microworld, conceptualisation involves scoping operator planning behaviour, observed in the simulation, with respect to the planning literature, and synthesising existing theory with concepts necessary to express the effectiveness of a planning horizon. The outcome here is the Theory of the Operator Planning Horizon (TOPH), which makes explicit the planning phenomena of concern to the research. The TOPH may be stated, in domain-independent terms, as follows:

• An interactive (planning) worksystem formulates plans for interventions to a domain that are intended to attain management goals.

• Plans are formed by planning behaviour that requires mental representations of: i) the domain; and ii) the devices (and how to interact with those devices, to bring about interventions).

• Plans specify: i) interventions to domain object attribute values; and ii) a ‘triggering condition’ for plan execution.

• One or more plans, that refer to the same domain object, constitute the planning horizon for that object.

• A planning horizon may be described in terms of its ‘extension’.

• A planning horizon’s extension is expressed in terms of the future state of the domain object in question, if all planned interventions are executed.

• The extension of a planning horizon may be described as being adequate or inadequate for ensuring that the goals of management are met. The adequacy of a planning horizon’s extension is determined by the individual plans it contains, that is to say, whether or not when implemented, those plans ensure management goals are attained. If an operator’s planning horizon extension is adequate, and assuming the planned interventions are executed, the horizon will support effective management.

The theory places emphasis upon the planning worksystem’s goals, mental representations, and plan details. The concept of an horizon’s ‘extension’ arises from the need to express how far into the future plans (within an horizon) account for the state of a managed object. For example, two planned interventions to an aircraft (in the microworld) may be sufficient to ensure that the aircraft leaves the sector in its planned ‘exit’ state. The horizon (comprising those two planned interventions) may then be said to ‘extend to’ that aircraft’s exit state (object goal state, at the time of planning). Alternatively, horizons for aircraft may extend to states, associated with particular beacons, or parts of airways. Expressing the planning horizon’s extension in terms of the state of an object may be contrasted with other theories, such as Boudes and Cellier’s (1997) Theory of Anticipation Range. In their theory, the extension of a set of plans is expressed in terms of time, for example, plans that extend over the next 5 minutes for a given aircraft (see also Anderson & Settle, 1996). Finally, given the present work’s focus upon expressing problems with plans, the notion of planning extension adequacy is critical and novel. The adequacy of an extension is a concept that is used to relate plans for particular interventions to management goals, and thereby enable the expression of whether or not the specified plans for an aircraft ensure safety and expedition for that aircraft, over the extension of the plans. In conclusion, while the theory embodies some familiar concepts from planning theories (e.g., use of mental representations in planning, existence of triggering conditions, and so forth), it also possesses some novel concepts that are important for the expression of planning effectiveness.

Use of the term ‘horizon’, to refer to the limits of a mental behaviour, is not new. Hutchins (1990) describes an ‘horizon of observation’ in ship navigation, and more recently Wong, Sallis and O’Hare (1997) have discussed the planning horizon with respect to ambulance dispatch, yet in a non-technical manner, without defining what is meant by the term. Hence the need to conceptualise the term, and thereby generate a theory. Perhaps the most complete and explicit exposition of an alternative theory to TOPH, that examines the horizon of some form of mental behaviour, is Boudes and Cellier’s (1997, 1998) Theory of Anticipation Range, which possesses two components: (1) a mechanism for anticipating future domain object states; and (2) a temporal horizon. In contrast, the TOPH possesses three components: (1) planning behaviour (including plans); (2) horizon extension (in terms of future object states, rather than time); and (3) the adequacy of horizon extension (explicitly addressed).

II.4            REQUIREMENTS FOR A MODEL OF THE PLANNING HORIZON

The TOPH conceptualises the planning phenomena of interest, for the modelling of an operator’s horizon. From the theory, requirements may be generated, concerning behaviours that a model of a planning horizon should capture (at Stage 3), if the model is to represent accurately an horizon. These requirements are:

• Models need to reference interventions with objects (aircraft) that can be associated with management goals (of safety and expedition).

• Models need to reference the operator’s mental representation of the state of the domain at the time of plan formation.

• Models need to specify the details of planned interventions, and the conditions for triggering such interventions.

• Planning horizon models need to be specified for each managed object, and moment-to-moment changes of horizon extension need to be expressed in terms of a change in state of the object to which reference is made – domain objects here may be abstractions, i.e. groups of functionally related objects (aircraft).

• Models need to specify all the interventions that actually take place with an object (planned and unplanned), and their impact upon management effectiveness, such that the adequacy of an horizon extension may be expressed retrospectively.

With respect to the microworld, it can be seen that following the TOPH, a model needs to represent: (1) operator plans for a particular aircraft (one aircraft per horizon); (2) the operator’s mental representation of the state of the domain at the time of planning; and (3) the interventions actually carried out. Such a model addresses the planning concern of this research. To examine the effectiveness of the plans formed, a model of the planning horizon needs to be considered alongside a second model, a model that captures the quality of work carried out (i.e. the extent to which the aircraft objects were actually managed in accordance with management goals). This second model of work quality is termed a domain model, and the focus for Stage 2 of the method. In the next section, a microworld-generated domain model is considered.

III.         DOMAIN MODEL

In the microworld, a computer-based simulation generates a radar image of the managed sector, updated as aircraft traverse the sector, or change speed/altitude in response to operator interventions. In addition to generating this image, after each operator intervention, the simulation calculates new domain model values for a hierarchy of aircraft attributes that may have been altered as a consequence of the intervention. At the lowest level of the hierarchy are what Dowell (1998) calls PASHT attributes, standing for: Position; Altitude; Speed; Heading; and Time.  From these low level attributes, intermediate attributes are calculated for aircraft: Progress (flight duration); Fuel Use; Separation and Number of Manœuvres. Finally, at the apex of the attribute hierarchy, values for aircraft safety and expedition are calculated from intermediate attribute values. Therefore, an intervention to change an aircraft altitude will be recorded in the domain model at the PASHT level, and consequences of that aircraft climbing/descending to the new altitude will be calculated in terms of progress and fuel use, and ultimately any consequences for aircraft safety and expedition at the new altitude (and during the ascent or descent) will be calculated. Following the last intervention with each aircraft, the set of attribute values calculated represent the final ‘actual’ values for: Progress, Fuel Use, and so forth; for that aircraft’s passage across the sector. In addition to calculating these values after each intervention, for each attribute the model possesses a goal value. These goal values are calculated, by the simulation software, based upon an optimal (goal) path across the sector. Therefore, given ‘actual’ attribute values (from management scenarios) and ‘goal’ values (from an optimal scenario), comparisons may be made, from intervention to intervention, between the actual and goal states of managed aircraft. Large discrepancies between actual and goal values will constitute the starting point for diagnosing worksystem design problems.

Figure 2. Part of a domain model for a single intervention with aircraft BAN

Figure 2. Partie du modèle du domaine pour une intervention unique sur l’avion BAN

Figure 2 shows part of a domain model for aircraft BAN, following an operator intervention to slow down BAN from 900Kmph to 720Kmph (the change to BAN’s PASHT attribute – Speed – is shown at A). At B and C, only Intermediate attribute values are shown, from which values for safety and expedition can be calculated. Bold values in brackets, alongside each attribute name, represent goal values for that attribute. Unbracketted values show calculated predictions for each attribute, given the aircraft’s state, following the most recent intervention. A set of such values, for all aircraft after each intervention, constitutes the domain model of interest. Therefore, as a consequence of an intervention to reduce BAN’s speed, its predicted Progress across the sector is slowed, from 1170secs to 2220secs; and Fuel Use is greatly reduced, as 720Kmph is the cruising speed for all aircraft (the speed at which fuel use is optimised). Within the model the Separation attribute shows the aircraft is safe, no separation conflicts with other aircraft have been predicted, given the new speed value. As a consequence of the intervention, the Number of Manœuvres attribute is increased by one.

When such data concerning the actual state of an aircraft are compared with goal values for that aircraft, an analysis of the quality of aircraft management is possible, both from moment to moment, and at the end of the management session. The data for BAN here show that prior to the speed intervention, BAN’s time to progress across the sector was 1170sec. When compared to BAN’s goal value for progress, this value indicates BAN to have been progressing too quickly across the sector, and due to exit the sector 1260secs earlier than planned (the difference between actual and goal values). The cost associated with such fast passage is clear from the fuel consumption figures, which show the aircraft, prior to intervention, consuming 328 more units of fuel than the goal value. Both before and after the intervention, BAN’s separation value is safe, and so BAN’s superordinate safety attribute can likewise be calculated as being safe – BAN is in a safe state, both before and after the intervention. Following the intervention, BAN’s progress and fuel consumption are more closely aligned with goal figures.

To establish instances of ineffective management, some criteria need to be applied to the size of any discrepancy between goal and actual attribute values. Here, it is assumed, for the purpose of expressing effectiveness, that if values for an aircraft’s progress or fuel consumption exceed 10% (either side) of the goal value, ineffective management has occurred. Likewise, if aircraft separation is violated, ineffectiveness has occurred. Finally, the Number of Manœuvres value should not exceed 3 (i.e. 50% in excess of the goal value shown). Consideration will now be given to the impact these criteria have (for intermediate attribute values), on high level attributes of safety and expedition. If separation is violated, the attribute value for safety likewise reflects this state (Safety = Unsafe). If any one of the values for: Progress; Fuel Use; or Number of Manœuvres exceed specified criteria, the aircraft may likewise be considered ‘unexpeditious’, i.e. the management goal of expedition is not being attained.

In conclusion, data from the microworld-generated domain model concern the quality of aircraft management, and instances of attribute values violating criteria may be associated with instances of ineffective management – unsafe aircraft, aircraft progressing too fast or slow, consuming too much or too little fuel, or undergoing intervention too often. Expressing the effectiveness of an operator’s planning horizon, therefore, involves associating domain model data with particular interventions, and establishing whether or not those interventions were planned.  In the next section, Stage 3 of the method for expressing the effectiveness of planning horizons is presented. A second model, a model of an operator’s planning horizon, is discussed. Stage 3 is followed in Stage 4, by an analysis of the effectiveness of that horizon, when the planning horizon model is considered in the light of data from the domain model (Stage 2), which contains data similar to that discussed above.

IV. Model of a Planning Horizon

IV.1 Data Requirements for Model Construction

From the Theory of the Operator Horizon, a number of requirements were identified for a model of the horizon. Each requirement is now considered in turn, and the means for obtaining data to address the requirement discussed.

Requirements

• Models need to reference interventions with objects (aircraft) that can be associated with management goals (of safety and expedition).

In the microworld, all interventions are made with the radar device, and are therefore observable. Continuous observation of operator interaction with worksystem technologies yields such data.

• Models  need to reference the operator’s mental representation of the state of the domain at the time of plan formation.

Operators establish the state of the domain by referencing worksystem devices. By observing head movements and pointing actions to fields on a Flight Progress Strip, it is possible to establish what data concerning the domain are being acquired by the operator. Traditionally, concurrent verbal protocols provide a rich source of data concerning what an operator knows about a problem, and how that knowledge is used in problem solving. Verbal protocol data, in addition to observation of operator head and hand movement, yields data that assist in inferring the operator’s changing mental representation of the domain. In addition, the mental representation can be inferred from interventions. For example, if an operator intervenes with an aircraft to give it a cruising speed, it is possible to infer that the operator knew that aircraft’s speed was not the desired cruising speed, prior to intervention.

• Models need  to specify the details of planned interventions, and the conditions for triggering such interventions.

In the absence of planning tools that support the explicit documentation of a set of plans, operator plans for interventions remain in the head of the operator. In consequence, only verbal protocol data can reveal the details of such plans, and associated triggering conditions.

• Planning horizon models need to be specified for each managed object, and moment-to-moment changes of horizon extension need to be expressed in terms of a change in state of the object to which reference is made.

Each model should reference an individual aircraft. As the horizon changes, the state of the aircraft in question can be established with reference to the computer-generated domain model for that scenario (see Section III).

• Models need to specify all interventions that actually take place with an object (planned and unplanned), and their impact upon management effectiveness, such that the adequacy of an horizon extension may be expressed retrospectively.

To satisfy the first requirement, all interventions are documented. The impact of each intervention, upon management effectiveness (how well the actual state of the aircraft matches its goal state), can be established with reference to the domain model generated for the management scenario.

Therefore, with regard to acquiring data for the purpose of modelling operator planning horizons, observational data, of operator hand and head movements are required (for information search and interventions), plus concurrent verbal protocol data (for plans and evidence of the content of the associated mental representation). With such data, a model of the operator’s planning horizon can be constructed, as in Figure 3, which separates each of these classes of data.

IV.2            MODEL OF AN OPERATOR PLANNING HORIZON

Figure 3 presents a model of an operator’s planning horizon for an aircraft LOG. The ‘Plan/Execution’ column of the model records all human-technology worksystem goals that concern interventions. Data in this column distinguish goals for immediate execution (in bold), from plans for future interventions (in italics). The ‘Intervention’ column documents those goals for interventions that were actually implemented.

Figure 3. Model of an operator’s planning horizon for aircraft LOG

Figure 3. Modèle pour un opérateur d’un horizon de planification pour l’avion LOG

Goals that concern interventions are formed, given a particular mental representation of the state of the aircraft in question, at the time of planning or instant execution. The model records such mental representations in the ‘category’ column. The set of possible categories reflect the range of possible states of the objects in the domain (Timmer & Long, 1997). Aircraft may therefore be ‘Active’, when they arrive on the sector, or ‘Incoming’, prior to arrival. Once active, they may be ‘Safe’ or ‘Unsafe’, ‘Expeditious’ or ‘Unexpeditious’ with respect to their speed (‘Unexpeditious (Speed)’) or altitude (‘Unexpeditious (Altitude)’). Once an aircraft is at its exit altitude and cruising speed, no further interventions are required as it is in its exit/goal state (‘Active Aircraft (Exit)’). The final ‘Encode’ column records device information fields that were searched by the operator as a means to forming a mental representation (category) of the state of an aircraft, for the purpose of goal/plan specification or monitoring.

The model for aircraft LOG is considered in detail. The model commences (Line 1) by showing that at the beginning of the management session, having encoded LOG’s Flight Progress Strip (FPS), the operator knows that LOG is an incoming aircraft. No plans are formed at this stage. LOG then enters the sector (Line 2), and by encoding LOG’s radar trace, the operator’s mental representation of the state of LOG is updated, from ‘Incoming Aircraft’ to ‘Active Aircraft’. Referencing again LOG’s FPS (Line 3), the operator establishes LOG as travelling at 900Kmph, and at an altitude of 13,000ft. Given LOG’s excessive speed, the aircraft is mentally represented as being unexpeditious with respect to its speed, and an intervention is immediately formed (Line 3) and executed (Line 4) to slow the aircraft down, and transform its state to ‘expeditious’ with respect to its speed. This intervention is an example of instant execution, and represented within the model as such. One consequence of the intervention is that the operator’s mental representation of LOG is now updated, to reflect its new expeditious state. LOG is safe and expeditious, resulting in the operator forming a default plan, to leave LOG in its current state for the foreseeable future. The model then shows that LOG is left to progress from its entry beacon to the intermediate beacon ‘Delta’, and then on to its exit beacon ‘Epsilon’, at altitude 13,000ft and speed 720Kmph. Once at its exit beacon (Line 5), the operator encodes FPSs for both LOG, and a second aircraft SAM, and realises that both aircraft occupy the same altitude, and safety conflict is possible. LOG is therefore mentally re-categorised as ‘Unsafe’, and a plan is formed to give LOG its exit altitude of 4,000ft later (in the near future). The model then shows (Line 6) that the operator’s mental representation of: (1) the state of LOG; and (2) the plan for future intervention (formed at Line 5), are both forgotten (decay) and the plan to ‘Give LOG (its exit) altitude later’ is re-formed (at Line 7). On the occasion of this re-planning, the model shows that no safety conflict with SAM was identified, and LOG was represented once more as a ‘Safe Expeditious Aircraft’. The next line of the model (Line 8) shows that the operator appears to have forgotten (again) the state of LOG, and by referencing data from FPSs (Line 9), reforms the mental representation that LOG is a safe expeditious aircraft. The plan formed at Line 7 is not assumed to have decayed on this occasion, as the model shows no further re-planning of the intervention, until it is executed at Line 10. The final line of the model shows that LOG is given its exit altitude as planned, and the default plan to leave LOG until it exits the sector is formed, as the aircraft is in its exit state, and no further interventions are necessary.

If the model is considered with respect to how it documents the extension of the operator’s planning horizon for LOG, the following observations can be made. At Line 4, a plan is formed to ‘Leave LOG’. Given that LOG is flying at a high altitude, and at cruising speed, and is safe (i.e. is an ‘Active Safe Expeditious Aircraft’), LOG may be left in such a state, until near its exit beacon, and then it should be given its exit altitude. The plan to ‘leave’ LOG may therefore be said to extend to ‘near LOG’s exit beacon’. At Line 5, when LOG is near its exit beacon, a further plan is formed to give LOG its exit altitude. At Line 5, with such a plan, the operator’s planning horizon may be described as extending to ‘LOG’s exit state’ (i.e. to the end of its management). At Line 7, the new duplicate plan (of the plan formed at Line 5) has an identical extension to that of the plan at Line 5. Therefore, with a model of an operator’s planning horizon, from moment to moment, it is possible to express the ‘extension’ of the plans formed, either in terms of: a) the state of the aircraft in question, for example, the plans extend to LOG’s exit state; or b) some position on the sector at which the aircraft’s state must change, for example, plans for aircraft LOG extend to near LOG’s exit beacon’. A description of the extension of a planning horizon, therefore, arises from the details of the operator’s plans for a given domain object (aircraft) at a given moment. As plan details change, so too does the horizon’s extension.

V.         EXPRESSING HORIZON EFFECTIVENESS

Having conceptualised the planning horizon (Stage 1), described how a domain model captures work quality (Stage 2), and illustrated how a planning horizon is modelled (Stage 3), in Stage 4 of the method, expression of the effectiveness of that horizon is undertaken. In the first instance, the model of the planning horizon is considered, line-by-line, alongside objective data from the domain model, concerning aircraft states after each intervention. The actual state of aircraft (from the domain model) can then be compared with the operator’s inferred mental representation of the state of that aircraft (‘Category’ column of a planning horizon model), and plans and interventions considered, to establish whether or not the plans/interventions were effective. Once completed, and in the event of an instance of ineffectiveness occurring, some expression of its cause can be undertaken.

The Theory of the Operator Planning Horizon expresses the concept of effectiveness (attaining management goals), in terms of the adequacy of an horizon’s extension. Adequacy of an horizon’s extension relates to the quality of work that will be brought about by the worksystem, if the plans that make-up the planning horizon are executed. An horizon’s extension may be considered adequate, if the plans (that make-up the horizon), when executed, lead to the attainment of worksystem goals of safety and expedition. Adequacy is, therefore, a difficult attribute of an horizon to assess. At one moment the horizon may extend to an Aircraft’s (Aircraft X) exit state, and be adequate to ensure goals are met. At another moment, a second Aircraft Y may be given the same altitude as Aircraft X, thereby rendering the plans, that make-up Aircraft X’s planning horizon extension, inadequate for ensuring the maintenance of safety. Re-planning is then necessary. Existing plans may need to be discarded, given the new unsafe state of the aircraft. For each line of the model in Figure 3 (starting at Line 4), an assessment of adequacy is made, following a short description of the plan being assessed.

• Line 4

Encode       Intervention          Category                                           Plan/Execution

description

LOG is given speed 720Kmph (instant execution), and a plan is formed to ‘Leave LOG’ at speed 720Kmph, and altitude 13,000ft (i.e. as an Active Safe Expeditious Aircraft). A single plan is formed that extends to near LOG’s exit beacon.

adequacy

Following the intervention to LOG’s speed, at the time of plan formation, the domain model’s prediction of LOG’s state is presented in Figure 4.

Figure 4. Domain model performance data for LOG

Figure 4. Données du modèle de performance du domaine pour LOG

The domain model shows that LOG is:

• safe in its new state

The planning horizon at Line 4 is made-up of a single plan, to leave LOG, and given that LOG is safe in its current state, it may be said that the horizon extension, at this point, is adequate for ensuring that aircraft safety is maintained.

• expeditious in its new state

Projected progress is within 10% of criterion. Projected fuel use is 18.8% in excess of criterion. Given LOG’s speed and high altitude, it would seem the projection of excessive fuel use refers to fuel already consumed, before the intervention, i.e. when travelling at 900Kmph earlier in the management scenario (see Line 3, Figure 3). The plan to leave LOG near its exit beacon is therefore adequate for ensuring aircraft expedition.

• Line 5

Encode                  Intervention    Category                    Plan/Execution

description

LOG progresses across the sector to its exit beacon. A plan is formed to give LOG its exit altitude of 4,000ft, ‘later’. This single plan extends to LOG’s exit state. While ‘later’ is a triggering condition of minimal specification, the assumption that the horizon extends to LOG’s exit state is considered justified, because all necessary interventions have been specified to ensure LOG leaves the sector in its exit state.

adequacy

At planning time, and as a consequence of the adequacy of the planning horizon’s extension at Line 4, LOG is safe and expeditious. This plan is actually executed at Line 10. It is therefore possible to consult the domain model’s assessment of LOG’s safety and expeditiousness after this intervention (Figure 5), and thereby assess the adequacy of the plan extension.

Figure 5. Performance data for LOG after the last intervention

Figure 5. Données de performance de LOG après la dernière intervention

From Figure 5, it is therefore possible to say retrospectively:

• the planning horizon extension (to LOG’s exit state) will ensure its safety

• while progress and exit altitude are as planned, LOG’s fuel

consumption increases with this intervention (40% in excess of the

criterion value). It would therefore appear that this horizon extension is not adequate for ensuring that aircraft expeditiousness is maintained.

• Line 6

Encode           Intervention               Category                              Plan/Execution

The details of the plan to ‘Give LOG altitude 40, later’ appear to have decayed, and so the model shows the operator has no plans at this time.

• Line 7:

Encode                    Intervention     Category                  Plan/Execution

description

The intervention is re-planned, as at Line 5.

adequacy

As at Line 5.

No further plans are formed. At Line 10, the plan formed at Line 7 is executed. The domain model, as discussed at Line 5, shows LOG is unexpeditious with respect to its fuel use (40% excess).

From this line-by-line analysis, it is clear that the effective management of aircraft LOG’s fuel consumption did not take place as desired. In consequence, the domain model shows LOG to have been unexpeditious as it exited the sector, having consumed 40% more fuel than desired (the bracketed goal value for fuel use in the domain model). Given this ineffectiveness, the following expression of the problem, with this ATM planning task, is possible:

The problem of managing aircraft expedition with respect to fuel use may originate, as in the case of LOG, with aircraft entering sectors at high speed (greater than cruising speed), thus already rapidly consuming large quantities of fuel. The timely reduction of aircraft speed can alleviate this problem. However it would seem a greater part of the problem arises from judging the appropriate moment to intervene with aircraft, to allocate low exit altitudes. Managing aircraft at cruising speed, as high as possible for as long as possible, is the best strategy for minimising fuel consumption. With a reduction of altitude comes a commensurate increase in fuel consumption. If an aircraft is to leave a sector at a low altitude (e.g., for airport approach), careful judgement is required as to when to execute such an intervention, so that the aircraft exits the sector at the exit altitude. If such an intervention is executed too early, the aircraft will fly for an extended duration at a low altitude, consuming higher quantities of fuel. In the case of LOG, this problem would appear to have been the most important. The operator’s plan to ‘Give LOG Altitude 4,000ft, later’ lacked accurate reference to a position within LOG’s final airway, when the intervention would be made, ‘later’ merely meaning ‘near LOG’s exit beacon’. It is assumed that in consequence, LOG was given its exit altitude too early, and flew too low for too long, to maintain an expeditious level of fuel consumption. While worksystem technologies supported the operator in forming a plan for the correct intervention, they offered no support for judging the most appropriate moment for plan execution.

This expression of the causes of ineffectiveness accounts for ineffectiveness in the management of a single aircraft. It is proposed that for each additional instance of ineffectiveness, attributed to fuel use, a similar analysis be undertaken. When considering the data over a number of instances of the same problem, a general expression of the problem is possible (Timmer, 1999). Nevertheless, from consideration of the expression above, it is proposed that problem-driven technological evolution can benefit from problem expressions, similar to that illustrated. In the case of the evolution of technologies to support better fuel use management, redesign may commence with consideration of how to support operator judgement in timing the moment of low-level aircraft descent, prior to exit.

V.         DISCUSSION

In this paper, a method has been proposed and illustrated for expressing human-technology worksystem effectiveness during a planning task. The method involves stages of: conceptualising behaviour of interest; modelling the domain to measure work quality; modelling planning behaviour; and considering work quality alongside worksystem behaviour, to enable an expression of effectiveness. Using an illustration of the poor quality management of expedition, ineffectiveness was identified, and expressed in terms of worksystem behaviour, and in a manner proposed to support problem-driven technological evolution.

Of the method, a number of observations may be made. The method is comprised of a set of general stages, rather than a set of detailed procedures. For example, following conceptualisation, a domain model needs to be constructed. The illustration discusses one such model (not how it was constructed), and it is accepted that there are many alternatives to the one discussed. For the method to express effectiveness successfully, it is merely a requirement that a domain model measure work quality in some way, using some performance criteria for judgement of ineffectiveness (design problems). Without such measurement, comparing worksystem behaviour with work quality during Stage 4 is not possible. One advantage of expressing the method in this manner is that it extends the method’s scope of application. An ATM-like microworld was the focus of the illustration, and planning ahead conceptualised with respect to the planning behaviour likely to be observed during management of that microworld. Using another domain (the domain of Railway Signal Management (RSM) has also been analysed), it is possible to envisage other behaviours being conceptualised, for example, the notion of planning extended to include strategic plans, (for example to maximise train throughput or even out train flow), as well as plans for discrete interventions (tactical) (for example, to particular signals). Provided such planning is modelled accurately at Stage 3, the logic of the method, and successful expression of effectiveness, should be maintained.

The success with which the method can be migrated from a microworld to an operational environment, is largely a function of the extent to which a) the planning horizon modelling requirements can met, and b) the target domain modelled (and performance measured). In current operational ATM, for example, the method, as it stands, will not scale-up, as the verbal protocol data (revealing planning behaviour) can not be obtained without task interference. In the future, with Datalink technology, method application to operational ATM may be more feasible. In RSM, the voice channel is largely free, except for control-to-train communication, and control room communication – here also, management is largely undertaken by a single signalman (with some communication with other managed line sections and a supervisor).

When this work is considered alongside the work of Boudes and Cellier (1997, 1998), concerning controller ‘anticipation range’ in ATM, some similarities and differences can be identified. Theoretically, Boudes and Cellier’s work has strong similarities, in that they too seek to establish relationships between operator mental representations of the domain and devices, and plans for interventions and flight strip management. However, the method presented here, for modelling the effectiveness of planning horizons, possesses a number of novel components. Firstly, it attempts to characterise the extension of a planning horizon in non-temporal terms, but rather in terms of the future states of intervened aircraft (whether or not aircraft will be safe or unsafe, and to what future point (on the sector) do such plans extend). Secondly, the method possesses a domain model, with which to assess the adequacy of plans for future interventions. The domain model is crucial to the successful execution of the method, as it enables an assessment to be made of how well the operator is planning, and how adequate the specified plans are for ensuring that management goals are achieved. Without a domain model, a method can only determine that an operator is planning. No assessment of the quality of those plans can be made.  As the focus of the method is to support the development of technologies that will improve operator planning abilities, the domain model is crucial for determining existing planning ineffectiveness and how such ineffectiveness can be overcome through re-design.

To conclude, this paper tries to make explicit a number of important relationships that need to be established before the expression of effectiveness is possible. The Theory of the Operator Planning Horizon tries to make clear the behavioural phenomena of concern, and enables the clear derivation of requirements for data that constitute a representation of such an horizon. Likewise, the domain model enables the identification of problems of worksystem performance, and subsequent construction of a worksystem model to establish the behaviour that brought about less than the desired quality of work. Establishing relationships from the data serves to augment the subjective (anecdotal) recall of operator ‘problems’ with technologies, and offers some basis for quantifying problems, and the subsequent generation of priorities for re-design. Such a method is therefore considered a useful tool, to compliment existing design practices, to support the explicit expression of the magnitude of a design problem. As such, it is considered to have advanced the ‘design for effectiveness’ approach, since without a well-specified expression of the design problem, there can be no (known) design solution, nor acquisition and validation of design knowledge supporting the transition from one to the other (Long & Timmer, 2001). Phenomena-driven and human performance-driven approaches are unable to support such a design origin and transition.

VI.         REFERENCES

Amalberti, R., & Deblon, F. (1992). Cognitive modelling of fighter aircraft process control: a step towards an intelligent on-board assistance system. International Journal of Man-Machine Studies, 36, 639-671.

Anderson, B. F., & Settle, J. W. (1996). The influence of portfolio characteristics and investment period on investment choice. Journal of Economic Psychology, 17, 343-358.

Barnard, P. (1991). Bridging between basic theories and the artifacts of human-computer interaction. In J. M. Carroll (Ed.), Designing interaction: Psychology at the Human-Computer Interface. Cambridge, UK: Cambridge University Press.

Blandford, A., & Young, R. M. (1993). Developing runnable user models: separating the problem solving techniques from the domain knowledge. In J. L. Alty, D. Diaper, & S. Guest (Eds.) People and Computers VIII, Proceedings of HCI’93. Cambridge, UK: Cambridge University Press.

Boudes, N., & Cellier, J-M. (1997). Evaluating the concept of anticipation range in air traffic control. Paper presented at the Ninth international Symposium of Aviation Psychology, Columbus, Ohio, May.

Boudes, N., & Cellier, J.-M. (1998). Étude du champ d’anticipation dans le contrôle du trafic aérien. Le Travail Humain, 6, 29-50.

Card, S. K., Moran, T. P, & Newell, A. (1983). The Psychology of Human- Computer Interaction. Hillsdale, New Jersey: Lawrence Erlbaum Associates.

Cox, M. (1992). The cognitive aspects of the air traffic control task: a literature review. IAM Report No. 718. Psychology Divison, RAF Institute of Aviation Medicine.

David, H. (1997). Radical revision of en-route air traffic control. EUROCONTROL Experimental Centre Report No. 307. http:/www.eurocontrol.fr/public/reports/eecreports/1997/rep307

de Groot, A. D. (1978). Thought and choice in chess. The Hague: Mouton.

Dowell, J., & Long, J. (1998). Conception of the cognitive engineering design problem. Ergonomics, 41, 126-139.

Dowell, J. (1998). Formulating the cognitive design problem of air traffic management. International Journal of Human-Computer Studies, 49, 743-766.

Engeström, Y. (2000). Activity theory as a framework for analyzing and redesigning work. Ergonomics, 43, 960-974.

Flach, J. M. (1998). Commentary. Cognitive Systems Engineering: putting things in context. Ergonomics, 41, 163-167.

Hayes-Roth, B., & Hayes-Roth, F. (1979). A cognitive model of planning. Cognitive Science, 3, 275-310.

Hoc, J-M. (1988). Cognitive Psychology of Planning. London: Academic Press.

Hoc, J-M. (1993). Main features of the human supervision of a long response latency process. In Proceedings of the IEEE/SMC’93 Conference on Systems, Man and Cybernetics, pp. 114-119. Piscataway, NJ: IEEE.

Hollnagel, E. (1998). Commentary. Comments on ‘Conception of the cognitive engineering design problem’ by John Dowell and John Long. Ergonomics, 41, 160-162.

Hughes, J. A., Somerville, I., Bentley, R., & Randall, D. (1993). Designing with ethnography. Making work visible. Interacting with Computers, 5, 239-253.

Hutchins, E. (1990). The technology of team navigation. In J. Galegher, R. E. Kraut, & C. Egido (Eds) Intelligent Teamwork: Social and Technological Foundations of Cooperative Work, pp.191-220. Hillsdale, New Jersey: Lawrence Erlbaum Associates.

Long, J., & Timmer, P. (2001). Design problems for research: what can we learn from ATM-like microworlds. Le Travail Humain, 64.

Meyer, D. E., & Kieras, D. E. (1999). A computational theory of executive cognitive processes and multiple-task performance: Part 1 Basic Mechanisms. Psychological Review, 104, 3-65.

Miaillier, B. (1998). ATM Strategy for 2000⁺: Volumes 1 and 2. EUROCONTROL Report FCO.ET1.ST07.DEL02. http://www.eurocontrol.be/ded/atmstrat

Miller, G. A., Galanter, E. & Pribram, K. H. (1960). Plans and the Structure of Behaviour. New York: Holt, Rinehart & Winston.

Moray, N. (1992). Mental models of complex dynamic systems. In P. Booth, & A. Sasse (Eds). Mental Models and Everyday Activities, Second Interdisciplinary Workshop on Mental Models, pp. 103-132. Cambridge, UK, March.

Newell, A., & Simon, H. A. (1972). Human Problem Solving.  Englewood Cliffs, NJ: Prentice-Hall.

O’Hara, K, P., & Payne, S. J. (1999). Planning and the user interface: the effects of lockout time and error recovery cost. International Journal of Human-Computer Studies, 50, 41-59.

Payne, S. J., Squibb, H. R., & Howes, A. (1990). The nature of device models: the yoked state space hypothesis and some experiments with text editors. Human-Computer Interaction, 5, 415-444.

Pietras, C. M., & Coury, B. G. (1994). The development of cognitive models of planning for use in the design of project management systems. International Journal of Human-Computer Studies, 40, 5-30.

Rasmussen, J. (1986). Information processing and human-machine interaction: An approach to cognitive engineering.  New York: North Holland.

Rasmussen, J. (1992). The ecology of work and interface design. In A. Monk, D. Diaper, & M. D. Harrison (Eds.), People and Computers VII. Proceedings of the HCI’92 Conference. Cambridge, UK: Cambridge University Press.

Reason, J. (1998). Commentary. Broadening the cognitive engineering horizons: more engineering, less cognition and no more philosophy of science, please. Ergonomics, 41, 150-152.

Sacerdoti, E. D. (1977). A Structure for Plans and Behaviour.  New York: Elsevier.

Shallice, T. (1982). Specific impairments of planning. Philisophical Transactions of the Royal Society of London, B 298, 199-209

Suchman, L. (1987). Plans and situated actions: the problem of human- machine communication. Camridge, UK: Cambridge University Press.

Suchman, L. (1993). Response to Vera and Simon’s situated action: a symbolic interpretation. Cognitive Science, 17, 71-75.

Timmer, P. (1999). Expression of operator planning horizons: a cognitive engineering approach. PhD Thesis, University of London.

Timmer, P., & Long, J. (1997). Separating user knowledge of domain and device: a framework. In H. Thimbleby, B. O’Conaill & P. Thomas (Eds.) People and Computers XII. Proceedings of HCI’97, pp379-395. London, UK: Springer.

Timmer, P., & Long, J. (2000). Plans versus outcomes: establishing the costs of planning. In E. Hollnagel, P. Wright, & S. Dekker, (Eds.) Proceedings  of the Tenth European Conference on Cognitive Ergonomics, pp158-165. Linkoping, Sweden: European Association of Cognitive Ergonomics.

Vera, A. H., & Simon, H. A. (1993). Situated Action: a symbolic interpretation. Cognitive Science, 17, 7-48.

Vincente, K. J. (1998). Commentary. An evolutionary perspective on the growth of cognitive engineering: The Risø genotype. Ergonomics, 41, 156-159.

Whitfield, D., & Jackson, A. (1982). The Air Traffic Controller’s ‘Picture’ as an Example of a Mental Model. In G. Johannsen, & J. E. Rijnsdorp, (Eds.) Analysis, Design, and Evaluation of Man-Machine Systems, pp. 45-52. New York: Pergamon.

Wong, W., Sallis, P., & O’Hare, D. (1997). Eliciting portrayal requirements: experiences with the critical decision method. In H. Thimbleby, B. O’Conaill, & P. Thomas, (Eds.) People and Computers XII. Proceedings of HCI’97, pp. 397-415.  London, UK: Springer.

Woods, D. D., & Roth, E. M. (1988). Cognitive Systems Engineering. In M. Helander (Ed.), Handbook of human-Computer Interaction, pp. 3-43. Amsterdam: North Holland.

Woods, D. D. (1988). Commentary: Cognitive Engineering in complex dynamic worlds. In E. Hollnagel, G. Mancini, & D. D. Woods (Eds.) Cognitive Engineering in Complex Dynamic Worlds. London: Academic Press.

Woods, D. D. (1998). Commentary. Designs are hypotheses about how artifacts shape cognition and collaboration. Ergonomics, 41, 168-173.

Young, R. M. (1983). Surrogates and mappings: two kinds of conceptual models for interactive devices. In D. Gentner, & A. L. Stevens (Eds.), Mental Models, Ch. 3. Hillsdale, New Jersey: Lawrence Erlbaum Associates.

Adam Stork, James Middlemass, and John Long

Ergonomics & HCI Unit, University College London
26 Bedford Way London WC1H OAP

ABSTRACT

MUSE, a structured Method for Usability Engineering, was created to improve the practice of Human-Computer Interaction practitioners, a practice that is primarily one of designing artefacts that fulfil user requirements. This paper offers a case-study application of MUSE to a set of domestic energy management user requirements to produce an artefact.  The paper presents: an overview of MUSE; the necessary features of an application; the user requirements; the details of the application; the resulting artefact; and an assessment of the artefact with respect to the user requirements.  Finally, it is argued that this case-study be considered ‘successful’, where a successful case-study extends the known frontiers of application of MUSE.

Keywords

Human-Computer Interaction; Human Factors; Structured Methods; Energy Management Systems; Planning and Control; Software Engineering

1. Introduction

The current practice of Human-Computer Interaction (HCI) is primarily one of designing artefacts that fulfil user requirements.  MUSE, a structured Method for Usability Engineering (Lim et al., 1992 for an overview; and Lim & Long, 1994 for detail), was created to improve the practice of HCI practitioners, i.e. their ability to design artefacts that fulfil user requirements.  MUSE was developed using a strategy for structured (human-factors) method development (Lim et. al, 1990) which involved (in the main) iterative proposition of a version of the method followed by testing through its application to the design of some interaction artefact.  These applications are termed case-studies.  Successful case-studies, those which produce interaction artefacts fulfilling the user requirements, extend the known frontiers of application of the method; while unsuccessful case-studies delimit the frontiers and provide input to the development of further versions of the method.  To be testable, the known frontiers of application of the method can be expressed in terms of how well-defined, complex, and observable are the user requirements of successful and unsuccessful case-studies.

MUSE has been fully documented in the HCI literature (Lim et al., 1991; Lim & Long, 1992).  These references include reports of the application of MUSE.  However, these case-studies have primarily exposed and exemplified the method.  In contrast, the focus here is on the successful or unsuccessful aspects of a case-study as defined above.  Thus, this paper reports a successful case-study of MUSE: an application that produced an artefact which is claimed to fulfil a set of user requirements.

The user requirements for the case-study resulted from an observed problem with the energy management of a specific home under study, the home of A and S.  These user requirements arose in the context of other research (Stork & Long, 1994) which required them to be well-defined and relatively simple.  These requirements make it possible to report the case-study in a paper of this length.  The clear definition and simplicity, however, do not detract from the consequences for MUSE of a successful case-study, since its frontiers of application still become better known .  Further, these consequences could be extended, since the user requirements can be claimed to have similarities with less well-defined and more complex, but equally observable, user requirements.  The claim that this case-study is successful is based on a demonstration that: the resulting artefact fulfils the user requirements; and that MUSE was applied in the development of that artefact from the user requirements.

The paper describes: an overview of MUSE; the necessary features of a MUSE application; the case-study’s user requirements; an appropriately detailed (to highlight those features) application of MUSE to the user requirements to produce an artefact; the resulting artefact; and an assessment of whether the artefact fulfils the user requirements.  Finally, some conclusions are suggested and further research is identified.

2. Overview of MUSE

MUSE is a structured analysis and design method for use by human factors engineers.  The method aims to improve the practice of HCI practitioners by providing support for the integration of human factors with existing structured methods for software engineering, such as Yourdon, JSD, or SSADM.  The output of MUSE is the specification of an interaction artefact, the software engineering method producing the specification of an implementable artefact, incorporating the interaction artefact.

MUSE supports design in a ‘top-down’ manner based on information derived ‘bottom-up’ and progresses from the specification of general features of the tasks to be performed by the user, derived from analysis of the user requirements and from existing systems, to the specification of the details of the interaction artefact.  The application of MUSE is an iterative process, both overall and internally, supporting the production of the best first-attempt artefact, following the initial complete application.

Figure 1 shows a schematic diagram of MUSE with an unspecified software engineering method.  MUSE has three phases: the Information Elicitation and Analysis phase; the Design Synthesis phase; and the Design Specification phase.  The Information Elicitation and Analysis phase supports primarily: the assessment and re-use of components of extant systems; and the maintenance of the consistency of the design with the user requirements.  The Design Synthesis phase supports primarily: the conceptual design of the interaction artefact; and the maintenance of the consistency of the design with the semantics of the domain and a human factors interpretation of the user requirements with respect to the analysis of extant systems.  The Design Specification phase supports primarily: the detailed design of the interaction artefact.  Mandatory checking and exchange of information with the software engineering method occurs to ensure that the interaction artefact is implementable and to support overall design agreement and consistency.

3. Features of a MUSE Application

MUSE, in order to improve the ability of HCI practitioners to design artefacts that fulfil user requirements, is concerned to ensure that (the features are labelled to permit later reference):

a.        the artefact is considered completely and appropriately at all levels of design, from conceptual to detailed;

b.         the artefact is consistent across all levels of design (including the user requirements);

c.         domain knowledge is assessed and applied to the artefact at appropriate levels of design;

d.         human factors knowledge (as well as software engineering knowledge) is assessed and can be applied to the artefact at appropriate levels of design;

e.         desirable qualities of extant systems are assessed and can be integrated into the artefact at appropriate levels of design;

f.         the design rationale contained in the previous three concerns can be made explicit with respect to the artefact;

g.         MUSE achieves these concerns, in part, by stipulating the generation of products.  An application should, therefore: contain the specified explicit products that have the scope defined by MUSE; are in the notation defined; and have been produced, using the processes defined.  If any product is not relevant, a suitable statement should explain why it is not relevant.

In addition to the products, evidence of the above concerns should be available in an application of MUSE, for example: that suitable human factors knowledge concerning the allocation of function between the user and the device has been applied.

4. The User Requirements

Other HCI research, reported elsewhere (Stork & Long, 1994), required the study of an energy management problem in a home.  This problem had to be observed (and observable), well-defined, and relatively simple.  Because it fulfilled these criteria, the following energy management problem in the home of A and S was selected from a list of such problems compiled by the occupants:

“The domestic routine of A occasionally requires him to remain at home to work in the mornings, rather than to leave earlier with his partner, S, to work at his office.  However, if A leaves after 8 o’clock in the morning, or stays at home to work, then the house is too cold until he turns the gas-powered central heating back on.  If he expects to be at home for a short time after 8 o’clock, then he often uses the one-hour boost facility on the heating controller to turn the heating back on, which can result in him being too cold, if he is at home for longer than expected.  A’s ability to work is adversely affected by being cold and having to control the heating.  A finds it difficult to plan much in advance, either whether he is staying home to work or, if he stays, how long he will stay to work.  The current gas bill is acceptable and an increase could be tolerated, although a decrease would be desirable.”

The user requirements for this application of MUSE were for a bespoke artefact to solve the above problem at a reasonable cost for the envisaged benefits.  The user was expected to be A and he was not prepared to spend much time using or learning to use the new artefact.

4. The Application of MUSE to the User Requirements

The features of a MUSE application are highlighted in this section by reference to their associated label (superscripted at the end of the sentence containing the feature).

4.1 Information Elicitation and Analysis Phase

A list of extant systems that could be assessed (Figure 1—Extant Systems Analysis) was constructed: the existing system; related systems with tasks in common, such as other home energy management systems (for example, home heating control systems); partially related systems, such as home management systems (for example, support for planning food purchases), energy management systems (for example, office heating systems), and general management systems (for example, decision support or security systems)e.  It was decided only to analyse the existing system for the first iteration of the application of MUSE. It was considered that this analysis would lead to a better understanding of the user requirements, would offer most potential for re-use to the design, and might suffice (b,e).

Human factors techniques were used to analyse the existing system in two ways: a task analysis was conducted on the basis of an interview in which A introspected about his days; and A was asked to keep a diary for several mornings during which he stayed at home and left for workd.  These analyses were expressed in ‘structured diagram’ form with accompanying tables as MUSE ‘Task Description’ productsg.  An extract from the highest level of the Task Description of the first and second days of the diary are shown in Figure 2 (the outlined boxes indicate that there is more detail below these nodes in the complete diagram).  The diagram for the first day shows that A performed some early morning tasks and stayed to work; and the diagram for the second day shows that A performed some morning tasks again, but left with S.

To gain an understanding of ‘generic’ mornings (which the design needs to support), a generic Task Description across all days was produced from the diary Task Descriptions of each day (b,g).  Figure 3 shows an extract from the highest level of this description and Figure 4 shows an extract from the accompanying table.  The table demonstrates that implications and speculations concerning the design were considered and explicitly documented throughout this stage of MUSE application (b, c,d,e,f).

Figure 4. An extract from the table of the generic Task Description of the diary analysis of the extant system .

The generic Task Descriptions produced using the diary and the task analysis (based on introspection) were combined into a structured diagram with an accompanying table as a MUSE ‘Generalised Task Model of the extant system’ product (Figure 1) (b,g).  An extract from the highest level of the Generalised Task Model of the extant system is shown in Figure 5.

At this point, it was decided that sufficient analysis of the extant system had been performed to consider the initial design of the target artefacte.  A task-level conceptual design was constructed based on: the user requirements; and the design implications and speculations produced by the analysis of the extant system (a,b,e).  This design was documented in a structured diagram and table as a MUSE ‘Generalised Task Model of the target system’ productg.  Figure 6 shows an extract from the highest level of the Generalised Task Model of the target system, and Figure 7 an extract from the accompanying table (Figure 1).  The extracts demonstrate that the design is consistent with the user requirements and that it arises from the understanding gained through the analysis of the extant systemb,e.  The table identifies a human factors concern that the user must remember to turn the heating off and that this requirement has yet to be addressed (a,d,f).

Figure 6. The structured diagram of the Generalised Task Model of the Target System.

Figure 7. An extract from the table of the Generalised Task Model of the target system.

The initial high-level design, when compared with the high level extant system analysis, suggested a potential for re-use of more detailed extant system features and it was decided to perform a more detailed analysis of the extant system to support that potentiala,e.  Accordingly, a range of MUSE products was developed that analysed the extant system from its conceptual to its detailed designg.  This range of products was complementary to the range of MUSE products to be produced later during design synthesis.  This paper will only show examples of these products to highlight the level of analysis of concern; the recording of design implications and speculations at that level of concern; and to facilitate later examples during the synthesis phase (as required by the features of MUSE application—see earlier) (e,f).

The tasks recorded in the Generalised Task Model that were directly supported by the existing controller (the on-line tasks) were analysed in further detail to produce a structured diagram and table for the MUSE ‘System Task Model of the extant system’ product (e,f,g).  Figure 8 shows the complete structured diagram and Figure 9 an extract from the table.  The table entry provides an example of the documentation of an assessment for re-use of a human factors guideline and its application to the extant system (d,e,f).

Figure 8. The structured diagram of the System Task Model of the extant system.

Figure 9. An extract from the table of the System Task Model of the extant system.

Further MUSE analysis products were developed that: determined the semantics of the domain of the extant system; assessed the impact on the content, format, and mode of presentation of the extant design from off-line tasks (those not supported by the extant controller); and considered a detailed analysis of the extant system (e).  The detailed analysis considered the relationships between the specific behaviours of the user of the extant controller and the tasks of the system; the specific behaviours of the interactive device and the tasks of the system; and the layout and content of the appearance of the extant systeme.  Some of these products are exemplified here: the semantics of the domain were captured by interview and observation in a semantic net diagram and a table in a MUSE ‘Domain of Design Discourse of the extant system’ product (Figure 10); the relationships between the specific behaviours of the interactive device and the tasks of the system were shown in a structured diagram and table in a MUSE ‘Interaction Model of the extant system’ product (extracts in Figures 11 and 12); and the layout of the appearance of the extant system were shown in a layout diagram as a MUSE ‘Pictorial Screen Layout of the extant system’ (Figure 13) (d,e,g).

Figure 12. An extract from the table of the Interface Model of the extant system.

Figure 13. The Pictorial Screen Layout of the extant system.

To ensure consistency with the user requirements and the early assessment of human factors issues a textual summary of the human factors’ concerns was constructed, based on the user requirements and the analysis of the extant system, as a MUSE ‘Statement of User Needs’ product (Figure 1) (b,d,g).  This statement detailed: any explicit design criteria, such as the artefact cost should be acceptable for the benefits; any implicit design criteria, such as the functionality of the existing controller should be retained to support non-weekday-morning tasks; any explicit system performance criteria, such as A must not be cold; any implicit performance criteria, such as A must be permitted to leave home when he desires (constraining should not be considered a solution); and any relevant human factors knowledge, such as an extension of a guideline by Shneiderman (1992) that “human action should be eliminated where no [human] judgement is required” to include “and minimise human action where human judgement is required” (b,c,d,e,g).  This extended guideline confirmed the high-level design expressed in the Generalised Task Model of the target system.

The semantics of the domain were considered and determined to be the same as the semantics of the domain of the extant system (a).  The Domain of Design Discourse of the extant system MUSE product developed during the extant system analysis was re-used as the MUSE ‘Domain of Design Discourse of the target system’ product (e,g).  The product emphasises the importance of the temperature of the house and its relationship with the comfort and location plan of A (c).  The explicit existence of this domain knowledge enabled it to influence the final artefact (c).

The conceptual design of the conjoint user and computer tasks was advanced, maintaining consistency with the accepted foundation of the Generalised Task Model of the target system, and was expressed in a structured diagram and table as the MUSE ‘Composite Task Model’ product (Figure 1 and extracts in Figure 14 and 15) (a,b,g).  Important design decisions were rationalised at this stage: the provision of a controller in the same location as the existing one; and the further provision of a controller near the front door (partially shown in the extracts which show some of the continuation of the design from the Generalised Task Model of the target system) (c,d,e,f).

There was significant re-use of extant systems analysis at this stage with modifications to support the designe.  For example, the decomposition of the ‘control location’ task was re-used from the generic Task Description of the extant system and modified to remove extant heating control tasks if A stays, and to add new heating control tasks on A leaving.  These additions were consistent with the user requirements and the guidelines in the Statement of User Needs (b).  A human factors concern was raised over the need for reminders, in addition to the locations of the controllers, to ensure that A turns off the heating (d).

Figure 15. An extract from the table of the Composite Task Model of the target system.

The on-line tasks (to be supported by the artefact) and off-line tasks (un-supported by the artefact) were identifieda.  Further decomposition of the off-line tasks ensured that there were no content, format, or mode of presentation issues for the target system (a).  (It was possible to re-use the off-line task descriptions from the extant system analysis of tasks (e)).

The design was considered at a lower level of detail by the decomposition of the on-line tasks, documented in a structured diagram and table as a MUSE ‘System Task Model’ product (Figure 1 and Figures 16 and 17) (a,g).  The on-line tasks description diagram shows that they are similar to the analysis of the extant system (e).  This similarity is rationalised in the table (shown) as consistent with the human factors guidelines of ‘transfer of learning’, ‘feedback’ (which the extant system achieved as shown earlier), and ‘consistency’ (Smith & Mosier, 1986) (d,e,f).

The final consideration at this level of detail was the allocation of function between the user and the artefacta.  In this case, it was considered difficult, if not impossible, to allocate the user’s leaving plan to the controller, so it was decided that the controller should simply respond to the user’s control commandsd.  This allocation corresponds with the human factors guideline that humans are generally better than computers at “drawing on experience and adapting decisions to situations” (Shneiderman, 1992) (c).

Figure 16. The structured diagram of the System Task Model of the target system.

Figure 17. An extract from the table of the System Task Model.

4.3 Design Specification Phase

The design of the relationships between the behaviours of the user and those of the designed controllers was documented in a structured diagram and table as a MUSE ‘Interaction Task Model’ product (Figure 1 and Figures 18 and 19) (a,g).  The design is consistent with the System Task Model product, as it represented a decomposition of the user (designated ‘H’—human) input required to perform the task (b).  The extract from the table documents the rationale behind re-using an advance push button to turn off the heating (e,f).  Application of the human factors guideline ensures that ‘consistency’ is maintained between the two controllers (d).

Figure 18. The structured diagram of the Interaction Task Model of the target system.

Figure 19. An extract from the table of the Interaction Task Model of the target system.

The corresponding design of the behaviours of the designed controllers was documented in a structured diagram and table as a MUSE ‘Interface Model’ product (Figure 1 and extracts in Figures 20 and 21) (a,g).  The design is consistent with the System Task Model product and the Interaction Task Model product in that it represents a decomposition of the device (designated ‘C’—computer) output required to perform the task (b).  The design also re-uses behaviour of the extant system, for example: a light (shown earlier) and an advance push button (shown in extract) are specified to support the same tasks, of confirming the change of status and changing the state of the heating, as they support in the extant system (e).

Figure 21. An extract from the table of the Interface Model of the target system.

The control panel layouts were specified and documented in a MUSE ‘Pictorial Screen Layout’ product (Figure 1—Display Design) (a,g).  Since the behaviour of the main controller is the same as the behaviour of the existing controller and the layout of the existing controller is considered satisfactory, the layout of the existing controller was re-used from the analysis of the extant systeme.  The front door controller layout was dimensioned to allow suitable access to the controls (extract in Figure 22) (c).  The layouts’ objects were specified in a MUSE ‘Dictionary of Screen Objects’ product (Figure 1—Display Design—and extract in Figure 23) (a,b,g).

Figure 22. An extract from the Pictorial Screen Layouts (the front door controller layout).

Figure 23. An extract from  the Dictionary of Screen Objects.

Two MUSE products were not considered to be relevant for this design: the ‘Dialogue and Error Message Table’; and the ‘Dialogue and Inter-Task Screen Actuation Diagram’ (Figure 1—Display Design) (a,g).  The former product details error messages and this design requires none, since there is no potential for interface error usage of the controls (a conclusion supported by the analysis of the extant system).  The latter product details the actuations of the screens and the presentation of the error messages.  However, the screens in this design do not change (they are control panel layouts).

4.4 Overall consideration of the features of the MUSE application

The above sections present sufficient detail of the application of MUSE to the user requirements to demonstrate, by reference to the features, that all necessary products were generated as required and that: the artefact was considered completely and appropriately for all levels of design; consistency was maintained across all levels of design (including the user requirements); domain knowledge was assessed and applied to the artefact at appropriate levels of design; human factors knowledge was assessed and applied to the artefact at appropriate levels of design; desirable qualities of extant systems were assessed and integrated, if appropriate, into the design at all levels of design; and the design rationale was made explicit with respect to the artefact.

5. The Artefact

The application of MUSE to the user requirements resulted in a designed artefact that can best be characterised by contrast with the existing heating controller.  The existing controller in the home was programmed to have two heating ‘on-off’ periods: ‘on’ early morning (6.40am) and ‘off’ a bit later (7.20am); and ‘on’ early evening (6.30pm) and ‘off’ at night (10pm).

This existing controller is to be replaced in the designed artefact by one that has the same two ‘on-off’ periods for the weekend, but during the week will: switch on in the morning (6.40am), and switch off at night (10pm); and if the heating is turned off during the day (using an advance button) will turn on again in the early evening (6.30pm).

To complete the artefact, an additional remote heating-controller, with an advance button and a bright status light, is to be added by the front-door.

The occupants of the home will be instructed to use the heating controls as before, except that A should press the advance button on either controller if the status light is ‘on’ just before leaving to go to work during the week.  A is to be considered the user of the designed artefact.

6. An Assessment of the Artefact

Three informal analytic assessments of whether the artefact fulfils the user requirements were conducted.  Firstly, an analytic argument was constructed to show that the introduction of the artefact into the home of A and S should remove the problem.  A form of this analytic argument, commensurate with the user requirements, follows:

“The artefact should support the domestic routine of A which occasionally requires him to remain at home to work in the mornings, rather than leave earlier with his partner, S, to work at his office.  If A leaves after 8 o’clock in the morning, or stays at home to work, then the house should remain warm without intervention, since the gas-powered central heating should remain on rather than turning itself off, which used to cause the house to become too cold and A to become uncomfortable.  Since the house is no longer too cold, A is not required to turn the heating back on.  Therefore, even if A expects to be at home for a short time after 8 o’clock, he should not need to use the one-hour boost facility which used to cause him to be too cold, if he was at home for longer than expected.  A’s ability to work should no longer be adversely affected by him being cold and having to control the heating, since the house is now warm and the heating does not need controlling, until he has finished working.  A finds it difficult to plan much in advance, whether he is staying at work and, if he stays, how long he will stay to work.  The artefact should support this difficulty with planning as the heating should only need controlling during execution of A’s plan and becomes independent of how far in advance of plan execution A can plan.  The gas bill may increase by a small amount which A and S consider tolerable.  A should not be overly taxed by turning the heating off when leaving, or learning to turn the heating off.  The cost of the artefact should be low (approximately £40 for a fully functioning prototype version).”

The second informal analytic assessment involved a group of nine practitioners, five human factors engineers and four software engineers, appraising the artefact specification produced using MUSE.  They were all familiar with the method and the user requirements.  Although some initial objections were raised, following discussion no objections were maintained such that the artefact was considered to fail to fulfil the user requirements.  Additional claims either asserted the artefact fulfilled more than the user requirements (but not less) or that the artefact might have embodied alternative design features.

The third, and last, informal analytic assessment was an expert walkthrough of the artefact specification performed by a human factors engineer.  His report contained the following concluding statement:

“The likely behaviour of the occupants of the house with respect to the system was estimated with respect to a number of scenarios concerning different types of morning events.  It was considered that in the scenario where there was previously a problem (i.e. when A remained at home after 8 o’clock), the system would solve the problem by maintaining A’s comfort, and that A would remember to switch the system off as long as the front door controller was located in a suitably prominent position.  In a situation where A left the house early, his expectations of the system based on the existing system may initially cause him to forget to switch the heating off, as he is currently not required to take any action if he leaves early in the morning.  However, it is to be expected that A would soon learn to adapt his morning routine to include the new task of switching the heating off.  Similarly, if A left the house earlier than S at any time, S might forget to switch the heating off, as the normal morning routine does not require any action on S’s part.  However, if the indication of the system status was designed to be sufficiently conspicuous, and the controller was prominently located, these problems would be less likely to occur than if the controller was located in a less visible position.  At present, there is no evidence in the user requirements or in the analysis of the existing systems that A will ever leave earlier than S; further consultation with A has revealed that it is very seldom the case that A leaves first, and so the problem of S having to remember to operate the system would occur very (and acceptably) infrequently.”

In addition, an empirical assessment has been performed by constructing an interactively-faithful prototype (which does not alter the state of the heating) of the remote heating controller and re-programming the existing controller.  The prototype was placed by the front door in the home and the occupants instructed as for the artefact.  This assessment, which is still ongoing, appears to confirm the above analytic argument (except that an empirical assessment of the gas bill increase has not been possible).

Taken together, the analytic and empirical assessments demonstrate, albeit informally, that the artefact indeed fulfils the user requirements.

7. Conclusions

This paper claims to describe a successful case-study of MUSE: an application of the method to a set of user requirements that produced an interaction artefact which fulfilled the user requirements.  This claim is substantiated in the paper by: describing assessments of the artefact that argue that it fulfils the user requirements; and describing the features of the application of MUSE to the user requirements that argue that the method was applied in the development of the artefact from the user requirements.  The known frontiers of MUSE applications can be said, therefore, to have been extended by this successful case-study.

The case-study user requirements aim to solve a planning and control problem for a human using a device.  They are, therefore, similar to other HCI user requirements that aim to solve planning and control problems (for example: user requirements for an executive information system), although these other HCI user requirements may be less well-defined and more complex.  This similarity suggests that MUSE may support successful case-studies based on less well-defined and more complex HCI planning and control user requirements.

8. Further Research

It is recognised that further assessment of the artefact here would strengthen the associated claims.  Firstly, a fully-functioning prototype is currently in production and will be installed and evaluated in the home of A and S.  This prototype will, unlike the currently installed prototype, control the heating and permit assessment of the change to the gas bill.  Secondly, further research is attempting to construct a more formal argument than the informal one presented here.  Lastly, any further expert assessment forthcoming from the readers of this paper will be happily incorporated into the research ((Analytic assessments based on this paper are welcomed and should be sent to the first author.  Empirical assessments are by prior arrangement.))

Further research is also needed concerning means of assessing applications of MUSE, particularly by larger project teams and in corporate situations.  Research on both these issues is currently being conducted by the Ergonomics and HCI Unit, UCL, under the aegis of the European Systems and Software Initiative (ESSI).  MUSE has been taught to, and configured for, a consortium of industrial partners.  They are now applying the method to assess the costs and benefits of its addition to differing software engineering baseline methods.  The user requirements involved in the applications vary in definition and complexity. (ESSI, 1994.)

A comparative assessment of MUSE against other human-factors methods would also be informative.  The well-defined and relatively simple nature of the user requirements (and artefact) described in this paper suggest their use as a preliminary benchmark for human-factors methods ((Perhaps as a welcome change to the ubiquitous automated bank teller user requirements and artefact.))

Acknowledgements

The research associated with this paper was carried out under an EPSRC CASE studentship sponsored by Schlumberger Industries.  Views expressed in the paper are those of the authors and should not be attributed to EPSRC or Schlumberger Industries.

References

ESSI (1994).  Mid-term report, ESSI project 10290, Benefits of Integrating Usability and Software Engineering Methods (BIUSEM).  Prepared by: Philips Research Laboratories, Redhill, UK; Philips Analytical, Almelo, The Netherlands; EDS, Middlesex, UK; Electrowatt Engineering Services (UK) Ltd, Horsham, UK; and Ergonomics & HCI Unit, University College London, London, UK.  December 1994.

Lim K.Y. and Long. J.B. (1992).  A Method for (Recruiting) Methods: Facilitating Human Factors Input to System Design.  In: Proceedings of the ACM Annual Conference on Human Factors in Computing Systems (CHI’92), Monterey (USA), May 3-7, R. Brooks (ed).  ACM.  pp. 549-556.  1992.

Lim K.Y. and Long. J.B. (1994).  The MUSE Method for Usability Engineering.  First Edition, 1995.  Cambridge Series on Human-Computer Interaction.  Cambridge University Press.  1995.

Lim K.Y., Long J.B., and Silcock N. (1990).  Requirements, Research and Strategy for Integrating Human Factors with Structured Analysis and Design Methods: The Case of the Jackson System Development Method.  In: Contemporary Ergonomics, Proceedings of the Ergonomics Society’s 1990 Conference, Leeds, April 1990, E.J. Lovesey (ed).  Taylor and Francis, London.  pp. 32-38.  1990.

Lim K.Y., Long J.B., and Silcock N. (1992).  Integrating Human Factors with System Development: An Illustrated Overview.  In: Ergonomics (Special Issue on Cognitive Ergonomics III), 1992, 33(12), P. Barber and J. Laws (eds).  Taylor and Francis, London.  pp. 1135-1161.  1992.

Lim K.Y., Silcock N., and Long J.B. (1991).  Case-Study Illustration of a Structured Method for User Interface Design.  In: Contemporary Ergonomics, Proceedings of the Ergonomics Society’s 1991 Conference, Southampton, April 1991, E.J. Lovesey (ed).  Taylor and Francis, London.  pp. 235-342.  1991.

Shneiderman B.(1992).  Designing the User Interface: Strategies of Effective Human-Computer Interaction.  Second edition, 1992.  Addison-Wesley Publishing Company.  1992.

Smith S.L. and Mosier J.N. (1986).  Guidelines for Designing User Interface Software.  Final report number MTR-10090 ESD-TR-86-278, 1986, The Mitre Corporation.  1986.

Stork A. and Long J.B. (1994).  A Planning and Control Design Problem in the Home:  Rationale and a Case Study.  In: Proc. International Working Conference on Home-Oriented Informatics, Telematics and Automation, Amager, Denmark, 27 June-1 July 1994, K. Bjerg and K. Borreby (eds).  Copenhagen, Denmark: University of Copenhagen.  pp. 419-428.  1994.

Adam Stork and John Long

Ergonomics and HCI Unit University College London
26 Bedford Way London WC1H 0AP

Accepted for Publication: 1994  .  In Proceedings of the International Working Conference on Home-Oriented Informatics, Telematics and Automation, Amager, Denmark.

Abstract

This paper seeks to contribute to more effective design practice in the longer term both for Human Computer Interaction in general and for Home Informatics in particular.  The ultimate goal of more effective design practice is held to be practice supported by ‘general design principles’ (Dowell and Long 89), which are conceptualised, operationalised, tested, and generalised.  The value of such a goal can be seen in the ‘hard’ Engineering disciplines, such as Electrical or Mechanical Engineering, which have general design principles that are better conceptualised, operationalised, tested, and generalised than those in other design disciplines, such as the emergent discipline of Human Computer Interaction.  These Engineering disciplines can offer earlier success and a better guarantee in the solution of design problems.

The paper first argues that a pre-requisite of, and so one potential contribution towards, the development of general design principles for Human Computer Interaction would be to develop operationalisations of specific design problems and specific design solutions that are better conceptualised and operationalised than existing such operationalisations.   The initial conceptualisation and operationalisation of a specific design problem is then described.

The specific design problem, object of the case study reported here, was one of the failure to maintain desired comfort and costs by a heating system of a typical home.  The resulting specific design problem specification is considered to be better conceptualised and operationalised than other specifications of such design problems.  It expresses the domain of application and interactive worksystem as they relate to the design problem of supporting the planning and control of comfort and costs of heating in the home.

The paper concludes that the conceptualisation and operationalisation raise no undue concerns for the aim of developing general design principles in the longer term and demonstrates some modest progress towards this aim.  Further research towards achieving this aim is identified.

Keywords

Energy Management Systems,  Engineering, Home Informatics, Human Computer Interaction, Planning and Control

Introduction: Scope and Aims of this Paper

Technology continues to penetrate the home (Home Informatics).  Even the home, however, is a challenge for the discipline of Human Computer Interaction (HCI), an emerging discipline which has demonstrated only limited success in other more developed technological areas.  This paper describes work that aims to improve the success of HCI in the longer term for Home Informatics in particular, but also for HCI in general.

The paper describes work that is part of an overall effort to improve HCI.  The research can be summarised by describing the ultimate goal of HCI knowledge to be practice supported by ‘general design principles’ (Dowell and Long 89), which are conceptualised, operationalised, tested, and generalised to offer an earlier and a better guarantee of success in application.  A strategy is described for the development of general design principles for which a pre-requisite is the development of operationalisations of specific HCI design problems and specific HCI design solutions that are better conceptualised and operationalised than existing such operationalisations.

The work can be summarised as the development of a conception of the general HCI design problem and the operationalisation of that conception for a specific HCI design problem.  This operationalisation is to be better conceptualised and operationalised than existing such operationalisations.

The achievements towards, and implications for, the overall research consequent on this detailed work are then considered.

Engineering: The Likely Future of Human Computer Interaction

Dowell and Long have made proposals concerning the nature of the discipline of HCI (Dowell and Long 89; Long and Dowell 89).  Firstly, they have characterised HCI as a design discipline rather than as a scientific discipline: HCI knowledge supports HCI practice which is to provide solutions to general HCI design problems.  Secondly, they have identified from this statement a means of assessing the effectiveness of HCI knowledge and used this means to argue for certain properties of that knowledge for it to be effective.  These properties can be used to assess the current HCI discipline and propose a more effective (a possible future) discipline.

The effectiveness of HCI knowledge is dependent on its ability to support HCI practice to solve general HCI design problems.  Dowell and Long identify four necessary properties of effective HCI knowledge: it must be conceptualisable; operationalisable; testable; and generalisable.

The current state of the HCI discipline is characterised as that of a ‘Craft’ discipline.  Knowledge is currently implicit and informal, consisting of ‘heuristics’; and practice is that of ‘implement and test’ (and iterate).  Heuristics are poorly, if at all, conceptualised (often only through example practice), which leads to them being difficult to operationalise, therefore unlikely to be testable, and so unlikely to be generalisable.  This characterisation represents an understanding as to the current state of the poor guarantee of success offered by current HCI knowledge and practice.

The alternative, more effective, HCI discipline is characterised as that of an ‘Engineering’ discipline.  Knowledge would need to be conceptualised, with explicit, complete, consistent, and formal definitions, to be operationalisable, testable, and generalisable.  Dowell and Long call this knowledge ‘engineering principles’, and practice would be that of ‘specify then implement’.  HCI engineering principles would offer a high guarantee of success of application in practice, similar to that currently enjoyed by the existing (‘hard’) Engineering disciplines like Electrical or Mechanical Engineering.

A Strategy for Developing Human Computer Interaction Engineering

The development of HCI engineering principles requires the development of knowledge to support practice.  HCI practice is the provision of general design solutions to general design problems, which are general over types of user, types of computer, and types of domain of application.  HCI knowledge contains general relationships between general HCI design problems and general HCI design solutions.

It is suggested that one possible means of developing this HCI knowledge is to identify general relationships between general design problems and solutions from specific relationships between specific design problems and solutions, where a specific design problem and solution is defined as being particular to a user, a computer system, and a domain of application.  Such a development requires the conceptualisation and operationalisation of specific HCI design problems and solutions from a conception of the complete general HCI design problem and solution which is general over all types of user, all types of computer, and all types of domain of application, hereafter referred to as ‘a conception of the general HCI design problem and solution’.

A conception is understood to be a set of concepts, which are abstractions on a class of objects based on their common aspects, and their relations.  Conceptualisation is the process of generating a conception.  Operationalisation is the process of instancing a conception to produce an operationalisation.  An operationalisation of a conception is a set of less abstract concepts (related to the concepts in the conception) that reference observables in the ‘real’ world.  It is convenient to introduce the concept of ‘a set of metrics’ and the process of ‘metrication’.  Metrication is the process of instancing an operationalisation (in the limit) to produce a set of metrics.  Metrics quantify the less abstract concepts of the operationalisation in an observable relation with the ‘real’ world.

For the knowledge to support engineering principles, the operationalisation of the specific design problems and solutions needs to be explicit and formal, where formal is understood as a representation that has defined rules of syntax and semantics, and is therefore understandable by some people for some purpose.  Formality requires the metrication of the operationalisations of the conceptions of the specific HCI design problem and solution.

It is proposed that tractable re-design problems be selected, then described as ‘user requirements’, and finally design solutions to these problems developed using best current HCI practice (craft ‘heuristics’, including evaluation, applied by an HCI designer).  These design problems and solutions will be operationalised according to the conception of the specific HCI design problem.  It is considered easier to operationalise installed design solutions than to operationalise design problems.  The selection of re-design problems, instead of design problems, enables the operationalisation (according to the conception of the specific HCI design solution) of the existing (installed) design ‘solution’ (to an earlier, probably implicit, design problem) to guide the operationalisation of the re-design problem.  The selection of a tractable re-design problem ensures: that the differences between the operationalisations of the current design ‘solution’ and the tractable re-design problem be minimal; and that a design solution exists.

This approach is the one adopted here.  The current state of the present work is that conceptions of the general HCI design problem and solution have been identified and instantiated to conceptions of the specific HCI design problem and solution.  An initial tractable re-design problem has been described as ‘user requirements’.  The current design ‘solution’ has been operationalised, according to the conception of the specific HCI design solution, and this operationalisation has guided the operationalisation of the initial tractable re-design problem, according to the conception of the specific HCI design problem.

The user requirements for this initial tractable re-design problem were intended to be operationalisable according to the conception of the specific design problem in order to prevent concentration, for the moment, on the relationship between the user requirements and the operationalisation of the specific design problem.

The strategy adopted to perform the operationalisations of the conceptions of the specific HCI design problem and solution was to develop an explicit operationalisation as a step towards an explicit and formal set of metrics.  The current work, as presented here, describes the explicit operationalisations of the conceptions of the specific HCI design problem and solution with the specific tractable re-design problem and current specific design ‘solution’.

Conceptions of the General Human Computer Interaction Engineering Design Problem and Solution

Dowell and Long (89) have proposed a conception of the general HCI design problem and this conception is presented here with some additional detail.  No attempt is made to indicate in all cases where their work ends and the present begins to achieve an uninterrupted exposition of the concepts.  Readers interested in the differences, however, are directed to read the original paper by Dowell and Long.

Dowell and Long state the general HCI design problem informally as ‘the design of interactive worksystems for performance’.  They propose a more precise description as follows (slightly amended for typographical considerations):

‘the design of behaviours constituting a worksystem {S} whose actual performance (Pa) conforms with some desired performance (Pd).  And to design {S} would require the design of human behaviours {U} interacting with computer behaviours {C}.  Hence conception of the general design problem of an engineering discipline of HCI is expressed as:

Specify then implement {U} and {C}, such that

{U} interacting with {C} = {S}Pa=Pd

where           Pd=fn(Qd, Kd)

Qd expresses the desired quality of the products of work within the given domain of application;
Kd expresses acceptable (i.e., desired) costs incurred by the worksystem, i.e. by both human and computer.’

This statement expresses Dowell and Long’s distinction between the behavioural system (that is the interactive worksystem) that performs work, and the world of work (that is the domain of application, within which the work is performed).  It is inferred from their work that Pa is a function of the actual quality of the products of work within a particular domain of application (Qa) and the actual costs incurred by a particular worksystem (Ka).

Furthermore, an expression of the general HCI design solution is inferred from their work as:

‘a specification and implementation of {U} and {C}, such that

{U} interacting with {C} = {S}Pa=Pd

where           Pd=fn(Qd, Kd)
and                      Pa=fn(Qa, Ka)’

It follows from these expressions that a statement of the general HCI design problem requires a statement for the desired performance of the desired interactive worksystem whereas a statement of the general HCI design solution requires a statement of the actual performance of the actual interactive worksystem.

The components of performance are now considered, following Dowell and Long, for desired performance and actual performance.  Important occurrences of the concepts are highlighted to aid their identification and permit an informal assessment that the later operationalisation indeed operationalises these concepts.

A Conception of Desired Performance

The desired performance, Pd, is conceptualised as a function of the desired quality of the products of work, Qd, within the domain of application and the acceptable or desired costs, Kd, incurred by the interactive worksystem.

The interactive worksystem boundary criteria allow assertion of the behavioural system which constitutes the interactive worksystem, that system ‘whose purpose is to achieve and satisfy … common goal[s]’.  The domain boundary critieria allow assertion of the world of work which constitutes the domain of application, that world of work which is determined by the requirement to express these common goals.

A Conception of Actual Performance

Actual performance, Pa, is conceptualised as a function of the actual quality of the products of work, Qa, within the given domain of application and the current or actual costs, Ka, incurred by the worksystem.

The interactive worksystem boundary criteria and domain boundary criteria are the same as for the conception of desired performance.

A Conception of Desired Quality

Dowell and Long conceptualise the world of work as consisting of objects that have attributes that have a set of possible states (defining their affordance for change).  The desired quality of the products of work to be achieved by the interactive worksystem are conceptualised as transformations of states of attributes of objects that are desirable, called product goals.  These objects and their attributes are conceptualised as abstract or physical, and related or unrelated.  The transformations described by a product goal can be identified for each attribute, and these transformations are termed task goals.

Dowell and Long describe the difference between abstract and physical attributes of objects as: ‘abstract attributes of objects are attributes of information and knowledge’; and ‘physical attributes of objects are attributes of energy and matter’.  They also conceive that ‘different attributes of an object emerge at different levels within an hierarchy of levels of complexity’ and in general abstract attributes emerge at a higher level than physical attributes.  Similarly, ‘objects are described at different levels of specification commensurate with their levels of complexity’.  Furthermore, attributes of objects are related to attributes of other objects both across and within levels of complexity.

A Conception of Actual Quality

The actual quality of the products of work achieved by the interactive worksystem are conceptualised as similar to desired quality, with transformations of states of attributes of objects that are achieved, called product achieved goals, and transformations for each attribute, called task achieved goals.

A Conception of Desired Costs

Dowell and Long conceptualise the interactive worksystem (the behavioural system) as ‘human and computer behaviours together performing work’.  They distinguish human behaviour as purposeful from computer behaviour as purposive.  They claim that human behaviours correspond with the transformation of objects in a domain and that an expression of them must ‘at least be expressed at a level commensurate with the level of description of the transformation of objects in the domain’.  These statements would appear to hold for computer and interactive worksystem behaviours.

These behaviours can be abstract or physical .  Abstract behaviours ‘are generally the acquisition, storage, and transformation of information.  They represent and process information at least concerning: domain objects and their attributes, attribute relations and attribute states, and the transformations required by goals’.  Physical behaviours express abstract behaviours and are ‘related in an hierarchy of behaviour types’.

Dowell and Long conceptualise the user as having cognitive, conative, and affective behaviours.  ‘The cognitive aspects of the user are those of knowing, reasoning and remembering, etc.; the conative aspects are those of acting, trying and persevering, etc.; and the affective aspects are those of being patient, caring, and assuring, etc.’

Dowell and Long conceptualise humans and computers as ‘having (separable) structures that support their (separable) behaviours’.  Furthermore, ‘Human structures may be physical (neural, bio-mechanical, and physiological) or mental (representational schemes and processes)’.  Similarly, computer structures may be physical or abstract.

Dowell and Long claim that ‘work performed by interactive worksystems always incurs resource costs’.  They identify resource costs as behavioural or structural and associated with the human or the computer (separable).  These costs can be further associated with abstract (mental) and physical behaviours or structures.  Examples of resource costs related to the human are: physical workload for human physical behavioural costs; mental workload for human abstract (mental) behavioural costs; physical development and deterioration for human physical structural costs; and mental development and deterioration for human abstract (mental) structural costs.  Examples of resource costs related to the computer are: energy emission and consumption for computer physical behavioural costs; software and functional resource (transaction and access resources) usage for computer abstract behavioural costs; system (hardware) development and degradation for computer physical structural costs; and software and functional development (and degradation) for computer abstract structural costs.

The desired costs are conceptualised as the necessary resource costs of the interactive worksystem to achieve the desired quality.

A Conception of Actual Costs

The actual costs are conceptualised as the actual resource costs of the interactive worksystem to achieve the actual quality.

Conceptions of the Specific Human Computer Interaction Engineering Design Problem and Solution

The conceptions of the specific HCI design problem and solution are operationalised from the conceptions of the general HCI design problem and solution.  The specific HCI design problem and solution are particular to a user, a computer system, and a domain of application (and by definition).

The (specific) desired performance is conceptualised as a function of the desired quality of the products of work within a particular domain of application and the desired costs incurred by a particular interactive worksystem consisting of a particular user and a particular computer system.

The (specific) actual performance is conceptualised as a function of the actual quality of the products of work within a particular domain of application and the actual costs incurred by a particular interactive worksystem consisting of a particular user and a particular computer system.

An Operationalisation of a Specific HCI Design Problem

The following tractable specific HCI (re-)design problem was selected as the first specific HCI design problem to be operationalised according to the conception of the specific HCI design problem developed above.  This operationalisation was guided by first operationalising the current (inadequate) design ‘solution’ consistent with the strategy adopted for this research.  The following is a (shortened) statement of the user requirements for the specific HCI (re-)design problem.

‘if A stays at home to work and leaves after 8 o’clock in the morning then the house is too cold until the gas-powered central heating is turned back on.  If he expects to be at home for a short time then he often uses the one-hour boost facility on the heating controller to turn the heating back on, which can result in him being too cold if he is at home for longer than expected.  The current gas bill is acceptable and an increase could be tolerated although a decrease would be desirable.’

The particular user is A, the particular computer system (or device) is the energy management heating system, and the particular domain of application is the planning and control of the comfort of A in the home of A.  The concepts are highlighted to aid identification and permit an informal assessment that this operationalisation in fact operationalises the earlier concepts.

The Specific Actual Performance

The specific actual performance is operationalised as the union of the specific actual quality and the specific actual costs.  The interactive worksystem boundary criteria are operationalised by the requirement that the constituents of the interactive worksystem have the common goals of the current (level of) achievement and satisfaction of the planning and control of the comfort of A in the home of A using the radiators.  The domain boundary criteria are operationalised by the requirement that the constituents of the domain of application express the current (level of) achievement and satisfaction of these common goals.

The Specific Desired Performance

The specific desired performance is operationalised as the union of the specific desired quality and the specific desired costs.  The interactive worksystem boundary criteria are operationalised by the requirement that the constituents of the interactive worksystem have the common goals of the desired achievement and satisfaction of the planning and control of the comfort of A in the home of A using the radiators.  The domain boundary criteria are operationalised by the requirement that the constituents of the domain of application express the desired achievement and satisfaction of these common goals.

The Specific Actual Quality

The specific actual domain of application has two main physical objects: A and the house.  A has a physical attribute of temperature and an abstract attribute of comfort.  The attribute of comfort is related to the attribute of temperature having a range of acceptable temperatures (between 35.75˚C and 37.5˚C).

The second physical object is the house, which has physical objects that are the rooms.  The rooms have a physical attribute of their temperature and physical objects of the radiators.  The radiators have a physical attribute of their temperature.  The temperature of the room is related to the temperature of A (an approximately linear relationship) and the temperature of the radiators (related through convection, u-value of the room, etc.).  The temperatures of the radiators are controlled by the heating system in the interactive worksystem.

The current states of the temperatures of the radiators result in the state of the comfort attribute of A being ‘not comfortable’ at some times.  This state is a task achieved goal and defines the product achieved goal of the actual quality by interpretation of the relationships between this attribute and the other attributes in the actual domain of application.

The Specific Desired Quality

In this case, the specific desired quality is very similar to the specific actual quality of the current inadequate ‘solution’ described above.  The difference is that the specific desired quality has a task goal to maintain the state of A’s comfort attribute as ‘comfortable’ instead of a task achieved goal of ‘not comfortable’.  This task goal results in the product goals being different from the product achieved goals.

The Specific Actual Costs

There are two main sub-systems in the interactive worksystem: the user (A) and the heating system (a common combination boiler system with a simple two-period time controller).  The heating system has the following interacting physical behaviours: receive control of the temperatures (at times) of the radiators; receive press of advance button; and receive press of a one-hour boost button.  The user has the following interacting physical behaviours: perform control of the temperatures (at times) of the radiators; perform press of advance button; and perform press of one-hour boost button.  The non-interacting physical behaviours include, as examples: for the heating system, fire-up the main burners from the pilot light; and for the user, move to (and from) the location of the heating controller.  A further non-interacting physical behaviour of the heating system, and an example of a behaviour that corresponds with the transformation of the attributes of objects in the domain of application, is flow of hot-water to and from the radiators.

The physical structures can be derived from the physical behaviours, for example the heating controller has a physical structure of a one-hour boost button and the user has a physical structure of a body.

The abstract behaviours of the heating controller are the execution of an algorithm on some data structures, where the algorithm and data structures are the abstract structures of the heating controller.  The algorithm and data structures can be described in pseudo-code as follows:

‘boostTime = off           // Counter decrements with time.  At zero is ‘off’.

task 1 – constant
currentTime                      // Maintains the current time.
heatingState = domain.radiatorOnOffState

if (currentTime > 6.40am & currentTime < 7.20am | boostTime != off)
heatingState = on
else
heatingState = off

task2 – on boostButton.pressed
if (boostTime = off)
boostTime = 1hour
else
boostTime = off’

The cognitive abstract (mental) behaviours of the user are the operation of mental processes on mental representations, where the mental processes and the mental representation are the cognitive abstract (mental) structures.  The mental processes and mental representation can be described in pseudo-code as follows:

‘comfort = domain.A.comfort
leaving  = leavingPlan(comfort).leaveWithinHalfHour?

if (comfort = uncomfortable.tooCold & !leaving)
moveToController
pressBoostButton
returnFromController’

The conative and affective abstract (mental) behaviours and structures are not operationalised here.

The physical behavioural costs of the heating system can be operationalised by the energy consumption and release due to the behaviours of the heating system, for example:  potential energy is stored and dissipated when the boost button is pressed and released.  The abstract behavioural costs of the heating system can be operationalised by the transaction and access resource usage incurred in performing the algorithm and data structures given above, for example: the frequency of access of the timer.

.

The Specific Desired Costs

The specific desired costs are similar to the specific actual costs of the current inadequate ‘solution’ described above.  The differences, in this case, are due to statements in the user requirements and interpretations of the user requirements.  The ‘current gas bill is acceptable and an increase could be tolerated although a decrease would be desirable’ (from the original problem statement) suggests that the operationalisation of the physical structural costs of the heating system should be within a range that allows for this desirable decrease or acceptable increase in gas usage.  It is assumed that the heating system can be upgraded and, therefore, the operationalisation of the physical structural costs of the heating system should also be within a range that allows for a different installation and maintenance price.

It is assumed that the other costs should either remain the same, and be operationalised in the same manner—for example the user physical structural costs—, or decrease if possible, and be operationalised to be within a range—for example the user physical behavioural costs.

Further Research

The operationalisation of the planning and control specific HCI design problem needs to be developed further, both in terms of the range of concepts operationalised and the operationalisation to the level of metrics.  A rigorous demonstration that the operationalisation indeed operationalises the concepts needs to be constructed.  A design solution has been developed using best current HCI practice, and this solution will be installed and evaluated soon.  The conceptualisation of the specific HCI design solution will allow this design solution to be operationalised.  The conceptualisation and operationalisation of this initial specific HCI design problem and solution is expected to enable specific relationships to be identified.  Further specific HCI design problems and solutions will then be addressed and general relationships, as putative general design principles, identified where possible.  The overall research strategy will then be assessed and pursued or amended.

Conclusion

The current operationalisation of the specific HCI design problem can, informally, be said: to be developed from the conception of the general HCI design problem and to have failed to identify properties in the conception of the general HCI design problem that could not be operationalised.  There arise, therefore, no negative implications for the conception of the general HCI design problem.  Hence, there are no negative implications for the strategy for the development of general HCI design principles.

The validation of the conception and operationalisation of general HCI design problems, and the strategy for the development of general HCI design principles, will be achieved if putative general HCI design principles are produced and validated.

The current operationalisation, however, can be considered to be better conceptualised and operationalised than other operationalisations of the general HCI design problem.  The very production of the operationalisation demonstrates that this conception is better than other conceptions of general HCI design problems (and HCI knowledge) that have not produced operationalisations, for example Storrs’ conception of HCI (89).  The authors are not aware of other operationalisations of general HCI design problems based on this conception (or indeed other conceptions) of the general HCI design problem.

The operationalisation also serves as a interesting example that contains explicit, partly structured, but informal HCI design knowledge of a planning and control Home Informatics design problem that could be used by Craft HCI practitioners.

Overall, this work is proceeding according to a well-developed view of the requirements for a discipline of HCI for Home Informatics.  The work aims to offer long term benefits towards the development of general engineering design principles for HCI and shows progress towards that aim.  Such engineering general HCI design principles would offer a better and earlier guarantee, one that is significantly greater than the current (low) guarantee of existing HCI Craft-based knowledge.  This improvement will benefit the challenging area for HCI of introducing technology into the home.

Acknowledgements

The research associated with this paper was carried out under a SERC CASE studentship sponsored by Schlumberger Industries.  Views expressed in the paper are those of the authors and should not necessarily be attributed to SERC or Schlumberger Industries.

References

Dowell J. & Long J. B. (1989). Towards a conception for an engineering discipline of human factors. Ergonomics, November 1989, 32(11), pp. 1513-1535.

Long J.B. & Dowell J. (1989).  Conceptions for the discipline of HCI: Craft, Applied Science and Engineering.  In: A. Sutcliffe & L. Macaulay (eds).  Proceedings of the Fifth Conference of the BCS HCI SIG, Nottingham, England, 5-8 September 1989, Cambridge: Cambridge University Press.  pp. 9-32.

Storrs, G (1989).  A conceptual model of human-computer interaction?  Behaviour and Information Technology, 5(4), pp. 323-334.

John Long and John Dowell

Ergonomics Unit, University College London,
26 Bedford Way, London. WC1H 0AP.

Pre-print: In: Sutcliffe, A. andMacaulay, L., (eds.) People and Computers V: Proceedings of the Fifth Conference of the British Computer Society.(pp. pp. 9-32). Cambridge University Press: Cambridge, UK.  http://discovery.ucl.ac.uk/15292/ 

The theme of HCI ’89 is ‘the theory and practice of HCI’. In providing a general introduction to the Conference, this paper develops the theme within a characterisation of alternative conceptions of the discipline of Human-Computer Interaction (HCI). First, consideration of disciplines in general suggests their complete definition can be summarised as: ‘knowledge, practices and a general problem having a particular scope, where knowledge supports practices seeking solutions to the general problem’. Second, the scope of the general problem of HCI is defined by reference to humans, computers, and the work they perform. Third, by intersecting these two definitions, a framework is proposed within which different conceptions of the HCI discipline may be established, ordered, and related. The framework expresses the essential characteristics of the HCI discipline, and can be summarised as: ‘the use of HCI knowledge to support practices seeking solutions to the general problem of HCI’. Fourth, three alternative conceptions of the discipline of HCI are identified. They are HCI as a craft discipline, as an applied scientific discipline, and as an engineering discipline. Each conception is considered in terms of its view of the general problem, the practices seeking solutions to the problem, and the knowledge supporting those practices; examples are provided. Finally, the alternative conceptions are reviewed, and the effectiveness of the discipline which each offers is comparatively assessed. The relationships between the conceptions in establishing a more effective discipline are indicated.

Published in: People and Computers V. Sutcliffe A. and Macaulay L. (ed.s). Cambridge University Press, Cambridge. Proceedings of the Fifth Conference of the BCS HCI SIG, Nottingham 5-8 September 1989.

Contents

1. Introduction

1.1. Alternative Interpretations of the Theme

1.2. Alternative Conceptions of HCI: the Requirement for a Framework

1.3. Aims

2. A Framework for Conceptions of the HCI Discipline

2.1. On the Nature of Disciplines

2.2. Of Humans Interacting with Computers

2.3. The Framework for Conceptions of the HCI Discipline

3. Three Conceptions of the Discipline of HCI

3.1. Conception of HCI as a Craft Discipline

3.2. Conception of HCI as an Applied Science Discipline

3.3. Conception of HCI as an Engineering Discipline

4. Summary and Conclusions

1. Introduction

HCI ’89 is the fifth conference in the ‘People and Computers’ series organised by the British Computer Society’s HCI Specialist Group. The main theme of HCI ’89 is ‘the theory and practice of HCI’. The significance of the theme derives from the questions it prompts and from the Conference aims arising from it. For example, what is HCI? What is HCI practice? What theory supports HCI practice? How well does HCI theory support HCI practice? Addressing such questions develops the Conference theme and so advances the Conference goals.

1.1. Alternative Interpretations of the Theme

Any attempt to address these questions, however, admits no singular answer. For example, some would claim HCI as a science, others as engineering. Some would claim HCI practice as ‘trial and error’, others as ‘specify and implement’. Some would claim HCI theory as explanatory laws, others as design principles. Some would claim HCI theory as directly supporting HCI practice, others as indirectly providing support. Some would claim HCI theory as effectively supporting HCI practice, whilst others may claim such support as non-existent. Clearly then, there will be many possible interpretations of the theme ‘the theory and practice of HCI’.

Answers to some of the questions prompted by the theme will be related. Different answers to the same question may be mutually exclusive; for example, types of practice as ‘trial and error’ or ‘specify and implement’ will likely be mutually exclusive. Answers to different questions may also be mutually exclusive; for example, HCI as engineering would likely exclude HCI theory as explanatory laws, and HCI practice as ‘trial and error’. And moreover, answers to some questions may constrain the answers to other questions; for example, types of HCI theory, perhaps design principles, may constrain the type of practice, perhaps as ‘specify and implement’.

1.2. Alternative Conceptions of HCI: the Requirement for a Framework

It follows that we must admit the possibility of alternative, and equally legitimate, conceptions of the HCI discipline – and therein, of its theory and practice. A conception of the HCI discipline offers a unitary view; its value lies in the coherence and completeness with which it enables understanding of the discipline, how the discipline operates, and its effectiveness. So for example, a conception of HCI might be either of a scientific or of an engineering discipline; its view of the theory and practice of the discipline would be different in the two cases. Its view of how the discipline might operate, and its expectations for the effectiveness of the discipline, would also be different in the two cases. This paper identifies alternative conceptions of HCI, and attempts a comparative assessment of the (potential) effectiveness of the discipline which each views. The requirement for identifying the different conceptions is both prompted and required by the development of the Conference theme.

To advance alternative conceptions of HCI, however, it is necessary first to formulate some form of analytic structure to ensure that conceptions supposed as alternatives are both complete and of the same subject, rather than being conceptions of complementary, or simply different, subjects. A suitable structure for this purpose would be a framework identifying the essential characteristics of the HCI discipline. By such a framework, instances of conceptions of the HCI discipline – claimed to be substantively different, but equivalent – might be established, ordered, and related. And hence, so might their views of its theories and practices.

The aims of this paper follow from the need to identify alternative conceptions of HCI as a discipline. The aims are described in the next section.

1.3. Aims

To address and develop the Conference theme of ‘the theory and practice of HCI’ – and so to advance the goals of HCI ’89 – the aims of this paper are as follows:

(i) to propose a framework for conceptions of the HCI discipline

(ii) to identify and exemplify alternative conceptions of the HCI discipline in terms of the framework

(iii) to evaluate the effectiveness of the discipline as viewed by each of the conceptions, and to indicate the possible relationships between the conceptions in establishing a more effective discipline.

2. A Framework for Conceptions of the HCI Discipline

Two prerequisites of a framework for conceptions of the HCI discipline are assumed. The first is a definition of disciplines appropriate for the expression of HCI. The second is a definition of the province of concern of the HCI discipline which, whilst broad enough to include all disparate aspects, enables the discipline’s boundaries to be identified. Each of these prerequisites will be addressed in turn (Sections 2.1. and 2.2.). From them is derived a framework for conceptions of the HCI discipline (Section 2.3.). Source material for the framework is to be found in (Dowell & Long [1988]; Dowell & Long [manuscript submitted for publication]; and Long [1989]).

2.1. On the Nature of Disciplines

Most definitions assume three primary characteristics of disciplines: knowledge; practice; and a general problem.

All definitions of disciplines make reference to discipline knowledge as the product of research or more generally of a field of study. Knowledge can be public (ultimately formal) or private (ultimately experiential). It may assume a number of forms; for example, it may be tacit, formal, experiential, codified – as in theories, laws and principles etc. It may also be maintained in a number of ways; for example, it may be expressed in journals, or learning systems, or it may only be embodied in procedures and tools. All disciplines would appear to have knowledge as a component (for example, scientific discipline knowledge, engineering discipline knowledge, medical discipline knowledge, etc). Knowledge, therefore, is a necessary characteristic of a discipline.

Consideration of different disciplines suggests that practice is also a necessary characteristic of a discipline. Further, a discipline’s knowledge is used by its practices to solve a general (discipline) problem. For example, the discipline of science includes the scientific practice addressing the general (scientific) problem of explanation and prediction. The discipline of engineering includes the engineering practice addressing the general (engineering) problem of design. The discipline of medicine includes the medical practice addressing the general (medical) problem of supporting health. Practice, therefore, and the general (discipline) problem which it uses knowledge to solve, are also necessary characteristics of a discipline.

Clearly, disciplines are here being distinguished by the general (discipline) problem they address. The scientific discipline addresses the general (scientific) problem of explanation and prediction, the engineering discipline addresses the general (engineering) problem of design, and so on. Yet consideration also suggests those general (discipline) problems each have the necessary property of a scope. Decomposition of a general (discipline) problem with regard to its scope exposes (subsumed) general problems of particular scopes. ((Notwithstanding the so-called ‘hierarchy theory ‘ which assumes a phenomenon to occur at a particular level of complexity and to subsume others at a lower level (eg, Pattee, 1973).)) This decomposition allows the further division of disciplines into sub-disciplines.

For example, the scientific discipline includes the disciplines of physics, biology, psychology, etc., each distinguished by the particular scope of the general problem it addresses. The discipline of psychology addresses a general (scientific) problem whose particular scope is the mental and physical behaviours of humans and animals. It attempts to explain and predict those behaviours. It is distinguished from the discipline of biology which addresses a general problem whose particular scope includes anatomy, physiology, etc. Similarly, the discipline of engineering includes the disciplines of civil, mechanical, electrical engineering, etc. Electrical engineering is distinguished by the particular scope of the general (engineering) problem it addresses, i.e., the scope of electrical artefacts. And similarly, the discipline of medicine includes the disciplines of dermatology, neurology etc., each distinguished by the particular scope of the general problem it addresses.

Two basic properties of disciplines are therefore concluded. One is the property of the scope of a general discipline problem. The other is the possibility of division of a discipline into sub-disciplines by decomposition of its general discipline problem.

Taken together, the three necessary characteristics of a discipline (and the two basic properties additionally concluded), suggest the definition of a discipline as: ‘the use of knowledge to support practices seeking solutions to a general problem having a particular scope’. It is represented schematically in Figure 1. This definition will be used subsequently to express HCI.

2.2. Of Humans Interacting with Computers

The second prerequisite of a framework for conceptions of the HCI discipline is a definition of the scope of the general problem addressed by the discipline. In delimiting the province of concern of the HCI discipline, such a definition might assure the completeness of any one conception (see Section 1.2.).

HCI concerns humans and computers interacting to perform work. It implicates: humans, both individually and in organisations; computers, both as programmable machines and functionally embedded devices within machines (stand alone or networked); and work performed by humans and computers within organisational contexts. It implicates both behaviours and structures of humans and computers. It implicates the interactions between humans and computers in performing both physical work (ie, transforming energy) and abstract work (ie, transforming information). Further, since both organisations and individuals have requirements for the effectiveness with which work is performed, also implicated is the optimisation of all aspects of the interactions supporting effectiveness.

Taken together, these implications suggest a definition of the scope of the general (discipline) problem of HCI. It is expressed, in summary, as ‘humans and computers interacting to perform work effectively’; it is represented schematically in Figure 2. This definition, in conjunction with the general definition of disciplines, will now enable expression of a framework for conceptions of the HCI discipline.

2.3. The Framework for Conceptions of the HCI Discipline

The possibility of alternative, and equally legitimate, conceptions of the discipline of HCI was earlier postulated. This section proposes a framework within which different conceptions may be established, ordered, and related.

Given the definition of its scope (above), and the preceding definition of disciplines, the general problem addressed by the discipline of HCI is asserted as: ‘the design of humans and computers interacting to perform work effectively’. It is a general (discipline) problem of design : its ultimate product is designs. The practices of the HCI discipline seek solutions to this general problem, for example: in the construction of computer hardware and software; in the selection and training of humans to use computers; in aspects of the management of work, etc. HCI discipline knowledge supports the practices that provide such solutions.

The general problem of HCI can be decomposed (with regard to its scope) into two general problems, each having a particular scope. Whilst subsumed within the general problem of HCI, these two general problems are expressed as: ‘the design of humans interacting with computers’; and ‘the design of computers interacting with humans’. Each problem can be associated with a different sub-discipline of HCI. Human Factors (HF), or Ergonomics, addresses the problem of designing the human as they interact with a computer. Software Engineering (SE) addresses the problem of designing the computer as it interacts with a human. With different – though complementary – aims, both sub-disciplines address the problem of designing humans and computers which interact to perform work effectively. However, the HF discipline concerns the physical and mental aspects of the human and is supported by HF discipline knowledge. The SE discipline concerns the physical and software aspects of the computer and is supported by SE discipline knowledge.

Hence, we may express a framework for conceptions of the discipline of HCI as:

‘the use of HCI knowledge to support practices seeking solutions to the general problem of HCI of designing humans and computers interacting to perform work effectively. HCI knowledge is constituted of HF knowledge and SE knowledge, respectively supporting HF practices and SE practices. Those practices respectively address the HF general problem of the design of humans interacting with computers, and the SE general problem of the design of computers interacting with humans’. The framework is represented schematically in Figure 3.

Importantly, the framework supposes the nature of effectiveness of the HCI discipline itself. There are two apparent components of this effectiveness. The first is the success with which its practices solve the general problem of designing humans and computers interacting to perform work effectively. It may be understood to be synonymous with ‘product quality’. The second component of effectiveness of the discipline is the resource costs incurred in solving the general problem to a given degree of success – costs incurred by both the acquisition and application of knowledge. It may be understood to be synonymous with ‘production costs’.

The framework will be used in Section 3 to establish, order, and relate alternative conceptions of HCI. It supports comparative assessment of the effectiveness of the discipline as supposed by each conception.

3. Three Conceptions of the Discipline of HCI

A review of the literature was undertaken to identify alternative conceptions of HCI, that is, conceptions of the use of knowledge to support practices solving the general problem of the design of humans and computers interacting to perform work effectively. The review identified three such conceptions. They are HCI as a craft discipline; as an applied scientific discipline; and as an engineering discipline. Each conception will be described and exemplified in terms of the framework.

3.1. Conception of HCI as a Craft Discipline

Craft disciplines solve the general problems they address by practices of implementation and evaluation. Their practices are supported by knowledge typically in the form of heuristics; heuristics are implicit (as in the procedures of good practice) and informal (as in the advice provided by one craftsperson to another). Craft knowledge is acquired by practice and example, and so is experiential; it is neither explicit nor formal. Conception of HCI as a craft discipline is represented schematically in Figure 4.

HCI as a craft discipline addresses the general problem of designing humans and computers interacting to perform work effectively. For example, Prestel uses Videotex technology to provide a public information service which also includes remote electronic shopping and banking facilities (Gilligan & Long [1984]). The practice of HCI to solve the general problem of Prestel interaction design is by implementation, evaluation and iteration (Buckley [1989]). For example, Videotex screen designers try out new solutions – for assigning colours to displays, for selecting formats to express user instructions, etc. Successful forms of interaction are integrated into accepted good practice – for example, clearly distinguishing references to domain ‘objects’ (goods on sale) from references to interface ‘objects’ (forms to order the goods) and so reducing user difficulties and errors. Screen designs successful in supporting interactions are copied by other designers. Unsuccessful interactions are excluded from subsequent implementations – for example, the repetition of large scale logos on all the screens (because the screens are written top-to-bottom and the interaction is slowed unacceptably).

HCI craft knowledge, supporting practice, is maintained by practice itself. For example, in the case of Videotex shopping, users often fail to cite on the order form the reference number of the goods they wish to purchase. A useful design heuristic is to try prompting users with the relevant information, for example, by reminding them on the screen displaying the goods that the associated reference number is required for ordering and should be noted. An alternative heuristic is to try re-labelling the reference number of the goods, for example to ‘ordering’ rather than reference number. Heuristics such as these are formulated and tried out on new implementations and are retained if associated with successful interactions. To illustrate HCI as a craft discipline more completely, there follows a detailed example taken from a case history reporting the design of a text editor (Bornat & Thimbleby [1989]).

Bornat and Thimbleby are computer scientists who, in the 1970s, designed a novel text display editor called ‘Ded’. The general problem of HCI for them was to design a text editor which would enable the user to enter text, review it, add to it, to reorganise its structure and to print it. In addition, the editor was to be easy to use. They characterise their practice as ‘production’ (implementation as used here) suffused by design activity. Indeed, their view is that Ded was not designed but evolved. There was always a fully working version of the text editor to be discussed, even from the very early days.

The evolution, however, was informed by ‘user interface principles’ (which they sometimes call theories and at other times call design ideas) which they invented themselves, tried out on Ded, retained if successful and reformulated if unsuccessful. The status of the principles at the time of their use would be termed here craft discipline knowledge or heuristics. (Subsequent validation of the heuristics as other than craft knowledge would of course be possible, and so change this status.) For example, ‘to indicate to users exactly what they are doing, try providing rapid feedback for every keypress’. Most feedback was embodied in changes to the display (cursor movements, characters added or deleted, etc.) which were visible to the user. However, if the effect of a keypress was not visible, there was no effect, but a bell rang to let the user know. In this way, the craft heuristic supporting the SE craft practice – by informing the design of the computer interacting with the human – can be expressed as: ‘if key depression and no display change, then ring bell’. The heuristic also supported HF craft practice – by informing the design of the human interacting with the computer. It may be expressed as: ‘if key pressed and no display change seen, and bell heard, then understand no effect of keypress (other than bell ring)’.

Another example of a craft heuristic used by Bornat and Thimbleby (and one introduced to them by a colleague) is ‘to ensure that information in the computer is what the user thinks it is, try using only one mode’. The heuristic supported SE practice, informing the design of the computer interacting with the human – ‘if text displayed, and cursor under a character, and key depression, then insert character before cursor position’. The heuristic also supported HF practice, informing the design of the human interacting with the computer – ‘if text seen, and cursor located under a character, and key has been pressed, then only the insertion of a character before the cursor position can be seen to be effected (but nothing else)’.

In summary, the design of Ded by Bornat and Thimbleby illustrates the critical features of HCI as a craft discipline. They addressed the specific form of the general problem (general because their colleague suggested part of the solution – one ‘mode’ – and because their heuristics were made available to others practising the craft discipline). Their practices involved the iterative implementation and evaluation of the computer interacting with the human, and of the human interacting with the computer. They were supported by craft discipline heuristics – for example: ‘simple operations should be simple, and the complex possible’. Such craft knowledge was either implicit or informal; the concepts of ‘simple’ and ‘complex’ remaining undefined, together with their associated operations (the only definitions being those implicit in Ded and in the expertise of Bornat and Thimbleby, or informal in their report). And finally, the heuristics were generated for a purpose, tried out for their adequacy (in the case of Ded) and then retained or discarded (for further application to Ded). This too is characteristic of a craft discipline. Accepting that Ded met its requirements for both functionality (enter text, review text, etc.) and for usability (use straight away, etc) – as claimed by Bornat and Thimbleby – it can be accepted as an example of good HCI craft practice.

To conclude this characterisation of HCI as a craft discipline, let us consider its potential for effectiveness. As earlier proposed (Section 2.3), an effective discipline is one whose practices successfully solve its general problem, whilst incurring acceptable costs in acquiring and applying the knowledge supporting those practices (see Dowell & Long [1988]). HCI as a craft discipline will be evaluated in general for its effectiveness in solving the general problem of designing humans and computers interacting, as exemplified by Bornat and Thimbleby’s development of Ded in particular.

Consideration of HCI as a craft discipline suggests that it fails to be effective (Dowell & Long [manuscript submitted for publication]). The first explanation of this – and one that may at first appear paradoxical – is that the (public) knowledge possessed by HCI as a craft discipline is not operational. That is to say, because it is either implicit or informal, it cannot be directly applied by those who are not associated with the generation of the heuristics or exposed to their use. If the heuristics are implicit in practice, they can be applied by others only by means of example practice. If the heuristics are informal, they can be applied only with the help of guidance from a successful practitioner (or by additional, but unvalidated, reasoning by the user). For example, the heuristic ”simple operations should be simple, and the complex possible’ could not be implemented without the help of Bornat and Thimbleby or extensive interpretation by the designer. The heuristic provides insufficient information for its operationalisation. In addition, since craft heuristics cannot be directly applied to practice, practice cannot be easily planned and coordinated. Further, when HF and SE design practice are allocated to different people or groups, practice cannot easily be integrated. (Bornat was responsible for both HF and SE design practice and was also final arbiter of design solutions.) Thus, with respect to the requirement for its knowledge to be operational, the HCI craft discipline fails to be effective.

If craft knowledge is not operational, then it is unlikely to be testable – for example, whether the ‘simple’ operations when implemented are indeed ‘simple’, and whether the ‘complex’ operations when implemented are indeed ‘possible’. Hence, the second reason why HCI as a craft discipline fails to be effective is because there is no guarantee that practice applying HCI craft knowledge will have the consequences intended (guarantees cannot be provided if testing is precluded). There is no guarantee that its application to designing humans and computers interacting will result in their performing work effectively. For example, the heuristic of providing rapid feedback in Ded does not guarantee that users know what they are doing, because they might not understand the contingencies of the feedback. (However, it would be expected to help understanding, at least to some extent, and more often than not). Thus, with respect to the guarantee that knowledge applied by practice will solve the general HCI problem, the HCI craft discipline fails to be effective.

If craft knowledge is not testable, then neither is it likely to be generalisable – for example, whether ‘simple’ operations that are simple when implemented in Ded are also ‘simple’ when implemented in a different text editor. Hence, the third explanation of the failure of HCI as a craft discipline to be effective arises from the absence of generality of its knowledge. To be clear, if being operational demands that (public) discipline knowledge can be directly applied by others than those who generated the knowledge, then being general demands that the knowledge be guaranteed to be appropriate in instances other than those in which it was generated. Yet, the knowledge possessed by HCI as a craft discipline applies only to those problems already addressed by its practice, that is, in the instances in which it was generated. Bornat and Thimbleby’s heuristics for solving the design problem of Ded may have succeeded in this instance, but the ability of the heuristics to support the solution of other design problems is unknown and, until a solution is attempted, unknowable. The suitability of the heuristics ‘ignore deficiencies of the terminal hardware’ and ‘undo one keystroke at a time’ for a system controlling the processes of a nuclear power plant could only be established by implementation and evaluation in the context of the power plant. In the absence of a well defined general scope for the problems to be addressed by the knowledge supporting HCI craft practice, each problem of designing humans and computers interacting has to be solved anew. Thus, with respect to the generality of its knowledge, the HCI craft discipline fails to be effective.

Further consideration of HCI as a craft discipline suggests that the costs incurred in generating, and so in acquiring craft knowledge, are few and acceptable. For example, Bornat and Thimbleby generated their design heuristics as required, that is – as evaluation showed the implementation of one heuristic to fail. Further, heuristics can be easily communicated (if not applied) and applied now (if applicable). Thus, with respect to the costs of acquiring its knowledge, HCI as a craft discipline would seem to be effective.

In summary, although the costs of acquiring its knowledge would appear acceptable, and although its knowledge when applied by practice sometimes successfully solves the general problem of designing humans and computers interacting to perform work effectively, the craft discipline of HCI is ineffective because it is generally unable to solve the general problem. It is ineffective because its knowledge is neither operational (except in practice itself), nor generalisable, nor guaranteed to achieve its intended effect – except as the continued success of its practice and its continued use by successful craftspeople.

3.2. Conception of HCI as an Applied Science Discipline

The discipline of science uses scientific knowledge (in the form of theories, models, laws, truth propositions, hypotheses, etc.) to support the scientific practice (analytic, empirical, etc.) of solving the general problem of explaining and predicting the phenomena within its scope (structural, behavioural, etc.) (see Section 3.1). Science solves its general problem by hypothesis and test. Hypotheses may be based on deduction from theory or induction from regularities of structure or behaviour associated with the phenomena. Scientific knowledge is explicit and formal, operational, testable and generalisable. It is therefore refutable (if not proveable; Popper [1959]).

Scientific disciplines can be associated with both HF – for example, psychology, linguistics, artificial intelligence, etc. and SE – for example, computer science, artificial intelligence, etc. Psychology explains and predicts the phenomena of the mental life and behaviour of humans (for example, the acquisition of cognitive skill (Anderson [1983])); computer science explains, and predicts the phenomena of the computability of computers as Turing-compatible machines (for example, as concerns abstract data types (Scott [1976])).

An applied science discipline is one which recruits scientific knowledge to the practice of solving its general problem – a design problem. HCI as an applied science discipline uses scientific knowledge as an aid to addressing the general problem of designing humans and computers interacting to perform work effectively. HCI as an applied science is represented schematically in Figure 5.

An example of psychological science knowledge which might be recruited to support the HF practice concerns the effect of feedback on sequences of behaviour, for example, noise and touch on keyboard operation, and confirmatory feedback on the sending of electronic messages (Hammond [1987]). (Feedback is chosen here because it was also used to exemplify craft discipline knowledge (see Section 3.1) and the contrast is informative.) Psychology provides the following predictive truth proposition concerning feedback: ‘controlled sequences need confirmatory feedback (both required and redundant); automated sequences only need required feedback during the automated sequence’. (The research supporting this predictive (but also explanatory proposition) would be expected to have defined and operationalised the terms – ‘feedback’, ‘controlled’, etc. and to have reported the empirical data on which the proposition is based.)

However, as it stands, the proposition cannot contribute to the solution of the HF design problem such as that posed by the development of the text-editor Ded (Bornat & Thimbleby [1989] – see Section 3.1). The proposition only predicts the modifications of behaviour sequences by feedback under a given set of conditions. It does not prescribe the feedback required by Ded to achieve effective performance of work (enter text, review it, etc.; to be usable straight away etc.).

Predictive psychological knowledge can be made prescriptive. For example Hammond transforms the predictive truth proposition concerning feedback into the following prescriptive proposition (or ‘guideline’): “When a procedure, task or sequence is not automatic to users (either because they are novice users or because the task is particularly complex or difficult), provide feedback in a number of complementary forms. Feedback should be provided both during the task sequence, to inform the user that things are progressing satisfactorily or otherwise, and at completion, to inform the user that the task sequence has been brought to a close satisfactorily or otherwise”.

However, although prescriptive, it is so with respect to the modifiability of sequences of behaviour and not with respect to the effective performance of work. Although application of the guideline might be expected to modify behaviour (for example, decrease errors and increase speed), there is no indication of how the modification (either in absolute terms, or relative to other forms of feedback or its absence) would ensure any particular desired effective performance of work. Nor can there be, since its prescriptive form has not been characterised, operationalised, tested, and generalised with respect to design for effective performance (but only the knowledge on which it is based with respect to behavioural phenomena).

As a result, the design of a system involving feedback, configured in the manner prescribed by the guideline, would still necessarily proceed by implementation, evaluation, and iteration. For example, although Bornat and Thimbleby appear not to have provided complementary feedback for the novice users of Ded, but only feedback by keypress (and not in addition on sequence completion – for example, at the end of editing a command), their users appear to have achieved the desired effective performance of work of entering text, using Ded straight away etc.

Computer science knowledge might similarly be recruited to support SE practice in solving the problem of designing computers interacting with humans to perform work effectively. For example, explanatory and predictive propositions concerning computability, complexity, etc. might be transformed into prescriptive propositions informing system implementation, perhaps in ways similar to the attempt to achieve ‘effective computability’ (Kapur & Srivs [1988]). Alternatively, predictive computer science propositions might support general prescriptive SE principles, such as modularity, abstraction, hiding, localization, uniformity, completeness, confirmability, etc. (Charette [1986]). These general principles might in turn be used to support specific principles to solve the SE design problem of computers interacting with humans.

However, as in the case of psychology, for as long as the general problem of computer science is the explanation and prediction of computability, and not the design of computers interacting with humans to perform work effectively, computer science knowledge cannot be prescriptive with respect to the latter. Whatever computer science knowledge (for example, use of abstract data types) or general SE principles (for example, modularity) informed or could have informed Bornat and Thimbleby’s development of Ded, the design would still have had to proceed by implementation, evaluation and iteration, because neither the computer science knowledge nor the SE principles address the problem of designing for the effective performance of work – entering text, using Ded straight away, etc.

To illustrate HCI as an applied science discipline more completely, there follows a detailed example taken from a case history reporting the design of a computer-aided learning system to induct new undergraduates into their field of study – cognitive psychology (Hammond & Allinson [1988]).

Hammond and Allinson called upon three areas of psychological knowledge, concerned with understanding and learning, to support the design of their system. These were ‘encoding specificity’ theory (Tulving [1972]), ‘schema’ theory (Mandler [1979]), and ‘depth of processing’ theory (Craik & Lockhart [1972]). Only the first will be used as an example here. ‘Encoding specificity’ and ‘encoding variability’ explain and predict peoples’ memory behaviours. ‘Encoding specificity’ asserts that material can be recalled if it contains distinctive retrieval cues that can be generated at the time of recall. ‘Encoding variability’ asserts that multiple exposure to the same material in different contexts results in easier recall, since the varied contexts will result in a greater number of potential retrieval cues.

On the basis of this psychological knowledge, Hammond and Allinson construct the guideline or principle: ‘provide distinctive and multiple forms of representation.’ They followed this prescription in their learning system by using the graphical and dynamic presentation of materials, working demonstrations and varied perspectives of the same information. However, although the guideline might have been expected to modify learning behaviour towards that of the easier recall of materials, the system design would have had to proceed by implementation, evaluation, and iteration. The theory of encoding specificity does not address the problem of the design of effective learning, in this case – new undergraduate induction, and the guideline has not been defined, operationalised, tested or generalised with respect to effective learning. Effective induction learning might follow from application of the guideline, but equally it might not (in spite of materials being recalled).

Although Hammond and Allinson do not report whether computer science knowledge was recruited to support the solution of the SE problem of designing the computer interacting with the undergraduates, nor whether general SE principles were recruited, the same conclusion would follow as for the use of psychological knowledge. Effective induction learning performance might follow from the application of notions such as effective computability, or of principles such as modularity, but equally it might not (in spite of the computer’s program being more computably effective and better structured).

In summary, the design of the undergraduate induction system by Hammond and Allinson illustrates the critical features of HCI as an applied science discipline. They addressed the specific form of the general problem (general because the knowledge and guidelines employed were intended to support a wide range of designs). Their practice involved the application of guidelines, the iterative implementation of the interacting computer and interacting human, and their evaluation. The implementation was supported by the use of psychological knowledge which formed the basis for the guidelines. The psychological knowledge (encoding specificity) was defined, operationalised, tested and generalised. The guideline ‘provide distinctive and multiple forms of representation’ was neither defined, operationalised, tested nor generalised with respect to effective learning performance.

Finally, consider the effectiveness of HCI as an applied science discipline. An evaluation suggests that many of the conclusions concerning HCI as a craft discipline also hold for HCI as an applied science discipline. First, its science knowledge cannot be applied directly, not – as in the case of craft knowledge – because it is implicit or informal, but because the knowledge is not prescriptive; it is only explanatory and predictive. Its scope is not that of the general problem of design. The theory of encoding specificity is not directly applicable.

Second, the guidelines based on the science knowledge, which are not predictive but prescriptive, are not defined, operationalised, tested or generalised with respect to desired effective performance. Their selection and application in any system would be a matter of heuristics (and so paradoxically of good practice). Even if the guideline of providing distinctive and multiple forms of representation worked in the case of undergraduate induction, it could not be generalised on the basis of this good practice alone.

Third, the application of guidelines based on science knowledge does not guarantee the consequences intended, that is effective performance. The provision of distinctive and multiple forms of representation may enhance learning behaviours, but not necessarily such as to achieve the effective undergraduate induction desired.

HCI as an applied science discipline, however, differs in two important respects from HCI as a craft discipline. Science knowledge is explicit and formal, and so supports reasoning about the derivation of guidelines, their solution and application (although one might have to be a discipline specialist so to do). Second, science knowledge (of encoding specificity, for example) would be expected to be more correct, coherent and complete than common sense knowledge concerning learning and memory behaviours.

Further, consideration of HCI as an applied science discipline suggests that the costs incurred in generating, and so in acquiring applied science knowledge, are both high (in acquiring science knowledge) and low (in generating guidelines). Whether the costs are acceptable depends on the extent to which the guidelines are effective. However, as indicated earlier, they are neither generalisable nor offer guarantees of effective performance.

In summary, although its knowledge when applied by practice in the form of guidelines sometimes solves the general problem of designing humans and computers interacting to perform work effectively, the applied science discipline is ultimately ineffective because it is generally unsuccessful in solving the general problem and its costs may be unacceptable. It fails to be effective principally because its knowledge is not directly applicable and because the guidelines based on its knowledge are neither generalisable, nor guaranteed to achieve their intended effect.

3.3. Conception of HCI as an Engineering Discipline

The discipline of engineering may characteristically solve its general problem (of design) by the specification of designs before their implementation. It is able to do so because of the prescriptive nature of its discipline knowledge supporting those practices – knowledge formulated as engineering principles. Further, its practices are characterised by their aim of ‘design for performance’. Engineering principles may enable designs to be prescriptively specified for artefacts, or systems which when implemented, demonstrate a prescribed and assured performance. And further, engineering disciplines may solve their general problem by exploiting a decompositional approach to design. Designs specified at a general level of description may be systematically decomposed until their specification is possible at a level of description of their complete implementation. Engineering principles may assure each level of specification as a representation of the previous level.

A conception of HCI as an engineering discipline is also apparent (for example: Dix & Harrison [1987]; Dowell & Long [manuscript submitted for publication]). It is a conception of HCI discipline knowledge as (ideally) constituted of (HF and SE) engineering principles, and its practices (HF and SE practices) as (ideally) specifying then implementing designs. This Section summarises the conception (schematically represented in Figure 6) and attempts to indicate the effectiveness of such a discipline.

The conception of HCI engineering principles assumes the possibility of a codified, general and testable formulation of HCI discipline knowledge which might be prescriptively applied to designing humans and computers interacting to perform work effectively. Such principles would be unequivocally formal and operational. Indeed their operational capability would derive directly from their formality, including the formality of their concepts – for example, the concepts of ‘simple’ and ‘complex’ would have an explicit and consistent definition (see Section 3.1).

The complete and coherent definition of concepts, as necessary for the formulation of HCI engineering principles, would occur within a public and consensus conception of the general problem of HCI. A proposal for the form of such a conception (Dowell & Long [manuscript submitted for publication]), intended to promote the formulation of HCI engineering principles, can be summarised here. It dichotomises ‘interactive worksystems’ which perform work, and ‘domains of application’ in which work originates, is performed, and has its consequences. An interactive worksystem is conceptualised as the interacting behaviours of a human (the ‘user’) and a computer: it is a behavioural system. The user and computer constitute behavioural systems in their own right, and therefore sub-systems of the interactive worksystem. Behaviours are the trajectory of states of humans and computers in their execution of work. The behaviours of the interactive worksystem are reflexive with two independent structures, a human structure of the user and a hardware and software structure of the computer. The behaviours of the interactive worksystem are both physical and informational, and so also are its structures. Further, behaviour incurs a resource cost, distinguished as the ‘structural’ resource cost of establishing and maintaining the structure able to support behaviour, and the ‘behavioural’ resource cost of recruiting the structure to express behaviour.

The behaviours of an interactive worksystem intentionally effect, and so correspond with, transformations of objects. Objects are physical and abstract and exhibit the affordance for transformations arising from the state potential of their attributes. A domain of application is a class of transformation afforded by a class of objects. An organisations` requirements for specific transformations of objects are expressed as product goals; they motivate the behaviours of an interactive worksystem.

The effectiveness of an interactive worksystem is expressed in the concept of performance. Performance assimilates concepts expressing the transformation of objects with regard to its satisfying a product goal, and concepts expressing the resource costs incurred in realising that transformation. Hence, performance relates an interactive worksystem with a domain of application. A desired performance may be specified for any worksystem attempting to satisfy a particular product goal.

The concepts described enable the expression of the general problem addressed by an engineering discipline of HCI as: specify then implement user behaviour {U} and computer behaviour {C}, such that {U} interacting with {C} constitutes an interactive worksystem exhibiting desired performance (PD). It is implicit in this expression that the specification of behaviour supposes and enables specification of the structure supporting that behaviour. HCI engineering principles are conceptualised as supporting the practices of an engineering HCI discipline in specifying implementable designs for the interacting behaviours of both the user and computer that would achieve PD.

This conception of the general problem of an engineering discipline of HCI supposes its further decomposition into two related general problems of different particular scopes. One problem engenders the discipline of HF, the other the discipline of SE; both disciplines being incorporated in HCI. The problem engendering the discipline of SE is expressed as: specify then implement {C}, such that {C} interacting with {U} constitutes an interactive worksystem exhibiting PD. The problem engendering the discipline of HF is expressed as: specify then implement {U}, such that {U} interacting with {C} constitutes an interactive worksystem exhibiting PD.

The disciplines of SE and HF might each possess their own principles. The abstracted form of those principles is visible. An HF engineering principle would take as input a performance requirement of the interactive worksystem, and a specified behaviour of the computer, and prescribe the necessary interacting behaviour of the user. An SE engineering principle would take as input the performance requirement of the interactive worksystem, and a specified behaviour of the user, and prescribe the necessary interacting behaviour of the computer.

Given the independence of their principles, the engineering disciplines of SE and HF might each pursue their own practices, having commensurate and integrated roles in the development of interactive worksystems. Whilst SE specified and implemented the interacting behaviours of computers, HF would specify and implement the interacting behaviours of users. Together, the practices of SE and HF would aim to produce interactive worksystems which achieved PD.

It is the case, however, that the contemporary discipline of HF does not possess engineering principles of this idealised form. Dowell & Long [manuscript submitted for publication) have postulated the form of potential HF engineering principles for application to the training of designers interacting with particular visualisation techniques of CAD systems. A visualisation technique is a graphical representational form within which images of artefacts are displayed; for example, the 21/2 D wireframe representational form of the Necker cube. The supposed principle would prescribe the visual search strategy {u} of the designer interacting with a specified display behaviour {c} of the computer (supported by a specified visualisation technique) to achieve a desired performance in the ‘benchmark’ evaluation of a design.

Neither does the contemporary discipline of SE possess engineering principles of the idealised form discussed. However, formal models of the interaction of display editors proposed by Dix and Harrison [1987] may show potential for development in this respect. For example, Dix and Harrison model the (behavioural) property of a command that is ‘passive’, a command having no effect on the ‘data’ component of the computer’s state. Defining a projection from state into result as r: SR, a passive command c has the property that r(s) = r(c(s)). Although the model has a formal expression, the user behaviour interacting with the (passive) computer behaviour is only implied, and the model makes no reference to desired performance.

It is likely the case, however, that some would claim the (idealised) conception of HCI as an engineering discipline to be unrealiseable. They might justify their objection by claiming the general problem of HCI to be ‘too soft’ to allow the development of engineering principles – that human behaviour is too indeterministic (too unspecifiable) to be subject to such principles. Yet human behaviour can be usefully deterministic to some degree – as demonstrated, for example, by the response of driver behaviour to traffic system protocols. There may well be at least a commensurate potential for the development of HCI engineering principles.

To conclude this summary description of the conception of an engineering discipline of HCI, we might consider the potential effectiveness of such a discipline. As before, effectiveness is evaluated as the success with which the discipline might solve its general problem, and the costs incurred with regard to both the acquisition and application of knowledge.

First, HCI engineering principles would be a generaliseable knowledge. Hence, application of principles to solving each new design problem could be direct and efficient with regard to costs incurred. The discipline would be effective. Second, engineering HCI principles would be operational, and so their application would be specifiable. The further consequence of this would be that the roles of HF and SE in Systems Development could be specified and integrated, providing better planned and executed development programmes. The minimisation of application costs would result in an effective discipline. Third, engineering principles would have a guaranteed efficacy. Because they would be operational, they would be testable and their reliability and generality could be specified. Their consequent assurance of product quality would render effective an engineering discipline of HCI.

Finally, consideration of HCI as an engineering discipline suggests that the costs of formulating engineering principles would be severe. A research programme committed to formulating even a basic corpus of HCI engineering principles might only be conceived as a long-term endeavour of extreme scale.

In summary, although the costs of their formulation would be severe, the potential of a corpus of engineering principles for improving product quality is large, and so also might be the potential for effectiveness of an engineering discipline of HCI.

4. Summary and Conclusions

This paper has developed the Conference theme of ‘the theory and practice of HCI’. Generalisation of the theme, in terms of a framework for conceptions of the HCI discipline, has shown that in addition to theory and practice, the theme needs to explicitly reference the general problem addressed by the discipline of HCI and the scope of the general problem.

The proposal made here is that the general problem of HCI is the design of humans and computers interacting to perform work effectively. The qualification of the general problem as ‘design’, and the addition to the scope of that problem of ‘…. to perform work effectively’, has important consequences for the different conceptions of HCI (see Section 3). For example, since design is not the general problem of science, scientific knowledge (for example, psychology or computer science) cannot be recruited directly to the practice of solving the general problem of design (see Barnard, Grudin & Maclean [1989]). Further, certain attempts to develop complete engineering principles for HCI fail to qualify as such, because they make no reference to ‘…. to perform work effectively’ (Dix & Harrison [1987]; Thimbleby [1984]).

Development of the theme indicated there might be no singular conception of the discipline of HCI. Although all conceptions of HCI as a discipline necessarily include the notion of practice (albeit of different types), the concept of theory is more readily associated with HCI as an applied science discipline, because scientific knowledge in its most correct, coherent and complete form is typically expressed as theories. Craft knowledge is more typically expressed as heuristics. Engineering knowledge is more typically expressed as principles. If HCI knowledge is limited to theory, and theory is presumed to be that of science, then other conceptions of HCI as a discipline are excluded (for example, Dowell & Long [manuscript submitted for publication]).

Finally, generalisation of the Conference theme has identified two conceptions of HCI as a discipline as alternatives to the applied science conception implied by the theme. The other two conceptions are HCI as a craft discipline and HCI as an engineering discipline. Although all three conceptions address the general problem of HCI, they differ concerning the knowledge recruited to solve the problem. Craft recruits heuristics; applied science recruits theories expressed as guidelines; and engineering recruits principles. They also differ in the practice they espouse to solve the general problem. Craft typically implements, evaluates and iterates (Bornat & Thimbleby [1989]); applied science typically selects guidelines to inform implementation, evaluation and iteration (although guidelines may also be generated on the basis of extant knowledge, e.g. – Hammond & Allinson [1988]); and engineering typically would specify and then implement (Dowell & Long [1988]).

The different types of knowledge and the different types of practice have important consequences for the effectiveness of any discipline of HCI. Heuristics are easy to generate, but offer no guarantee that the design solution will exhibit the properties of performance desired. Scientific theories are difficult and costly to generate, and the guidelines derived from them (like heuristics) offer no final guarantee concerning performance. Engineering principles would offer guarantees, but are predicted to be difficult, costly and slow to develop.

The development of the theme and the expression of the conceptions of HCI as a discipline – as craft, applied science and engineering – can usefully be employed to explicate issues raised by, and of concern to, the HCI community. Thus, Landauer’s complaint (Landauer [1987a]) that psychologists have not brought to HCI an impressive tool kit of design methods or principles can be understood as resulting from the disjunction between psychological principles explaining and predicting phenomena, and prescriptive design principles required to guarantee effective performance of work (see Section 3.2). Since research has primarily been directed at establishing the psychological principles, and not at validating the design guidelines, then the absence of an impressive tool kit of design methods or principles is perhaps not so surprising.

A further issue which can be explained concerns the relationship between HF and SE during system development. In particular, there is a complaint by SE that the contributions of HF to system development are ‘too little’, too late’ and unemployable (Walsh, Lim, Long, & Carver [1988]). Assuming HCI to be an applied science discipline, HF contributions are too little because psychology does not address the general problem of design and so fails to provide a set of principles for the solution of that problem. HF contributions are too late, because they consist largely of evaluations of designs already implemented, but without the benefit of HF. They are unemployable, because they were never specified, and because implemented designs can be difficult, if not impossible, and costly to modify. Within an HCI engineering discipline, HF contributions would be adequate (because within the scope of the discipline’s problem); on time (because specifiable); and implementable (because specified). Landauer’s plea (Landauer [1987b]) that HF should extend its practice from implementation evaluation to user requirements identification and the creation of designs to satisfy those requirements can be similarly explicated.

Lastly, Carroll and Campbell’s claim (Carroll & Campbell [1988]) that HCI research has been more successful at developing methodology than theory can be explicated by the need for guidelines to express psychological knowledge and the need to validate those guidelines formally, and the absence of engineering principles, plus the importation of psychology research methods into HCI and the simulation of good (craft) practice. The methodologies, however, are not methodological principles which guarantee the solution of the design problem (Dowell & Long [manuscript submitted for publication]), but procedures to be tailored anew in the manner of a craft discipline. Thus, relating the conceptions of HCI as a set of possible disciplines provides insight into whether HCI research has been more successful at developing methodologies than theories.

In addition to explicating issues already formulated, the development of the Conference theme and the expression of the conceptions of HCI as a discipline raise two novel issues. The first concerns reflexivity both with respect to the general design problem and with respect to the creation of discipline knowledge. It is often assumed that only HCI as an applied scientific discipline (by means of guidelines) and as an engineering discipline (by means of principles) are reflexive with respect to the general design problem. The conception of HCI as a craft discipline, however, has shown that it is similarly reflexive – by means of heuristics. Concerning the creation of discipline knowledge, it is often assumed that only the solution of the general discipline problem requires the reflexive cognitive act – of reason and intuition concerning the objects of activity (Kant [1781]). However, the conceptions of HCI as a craft discipline, as an applied science discipline, and as an engineering discipline suggest that the intial creation of discipline knowledge, whether heuristics, guidelines or principles, in all cases requires a reflexive cognitive act involving intuition and reason. Thus, contrary to common assumption, the craft, applied science, and engineering conceptions of the discipline of HCI are similarly reflexive with regard to the general design problem. The intial generation of albeit different discipline knowledges requires in each case the reflexive cognitive act of reason and intuition.

The second novel issue raised by the development of the Conference theme and the conceptions of HCI as a discipline is the relationship between the different conceptions. For example, the different conceptions of HCI and their associated paradigm activities might be considered to be mutually exclusive and uninformative, one with respect to the other. Alternatively, one conception and its associated activities might be considered to be mutually supportive with respect to another. For example, engineering principles might be developed bottom-up on the basis of inductions from good craft practice. Alternatively, engineering principles might be developed top-down on the basis of deductions from scientific theory – both from psychology and from computer science. It would be possible to advance a rationale justifying either mutual exclusion of conceptions or mutual support. The case for mutual exclusion would be based on the fact that the form of their knowledge and practice differs, and so one conception would be unable directly to inform another. For example, craft practice will not develop a theory which can be directly assimilated to science; science will not develop design principles which can be directly recruited to engineering. Thus, the case for mutual exclusion is strong.

However, there is a case for mutual support of conceptions and it is presented here as a final conclusion. The case is based on the claim made earlier that the creation of discipline knowledge of each conception of HCI requires a reflexive cognitive act of reason and intuition. If the claim is accepted, the reflexive cognitive act of one conception might be usefully but indirectly informed by the discipline knowledge of another. For example, the design ideas, or heuristics, which formed part of the craft practice of Bornat and Thimbleby in the 1970s (Bornat & Thimbleby [1989]), undoubtedly contributed to Thimbleby’s more systematic formulation (Thimbleby [1984]) and the formal expression by Dix and Harrison (Dix & Harrison [1987]). Although the principles fail to address the effectiveness of work and so fail to qualify as HCI engineering principles, their development towards that end might be encouraged by mutual support from engineering conceptions of HCI. Likewise, scientific concepts such as compatibility (Long [1987]) may indirectly inform the development of principles relating users’ mental structures to the analytic structure of a domain of application (Long [1989]), and even provide an indirect rationalisation for the concepts themselves and their relations with other associated concepts. Mutual support of conceptions, as opposed to mutual exclusion, has two further advantages. First, it maximises the exploitation of what is known and practised in HCI. The current success of HCI is not such that it can afford to ignore potential contributions to its own advancement. Second, it encourages the notion of a community of HCI superordinate to that of any single discipline conception. The novelty and complexity of the enterprise of developing knowledge to support the solution of the general problem of designing humans and computers interacting to perform work effectively requires every encouragement for the establishment and maintenance of such a community. Thus, the mutual support of different conceptions of HCI as a discipline is recommended.

References

J R Anderson [1983], The Architecture of Cognition, Harvard

University, Cambridge MA.

P Barnard, J Grudin & A Maclean [1989], “Developing a Science Base for the Naming of Computer Commands”, in Cognitive Ergonomics and Human Computer Interaction, J B Long & A D Whitefield, eds., Cambridge University Press, Cambridge.

R Bornat & H Thimbleby [1989], “The Life and Times of Ded, Text Display editor”, in Cognitive Ergonomics and Human Computer Interaction, J B Long & A D Whitefield, eds., Cambridge University Press, Cambridge.

P Buckley [1989], “Expressing Research Findings to have a Practical Influence on Design”, in Cognitive Ergonomics and Human Computer Interaction, J B Long & A D Whitefield, eds., Cambridge University Press, Cambridge.

J M Carroll & R L Campbell [1988], “Artifacts as Psychological Theories: the Case of Human Computer Interaction”, IBM research report, RC 13454(60225) 1/26/88, T.J. Watson Research Division Center, Yorktown Heights, NY. 10598.

R N Charette [1986], Software Engineering Environments, Intertexts Publishers/McGraw Hill, New York.

F I M Craik & R S Lockhart [1972], “Levels of Processing: A Framework for Memory Research”, Journal of Verbal Learning and Verbal Behaviour, 11, 671-684.

A J Dix & M D Harrison [1987], “Formalising Models of Interaction in the Design of a Display Editor”, in Human-Computer Interaction – INTERACT’87, H J Bullinger & B Shackel, (ed.s), North-Holland, Amsterdam, 409-414.

J Dowell & J B Long [1988], “Human-Computer Interaction Engineering”, in Designing End-User Interfaces, N Heaton & M Sinclair, eds., Pergamon Infotech, Oxford.

J Dowell & J B Long, “Towards a Conception for an Engineering Discipline of Human Factors”, (manuscript submitted for publication).

P Gilligan & J B Long [1984], “Videotext Technology: an Overview with Special Reference to Transaction Processing as an Interactive Service”, Behaviour and Information Technology, 3, 41-47.

N Hammond & L Allinson [1988], “Development and Evaluation of a CAL System for Non-Formal Domains: the Hitchhiker`s Guide to Cognition”, Computer Education, 12, 215-220.

N Hammond [1987], “Principles from the Psychology of Skill Acquisition”, in Applying Cognitive Psychology to User-Interface Design, M Gardiner & B Christie, eds., John Wiley and Sons, Chichester.

I Kant [1781], The Critique of Pure Reason, Second Edition, translated by Max Muller, Macmillan, London.

D Kapur & M Srivas [1988], “Computability and Implementability: Issues in Abstract Data Types,” Science of Computer Programming, Vol. 10.

T K Landauer [1987a], “Relations Between Cognitive Psychology and Computer System Design”, in Interfacing Thought: Cognitive Aspects of Human-Computer Interaction, J M Carroll, (ed.), MIT Press, Cambridge MA. 23

T K Landauer [1987b], “Psychology as Mother of Invention”, CHI SI. ACM-0-89791-213-6/84/0004/0333

J B Long [1989], “Cognitive Ergonomics and Human Computer Interaction: an Introduction”, in Cognitive Ergonomics and Human Computer Interaction, J B Long & A D Whitefield, eds., Cambridge University Press, Cambridge.

J Long [1987], “Cognitive Ergonomics and Human Computer Interaction”, in Psychology at Work, P Warr, eds., Penguin, England.

J M Mandler [1979], “Categorical and Schematic Organisation in Memory”, in Memory Organisation and Structure, C R Puff, ed., Academic Press, New York.

H H Pattee [1973], Hierarchy Theory: the Challenge of Complex Systems, Braziller, New York.

K R Popper [1959], The Logic of Scientific Discovery, Hutchinson, London.

D Scott [1976], “Logic and Programming”, Communications of ACM, 20, 634-641.

H Thimbleby [1984], “Generative User Engineering Principles for User Interface Design”, in Proceedings of the First IFIP Conference on Human-Computer Interaction, Human-Computer Interaction – INTERACT’84. Vol.2, B Shackel, ed., Elsevier Science, Amsterdam, 102-107.

E Tulving [1972], “Episodic and Semantic Memory”, in Organisation of Memory, E Tulving & N Donaldson, eds., Academic Press, New York.

P Walsh, K Y Lim, J B Long & M K Carver [1988], “Integrating Human Factors with System Development”, in Designing End-User Interfaces, N Heaton & M Sinclair, eds., Pergamon Infotech, Oxford.

Acknowledgement. This paper has greatly benefited from discussion with others and from their criticisms. In particular, we would like to thank: Andy Whitefield and Andrew Life, colleagues at the Ergonomics Unit, University College London; Charles Brennan of Cambridge University, and Michael Harrison of York University; and also those who attended a seminar presentation of many of these ideas at the MRC Applied Psychology Unit, Cambridge. The views expressed in the paper, however, are those of the authors.