Steve Cummaford and John Long (2010)

Introductory Comment

 

John Long Comment 1

 

 

Engineering Design Principles: Validating Successful HCI Design Knowledge to Support its Re-use

 

 

Abstract

John Long’s contribution to the discipline of Cognitive Ergonomics spanned over 30 years. A long-term goal of Long’s research was to specify validated HCI design knowledge, in a form which supports guarantees of its successful application to any design problem within its scope. This HCI design knowledge was characterized as HCI Engineering Design Principles (EDPs). The research problem addressed during the research was: how can EDPs be developed, such that HCI design knowledge can be specified and validated?

In order to operationalise the development of EDPs it was necessary first to extend the 1989 Conception of HCI as an Engineering Discipline (Dowell & Long, 1989) with additional concepts to form a Conception of EDPs, and then to define a method for EDP development. This paper describes the Conception of EDPs, and the method for their development. The method is illustrated with a case study of its initial application, in the domain of e-commerce. The extent to which the research made progress towards the development of EDPs is reviewed, and followed by reflections on how future research towards EDPs might proceed.

Keywords: HCI, Principles, Cognitive Engineering

1 Research Need

Validated Human-Computer Interaction (HCI) design knowledge offers the benefit of reducing system development costs for the solution of design problems, similar to previously solved problems. However, the validation of HCI design knowledge has been limited, if compared with related disciplines, e.g. Software Engineering [1].  It has been argued that this deficit is due to the lack of (explicit) conceptualisation of design knowledge, to solve design problems. This under-specification limits testing, and so validation [3].

 

This paper describes the initial operationalisation of a research strategy; intended more adequately to specify HCI design knowledge, to support validation. Dowell & Long [1] state that “established engineering disciplines possess formal knowledge: a corpus of operationalised, tested and generalised principles. Those principles are prescriptive, enabling the complete specification of  [classes of] design solutions before those designs are implemented.” To support validation, design knowledge must be expressed explicitly (conceptualised), to enable its application to design (operationalised), so its relationship with interactive system performance can be established (tested). On the basis of this relationship, the class of design problems for which this performance is achieved can be specified (generalised) [1].

 

The Dowell & Long conception expresses the general design problem of HCI in terms of a domain model; a user model and a computer model, which together comprise the interactive worksystem (IWS) model; and a statement of inequality between actual and desired performance (Pa ≠ Pd). The domain model comprises objects, which have attributes having particular values. The work to be performed is expressed as domain object attribute value transformations, collectively termed the product goal. The user and computer are characterised by structures and behaviours. These structures are physical, and abstract, and they support behaviours, which are physical, and abstract, and effect transformations in the domain. Performance is expressed as task quality (Tq), (i.e. how well the work is effected) and IWS costs (i.e. the workload incurred in effecting the work). IWS costs comprise user costs (Uc) and computer costs (Cc), and can be physical and mental.

 

EDPs comprise conceptualised, operationalised, tested and generalised design knowledge, which supports the diagnosis and prescription of a class of design solutions (CDS), which satisfies a class of design problems (CDP). EDPs are defined functionally, by reference to the class of design problems to which they may be applied. Thus, the specification of design problems and design solutions, at the class level, is a pre-requisite, and therefore, necessary process of EDP development. To support the ultimate specification of EDPs, there is a need iteratively to identify CDPs and their CDSs and extract both commonalities between them, and the non-commonalities, which are exhibited only by the CDS. The latter would constitute an initial EDP, which applies to all design problems within its scope.

 

The conception of Engineering Design Principles (EDPs), proposed by Cummaford & Long [4], specifies the knowledge representations, sufficient to support development and validation of EDPs. This conception is shown in Figure 1.

 

Figure 1: EDP conception

 

Specifying criteria for identifying design problems, to which an EDP may be applied, ensures that the knowledge is applied only to those design problems for which it supports the specification of a design solution. Design problems contain not less than one or more users, interacting with not less than one or more computers, and some desired performance. The scope of the EDP thus comprises a class of users (U-class), a class of computers (C-class) and a class of achievable performances (P-class). If the user and computer in the design problem are members of U-class and C-class respectively, and the desired performance stated in the design problem is a member of P-class, then a design solution would be produced and the actual performance of the solution would equal the desired performance as stated in the design problem. The relationship between U-class, C-class and P-class is developed by empirical testing of the implemented design solutions, produced by EDP operationalisation. If the user, computer, or the desired performance, are outside the scope of the principle, then there is no guarantee that the design solution may be specified then implemented.

 

The EDP conception specifies relations between its representations, which can be validated. First, the user and computer models contain sufficient structures, to support behaviours, which carry out the task-goal structure (TGS), whilst incurring IWS costs. Second, the TGS achieves the product goal, for some Tq. The EDP conception thus relates statements of performance to the models, supporting operationalisation, and so, validation. EDPs comprise conceptualised, operationalised, tested and generalised design knowledge, which supports the diagnosis and prescription of a class of design solutions (CDS), which satisfies a class of design problems (CDP). Thus, the specification of design problems and design solutions, at the class level, is a pre-requisite, and so, necessary process of EDP development. To support the ultimate specification of EDPs, there is a need iteratively to identify CDPs and their CDSs, and extract both commonalities between them, and between the commonalities themselves. The latter constitute an EDP, which applies to all design problems within its scope.

 

In this paper, the process of CDP and CDS development is reported, to illustrate the application of an EDP development strategy. The result, comprising class representations of the user, computer, domain and TGS for the CDP and CDS, is exemplified (due to space limitations) with models from the first of two cycles of operationalisation. The process of EDP specification is then reported. The resulting initial EDP is exemplified again with reference to the first cycle of research.

 

2. Summary of EDP Conception

The conception proposed thus far comprises the following concepts and relationships.

 

For any design problem {user, computer, Pd} and any EDP {U-class, C-class, P-class, substantive component, methodological component}:

 

1. If user is a member of U-class and computer is a member of C-class, then user structures and behaviours and computer structures and behaviours, as stated in the substantive component, are present.
2. If user structures and behaviours and computer structures and behaviours as specified by the substantive component are present, then the task goal structure specified by the methodological component is achievable.
3. If the task goal structure specified in the methodological component is effected by a worksystem comprising the structures and behaviours specified in the substantive component, then the product goal will be achieved, task quality will be x, user costs will be y and computer costs will be z.
4. If task quality x, user costs y and computer costs z are achieved, then Pa = Pd.

5. Therefore, Pd is a member of P-class for a worksystem comprising instances of U-class and C-class.

 

This conception may be said to be coherent, as it is based on two relationships: the relationship between the task goal structure and Tq for some product goal, and the relationship between the worksystem structures and behaviours, sufficient to achieve this task goal structure, and Uc and Cc. These relationships may be said to be coherent, as performance is a function of the effectiveness with which some task goal structures are achieved by some worksystem structures and behaviours. The EDP conception may be considered complete, as the concepts of the conception of the engineering discipline of HCI, which inform its development, appear within it. The issue of fitness-for-purpose will be addressed via operationalisation of the EDP conception to inform the development of EDPs, which may then be tested and generalised.

 

3. Validation and Ascription of Guarantees to EDPs

Operationalisation of the EDP conception as EDPs supports empirical testing of the class-level design solutions prescribed. This testing establishes whether the EDP conception is fit-for-purpose, that is, it supports the specification then implementation of a design solution, which achieves the desired level of performance as stated in the design problem. The fourth stage of validation, generalisation, involves establishing the generality of the EDP conception. These four stages of validation support the ascription of a guarantee that a worksystem, which performs the task goal structure, and specified in the methodological component of the EDP, achieves a level of Tq within the P-class, as stated in the EDP. A second guarantee, that the substantive component supports the specification of a worksystem which exhibits the structures and behaviours sufficient to achieve the task goal structure, as specified in the methodological component, whilst incurring a level of costs within the P-class, as stated in the EDP, may then be ascribed. A third guarantee, that correct application of the EDP to a design problem within its scope supports the specification then implementation of a design solution, which achieves Pd, is then ascribed on the basis of the former guarantees and further empirical testing. EDPs thus support the specification then implementation of a design solution, which achieves the desired performance if the design problem is within the scope of the EDP.

 

4. ‘Instance first’ and ‘class first’ strategies

Stork & Long [5, 6] have applied the Dowell & Long conception of HCI, to establish a basis for developing EDPs. They have operationalised the general HCI design problem by metricating the concepts, of which it is comprised, to express a specific design problem (SDP) in the domain of domestic energy management. Metrication provides observable and measurable criteria against which to assess performance. The design solutions (SDSs) of such specific design problems and the abstraction of prescriptive knowledge would constitute an EDP – the goal of Stork & Long. However, the operationalisation of an SDP does not, as such, ensure that a CDP, of which the SDP is an instance, will be found. This strategy may be termed the ‘instance first’ strategy, as it seeks to develop EDPs by specifying design knowledge for SDPs, by means of SDSs, and then generalising across instances. This approach may be contrasted with an alternative ‘class first’ strategy for EDP development, as proposed in this research [7].

 

5. Class development

The first phase of the ‘class first’ strategy for CDP development involves identification of an SDP and a corresponding SDS. The second stage involves identifying further SDPs, which require a similar Pd, which supports specification of P-class. The user(s) and computer(s), which are to achieve P-class are then assessed for similarities. They may be considered similar, if the user(s) and computer(s), specified in each SDP, comprise sufficient structures supporting sufficient behaviours to achieve P-class. If this sufficiency holds, these user(s) and computer(s) form U-class and C-class of the CDP respectively. In practice, once P-class has been specified, developing CDPs involves identifying U-class and testing instances (members) of this class interacting with instances of C-class. These instances of U-class and C-class are then used to inform the development of a CDS, which achieves P-class. No claims are made for the objective ‘reality’ of such classes, they are abstractions which are validated by their utility in supporting the definition of successful design solutions.

6. Identification of promising classes

A class may be considered promising for development if, for some SDP, a corresponding SDS exists and there are other SDPs, which share features of the solved SDP. Once such an initial class hypothesis has been formulated, the viability of the class may be assessed by examination of the work performed and the worksystem structures and behaviours sufficient to achieve Pd. If the performance achieved by the worksystem (Pa), specified in the SDS, is similar to the Pd of other SDPs (i.e. Pa = Pd), then the SDPs show promise for CDP development.

7. Costs matrix

The need for a systematic means of evaluating the SDP and SDS, and CDP and CDS systems in terms of their relative performance led to the development, in this research, of the ‘costs matrix’ [8, 9]. The costs matrix provides a systematic means of representing the relationship between the work, how well it is achieved by the worksystem, and the costs which are consequently incurred by the user. Specification of these relationships facilitates the comparison of competing solutions to a design problem in a systematic manner.

 

The costs matrix is constructed by listing the user abstract and physical behaviours on the y axis and tasks on the x axis. The cells of the matrix are instantiated by analysis of user data in the task goal structure. The cumulative abstract and physical costs are represented separately, as it is assumed that there is no equivalence in cost between an abstract and a physical behaviour. The costs matrix is also used to collate measurement of task quality, measured in terms of task completion rates, and time taken to complete each task.

 

The costs matrix supports comparison of competing design solutions to a design problem in terms of their relative performance. However, such comparisons are only valid if the elements on both axes are the same for each analysed design solution, and the means of evaluation is consistent between the systems analysed. The operationalisation of the costs matrix to measure the comparative performance of CDP and CDS systems is exemplified in the case study used to illustrate the stages of the method that follows.

8. CDP / CDS specification method

The following method for CDP and CDS specification was developed during this research [7] and operationalised during two cycles of research. The method stages are shown in Figure 2. CDP and CDS specification involves first specifying SDPs (1), by testing existing systems and identifying instances for which Pa ≠ Pd. The CDP is then specified, by abstracting commonalities from them (2). The resulting model is evaluated, by checking that the class user and computer models can be operationalised to complete the TGS for each SDP (3). Existing HCI design methods are applied to the CDP, to specify a CDS (4). To evaluate Pa of the CDS, it is decomposed into SDSs to enable testing by the target user group (5). Pa is abstracted from the SDSs’ performances (6). A complete CDP and CDS were specified for Cycle 1 and for Cycle 2 of this research, using this method. Each stage in the method is illustrated with a summary from Cycle 1 of the research.

 

                                                                         Figure 2: CDP & CDS specification method

 

8.1: Specify SDPs

8.1.1 Requirement

First, a promising class of systems is identified on the basis of (informal) similarities in work performed. Second, examples of such systems are selected for testing. If these systems achieve a desired level of performance, they constitute specific design solutions (SDSs). If Pa ≠ Pd, they constitute specific design problems (SDPs). The SDP representations specify the domain objects, sufficient to characterise the work performed by the IWS, to achieve the product goal by operationalisation of the TGS. The IWS, comprising user and computer models, specifies the structures sufficient to support behaviours, to achieve the TGS. Performance, expressed as Tq and Uc and Cc, is established empirically.

8.1.2 Instantiation

Two e-shops were selected on the basis that they exhibited similarities in the type of work supported, namely the sale of homogeneous physical goods. Both e-shops informally appeared to have actual performances that did not equal desired performance, and so were considered promising for the specification of SDPs. The two e-shops selected were both live commercial operations at the time of the research.

 

Stash Tea (www.stashtea.com) is a US-based retailer offering a range of tea and tea-related products. Herbs of Grace (www.herbsofgrace.co.uk) is a UK-based retailer offering a range of herbal products, e.g. essential oils, natural herbs and herbal tablets. The e-shops achieved similar product goals, but exhibited different behaviours.

 

In order to ensure that the systems did not change during the course of the empirical testing, screenshots were taken and used to create prototypes (offline simulations) of each e-shop. The use of prototypes ensured that all participants interacted with the same e-shop design, and that the analytical data in the relevant Costs Matrix was also based on exactly the same system as the empirical data.

 

Six users, who had experience of online shopping, were recruited for testing. The testing required the users to attempt to complete a range of tasks. The latter ensured that relevant aspects of the transaction were evaluated, e.g. removing items from the order, as well as adding them. Actual Tq was lower than desired, as not all users completed the tasks. Uc were inferred from user behaviours[1] and verbal protocols, and enumerated in a Costs Matrix. The SDPs are not included here, due to space limitations, but differences between the SDPs and the CDP are reported later.

8.2: Specify CDP

8.2.1 Requirement

Following SDPs specification, commonalities are abstracted to construct the CDP. This abstraction comprises common aspects of the SDPs, to provide an initial CDP expression. The domain model is also abstracted, to express the product goal in terms of domain transformations. The TGS is then similarly abstracted. The IWS model is likewise abstracted from the SDPs. As class users are an abstraction, which cannot be tested directly, Pa for the CDP is derived from the SDPs tested.

 

8.2.2 Instantiation

Here, the CDP domain model was abstracted from the SDP domain models tested. The domain transformations to achieve the product goal were then specified. The TGS, to achieve the product goal was then specified, by abstraction from the SDP TGSs. The U-class and C-class, comprising the IWS, were then specified, to contain the minimal set of behaviours to enable the domain transformations to effect the TGS. Performance was then derived by calculating the mean performances of the SDPs tested.

8.2.2.1 Domain Model

The domain model was the same for the two tested SDPs, and so was synthesised as the CDP. However, the reviewed e-shops sold different goods (e.g. herbal remedies, tea, etc.). The goods were therefore specified generally as ‘homogeneous physical goods, of which multiple units of multiple items are typically ordered’. This specification constrains instances of the CDP to transaction systems, which support collation of an order, containing multiple items, and which incur shipping (i.e. delivery) price, as part of the total transaction price.

 

Figure 3: CDP Domain Model

 

8.2.2.2 Product Goal

The affordant domain attributes (shown in bold in the domain model) are changed by the worksystem, in order to achieve the Product Goal. The dispositional domain attributes must have the values specified in the Product Goal, in order for it to be possible for the work to be done. The class domain model does not contain specific values for the object attribute values. The requirements for the object attribute values are specified in the Product Goal. The Product Goal (termed ‘CDP1-PG’) specifies the required values for the dispositional domain object attributes, and a specification of the affordant domain attribute value transformations that comprise the work.

 

Table 1: CDP1-PG: Dispositional Object Attribute value requirement

Table 2: CDP1-PG: Affordant Object Attribute value transformations

 

The relationship between domain transformations, and IWS behaviours to effect these transformations, is represented in the TGS.

 

8.2.2.3 Worksystem Models

The class worksystem comprises a class user, termed ‘CDP1-U’ and shown in Figure 4, who interacts with a class computer, termed ‘CDP1-C’ and shown in Figure 5. CDP1-U and CDP1-C both comprise Representation Structures and Process Structures. Both CDP1-U and CDP1-C were abstracted from the commonalities between the respective specific user and computer models in SDP1a and SDP1b.

 

                                                                Figure 4: CDP1 User Model (CDP1-U)

 

CDP1-U comprises two types of abstract structure. Representation structures (shown in boxes in the model) have particular states, e.g. ‘items ordered’, which are transformed by process structures (shown in ovals in the model, e.g. ‘Encode’). The representation structure states for each stage of the work are detailed in the representation structure states matrix. Process structure activation of representation states was assumed to support user abstract behaviours. The user abstract behaviours incurred during the work, are specified in the TGS, described later.

 

 

Table 3: CDP1-U Representation Structure States Matrix

 

The process structures in CDP1-U support abstract behaviours, which are defined as ‘planning’, ‘controlling’, ‘perceiving’ and ‘executing’. The abstract structures in CDP1-U are embodied by its physical structures, which also support physical behaviours, i.e. ‘clicking’, ‘keying’, and ‘searching’.

 

CDP1-C, like CDP1-U, comprises both representation structures and process structures, which support abstract behaviours. The abstract structures are embodied by physical structures, e.g. memory, processors, which support physical behaviours.

 

                                                              Figure 5: CDP1 Computer Model (CDP1-C)

 

8.2.2.4 Task-Goal Structure

CDP1-U and CDP1-C interact to effect a number of task goals in order to achieve CDP-PG. The user and computer behaviours, which interacted to achieve the task goals, are specified in the class Task-Goal Structure (termed ‘CDP1-TGS’). An excerpt from CDP1-TGS is presented in Table 4. The total number of each behaviour required to complete the tasks in the TGS-class was derived by taking the mean value for each behaviour across SDP1a and SDP1b. These values are represented in the CDP1 Costs Matrix, specified in Table 5.

 

Table 4: CDP1-TGS: T3 ‘Order 2 units of item 3’

 

8.2.2.5 Performance

Performance is abstracted from testing the SDPs, instances of the CDP. Tq is expressed as the percentage of successful attempts to complete the product goal. IWS costs are expressed as the number of behaviours performed. Costs are measured, using the Costs Matrix. Cc are not reported here, due to space limitations.

8.2.2.5.1 Task quality

81% of tasks were successfully completed during SDP testing. Tq is less than desired, that is, 100%.

8.2.2.5.2 User costs

Uc are incurred, whilst learning to achieve the product goal (‘set-up costs’), or during execution of the TGS behaviours (‘runtime costs’). Acceptable setup costs are essentially zero, as target users are able to complete a transaction with no previous training (they are specified as having used the internet previously, and having experience of using credit-card-style payment technologies). Pd is for lower runtime Uc than the SDPs. An increase in Cc is considered acceptable, if this supports a reduction in Uc, or an increase in Tq (as required earlier).

 

Runtime costs were calculated using the Costs Matrix, one of which was constructed for each user interacting with each SDP. The matrix was completed by giving a score of 1 for every observed or inferred user behaviour. The mean values for users were then calculated (shown in Table 5). Costs support diagnosis of ineffectiveness in the SDPs, and enable comparison with potential design solutions. The tasks in Table 1 constitute the TGS to achieve the product goal. Similar tasks were used during evaluation of the CDS, to support direct comparison of performance. Tasks 1-3 involved ordering goods; Tasks 4 & 6 were ‘find total price’; Task 5 was ‘delete items’; and Task 7 was ‘complete transaction’.

 

 

Table 5: CDP1 Costs Matrix (CDP1-CM)

 

The CDP, here, comprises all the representations, specified in the EDP conception, and as such, is considered acceptable.

8.3: Evaluate CDP

8.3.1 Requirement

To ensure that the CDP is sufficient to characterise SDP behaviours, it is evaluated analytically, with respect to the TGS for each SDP. The IWS behaviours of the CDP are operationalised to achieve the TGS for each SDP. If the CDP IWS achieves the TGS, then the CDP is retained. If there are insufficient behaviours, then SDP / IWS differences must be re-synthesised. If they are too dissimilar to support synthesis, then a CDP cannot be abstracted.

8.3.2 Instantiation

The e-shops evaluated during the specification of SDP1a and SDP1b exhibited different behaviours at the device level of description, e.g. shipping prices required the user to find international shipping rates and calculate the likely shipping cost in SDP1a, whilst the user had to register to find shipping costs in SDP1b. These differences resulted in a similar product goal being achieved, but with different actual performances. However, aspects of the work resulting in high workload were similar. They were therefore included in the CDP. The CDP user model (CDP1-U) and domain model (CDP1-D) were operationalised analytically, to check that the TGS in each SDP could be achieved. CDP1-U contained sufficient behaviours to achieve the TGS in each of the SDPs tested. The CDP was therefore retained.

8.4: Specify CDS

8.4.1 Requirement

Existing HCI design knowledge and methods are used to specify the CDS. The design stage specifies a TGS, which is achievable by the IWS, whilst incurring acceptable IWS costs and attaining desired Tq. Whilst performance can only be established by testing SDSs instantiated from the CDS, analytic methods can be used to establish the likely performance of the CDS prior to testing.

8.4.2 Instantiation

Best practice craft knowledge and guidelines were recruited to develop CDS1 [10-15]. In addition, behaviours resulting in low task quality or high user costs were identified and re-engineered to increase potentially achievable performance. Substantive HCI design knowledge was also recruited in the design [17-29]. In particular, the TGS, required to effect the product goal, was informed by mercantile models [30, 31], which specify transactions as having distinct phases, described as Negotiation, Agreement and Exchange. The outcome of the redesign was a re-engineered TGS. An example for the task ‘order two units of item 3’ is shown in Table 6. During CDP testing, calculation of total price, during order collation, was found to be a high-cost behaviour. The CDS TGS differed from the CDP TGS, in that shipping options may be selected at any point in the interaction, hence supporting calculation of shipping costs by the computer, during the Negotiation Phase of the transaction.

 

 

Table 6: CDS1-TGS T3 ‘Order 2 units of item 3’

The CDP TGS required credit card details to be entered by the user, before the computer displayed the total price, including shipping. The CDP testing indicated that this interaction reduced Tq, as some users were unwilling to enter this information, prior to knowing the total price. The CDS TGS presents the total price throughout the Negotiation Phase, enabling the user to complete a transaction, based on the total price, including shipping price (and so to achieve Tq). IWS costs, incurred whilst effecting the TGS, and the Tq achieved, were measured by testing SDSs, instantiated from the CDS. This testing is described in the next stage of the process.

 

8.5: Specify SDSs

8.5.1 Requirement

To evaluate the CDS, it is necessary to re-express it as specific design solutions (SDSs). As class users are an abstraction, and so cannot be tested directly, SDSs must be designed to enable CDS testing. It is necessary to instantiate more than one SDS to abstract CDS commonalities in performance.

8.5.2 Instantiation

Here, to evaluate the CDS, it was instantiated as two SDSs, to enable testing with specific users. The two SDS computer models were similar to the SDP computer models, but featured controls to allow the shipping options to be selected at any point during the Negotiation Phase. Two SDS prototypes ensured class performance.

 

8.6: Evaluate CDS

8.6.1 Actual Performance

The task descriptions contained in CDS1-TGS were abstracted from the commonalities between the TGSs in SDS1a and SDS1b. CDS1-TGS is therefore computer independent, as the computer-specific aspects of the SDS TGSs were not common between SDS1a and SDS1b. The class actual performance is derived from the mean number of each behaviour contained in the TGSs for SDS1a and SDS1b. These mean values are represented in the CDS1 Costs Matrix (termed ‘CDS1-CM’).

 

The class actual task quality was derived from the actual task quality of SDS1a and SDS1b. Both user costs and task quality are expressed in CDS1-CM, as shown in Table 7.

 

 

Table 7: CDS1 Costs Matrix (CDS1-CM)

8.6.2 Comparison of Pa and Pd

Task Quality

All instances which satisfy the pre-purchase requirements should result in the product goal being achieved. Actual performance is 100% task completion, therefore for CDS1 Pa = Pd, and can be considered a design solution.

IWS costs

User costs are acceptable and lower than the class actual costs, derived during specification of CDP1. A comparison of the cost differences between CDP1 and CDS1 is shown in the comparison Costs Matrix shown in Table 8.

 

Negative values indicate a reduction in costs between the CDP and CDS. Where costs increased between CDP and CDS, they are shown in red. Task completion rates show the increase in task completion percentages, rather than reductions in tasks failures, in order to maintain a consistency with the labelling of the Costs Matrices.

 

 

Table 8: Comparison of CDP1 / CDS1 Costs Matrices

 

The design solution is implementable using existing technologies, and so can be considered a valid design solution.
The user ratings of workload and comparative workload supported the conclusion that less workload was required to complete the tasks with the SDS systems. The SDS systems, and therefore by extension the CDS, were therefore considered to be acceptable, as they fulfilled the requirement in the Product Goals for the CDS and SDSs that the systems should result in lower user costs during task completion.

 

The CDS was therefore considered acceptable and retained.

 

9. EDP specification method

The method for specification of EDPs involves identification of commonalities between a CDP and its CDS, to form the scope of the EDP. Those aspects of the CDS which are not included in the CDP-CDS commonalities, and the negation of those aspects of the CDP which are not included in the CDP-CDS commonalities, are then used to define the prescriptive component of the EDP. The EDP achievable performance is then defined as the actual performance achieved by the CDS.

 

An EDP comprises three components: a scope (i.e. supports diagnosis), a specification (i.e. supports prescription), and a class of achievable performances (i.e. supports validation, leading to guarantee). The EDP defines class design knowledge, recruited from class design solutions. The method for EDP construction is as follows.

 

9.1: Define EDP scope

9.1.1 Requirement

The EDP scope defines the boundary of applicability of the EDP. The scope comprises a class of users and a class of computers, which interact to achieve a specified class of domain transformations within a specified class of domains. The EDP scope is defined by generification of the commonalities between the CDP and CDS. Defining the scope in this way ensures that sufficient components are in the CDP to enable the CDS components to be operationalised.

 

An EDP is prescriptive class-level design knowledge, with a specified scope of application. The scope of the EDP enables comparison with the scope of other related EDPs, to determine their relative generality.

9.1.2 Instantiation

The respective Product Goals, Domain Models, and User Models were identical for both the CDP and CDS. These models were therefore recruited extant to the EDP Scope.

 

The CDP1 and CDS1 user representation states had some commonalities. These are shown in Table 9. The remaining user representation states were not common, and were therefore not recruited to the EDP scope.

 

Table 9: EDP1-U representation structure states matrix

 

 

Not all CDP1-C and CDS1-C representation structures were common. In particular, the physical representation structures (i.e. screens and their components), and the abstract representation structures necessary to render these (i.e. page layouts and form processes) were not the same in CDP1-C and CDS1-C. These were therefore not recruited to the EDP scope. The commonalities between the computer models CDP1-C and CDS1-C were recruited to the EDP scope, and are represented in Figure 6.

 

                                                              Figure 6: EDP1 Computer Model (EDP1-C)

 

The CDP1 and CDS1 computer representation states had some commonalities. These are represented in Table 10. The remaining user representation states were not common, and were therefore not recruited to the EDP scope.

 

Table 10: EDP1 Scope: computer representation structure states matrix

 

The remaining components of CDP1 and CDS1 were not identical, and were therefore not recruited into the EDP1 scope. These non-identical components are presented in the next section.

 

9.2: Define Prescriptive design knowledge

The EDP prescriptive design knowledge is now synthesised from the non-common aspects of CDS1 and CDP1. The relevant components of CDS1 and CDP1 are first compared, in order to identify their non-commonalities. These non-commonalities are then used to construct the Prescriptive Component of EDP1. User and computer representation structure states are compared first, then user and computer behaviours. The computer physical structures, sufficient to support the user and computer behaviours identified, are also identified by reference to the user and computer behaviours.

 

9.2.1: Identify CDS-only and CDP-only components

The remaining components of the CDS comprise the aspects of the CDS which, if operationalised for the EDP scope components, will achieve the level of performance stated in the CDS. The CDS-only components are the crux of the EDP, offering a prescriptive specification of a TGS; user and computer representation structure states; supported user and computer behaviours (assumed to be commensurate with process structure activations); and achievable performance, as task quality and worksystem costs.

 

9.2.1.1 Identification of CDS-only and CDP-only abstract structures

9.2.1.1.2 User representation structure states

The CDS1 and CDP1 user representation structure states had some commonalities. These are represented in the EDP1 Scope. The remaining CDS1 and CDP1 user representation states were not common, and are therefore included here.

 

Table 11: CDS1 user representation states: non-commonalities

 

Table 12: CDP1 user representation states: non-commonalities

 

 

 

9.2.1.1.3 Computer representation structure states

The CDS1 and CDP1 computer representation states had some commonalities. These are represented in the EDP1 Scope. The remaining CDS1 and CDP1 computer representation states were not common, and are therefore included here.

 

Table 13: CDS1 computer representation states: non-commonalities

 

Table 14: CDP1 computer representation states: non-commonalities

 

 

9.2.1.1.4 Identification of CDS-only and CDP-only physical structures

The Computer physical structures, which supported the user behaviours specified in the TGS, are embodied in the screens, specified for CDS1 and CDP1. The CDS1 and CDP1 TGSs were each analysed by task. The computer physical structures, which supported the user behaviours in the TGS, were specified for each screen. The computer physical structures which were not also present in the CDP were then identified. These computer physical structures, which are present in the CDS, but not in the CDP, will be carried forward to the EDP prescriptive component later.

 

Space limitations preclude a full specification of the complete analysis here. The computer physical structures for Screen 9, referenced in Row 9 of CDS-TGS T3 (shown earlier in Table 6) are shown below, for illustration.

 

Figure 7: CDS1-C S9: 2x item 3 added

Required computer structures to support user behaviours (on the screen shown in Figure 7)

• Item quantity box
• Add to cart button
• Last item added

• Item subtotal

• Ship cost
• Total price

 

The method specified in stage 2A is now used to identify those aspects of the CDP, which are not represented in the CDP-CDS commonalities. Whilst the CDP-only components are not candidates for inclusion in the EDP, they may contribute to its specification by negation – i.e. if the CDP-only components are x, the EDP should specify not x. The CDP-only components are not exemplified here for reasons of space, but included computer structures such as a Recalculate button on the shopping cart page.

9.3: Synthesise EDP prescriptive component

The CDS-only and the negation of the CDP-only components are now synthesised to construct the EDP prescriptive component, comprising user and computer representation structure states, and supported user and computer behaviours.

 

9.3.1 User representation structure states

The user should be aware of the shipping cost and total price throughout their interaction with the e-shop. The user representation structure states will then be identical to those shown in Table 11.

 

 

9.3.2 Computer representation structure states

The computer should elicit sufficient information to enable the total price to be known throughout the transaction, i.e. sufficient information to calculate the shipping costs. The prescribed computer representation structure states are identical to those shown earlier in Table 13.

 

 

9.3.3 Computer physical structures

9.3.3.1 CDS-only components

The CDS screens are sufficient to support the behaviours in the CDS TGS. The CDS1 computer structures, which were not present in the CDP1 computer structures, are listed below, grouped by CDS1 screen. These non-commonalities are therefore necessary components of the EDP computer structures: screens.

 

Screen 2

• Last item added to cart
• Goods subtotal
• Shipping option selector

 

Screen 3

• Ship cost
• Total price

 

Screen 5

• Last item added
• Item subtotal
• Ship cost
• Total price

 

Screen 7

• Add to cart button

 

Screen 8

• Last item added
• Item subtotal
• Ship cost
• Total price

 

Screen 9

• Item quantity box
• Add to cart button
• Last item added
• Item subtotal
• Ship cost
• Total price

 

Screen 10

• Up and down arrows to change quantity

 

Screen 11

• Ship cost
• Total price

 

Screen 13

• Total price

 

Screen 15

• Items in order
• Total price

 

9.3.3.2 CDP-only components

The following structures were present in the CDP1 computer structures, but not in the CDS1 computer structures.

 

Screen 3

• Change quantity textbox
• Recalculate button

 

Screen 7.1a

• Shipping times and prices link

 

Screen 8.2a

• International shipping rates link

 

Screen 8.3a

• International shipping tariffs

 

The structures present in Screen 3 were not required in the CDS screens, as the item quantities in the shopping basket screen were updated through the use of the up and down arrow buttons.

 

The shipping times and prices information, and the links to this information, were not required in the CDS screens, as the shipping costs were calculated and displayed by the computer throughout the transaction.

9.3.4 Screens

The CDS1 screens are recruited to form the EDP1 screens. The CDS1-only non-commonalities identified earlier are the necessary components to support the EDP1-TGS, and the EDP1 screens are sufficient to achieve performance Pa = Pd.

9.3.5 Task-Goal Structure

CDS1-TGS is recruited to form the EDP-TGS. The TGS is not shown here, due to space limitations.

9.4: Define EDP achievable performance

The CDS actual performance, as task quality and worksystem costs, is recruited from the CDS to form the EDP class of achievable performances. The EDP1 achievable performance is thus shown in Table 7, the CDS Costs Matrix.

 

The EDP is thus defined. The method reported and exemplified here was operationalised for the CDPs and CDSs specified during Cycle 1 and Cycle 2 of the research to define initial EDPs.

 

10. Strategy assessment

The operationalisation of the ‘initial class’ strategy resulted in the specification of initial EDPs, and so offers promise for the eventual development of EDPs, supported by performance guarantees. This operationalisation indicates that the specification of class design problems and solutions is possible, and, as such, the ‘class first’ strategy was successful in this instance.

 

A number of specific issues were identified during the research, which should be addressed during future research. The specific issues are ordered, following the process stages of the method specified earlier.

 

10.1 Specify SDPs

10.1.1 Selection of systems for SDP development

The number of systems tested, to inform development of each CDP, was somewhat limited (i.e. two). Testing more systems would give greater confidence in abstracted similarities between instances. However, the work described here is intended to demonstrate the possibility of specifying CDPs and their CDSs, as an initial operationalisation of the ‘initial class’ strategy of EDP development. Once the process of developing CDPs and their CDSs has been shown to be feasible, such additional development efforts may then be justified.

 

10.1.2 Task selection

The tasks selected for inclusion in the TGS included a range of typical shopping activities. The tasks are sufficient to characterise any transaction performed on e-commerce sites, although the number of times each task is completed will vary for each transaction. It was, however, considered desirable to standardise the tasks here in order to support systematic comparison of the SDP systems, both with respect to each other and to the SDS prototypes. Varying sub-task frequency, however, should be considered in future work.

 

For the domain of e-commerce transactions, the selection of goods, receiving of the collated order (for physical goods transaction systems) and payment are necessary tasks to enable completion of a transaction. In practice, the product goals for each SDP/S were defined in order to ensure that completion of the tasks was necessary in order to achieve the relevant product goal. Future research should address the process of task selection based on the frequency and occurrence of tasks completed using any extant systems available. Other domains, e.g. Air Traffic Management, will have a commonly agreed corpus of tasks to be supported by the system, and development of EDPs for such domains would avoid these issues of completeness, with regard to task selection.

 

10.1.3 SDP prototypes

Whilst actual payment was simulated in the testing, users reported that they would be uncomfortable entering their own payment authorisation before knowing the total price payable. Tq was thus based on a simulation, with its inevitable differences with actual transactions, and so might be questioned. However, such ineffectiveness has also been shown to occur for e-shops generally [27].

 

Simulation of system functionality during the empirical testing was considered to have been successful, in that the results agreed with the users’ subjective ratings of workload incurred using the SDP and SDS systems. However, it would have been preferable to have used functioning e-shops, instead of simulating computer functionality, in order to measure time-to-complete for each task more systematically, and at a finer level of granularity. The decision to use prototypes with system behaviours simulated by the researcher was taken in order to allow the project to be completed within the practical constraints of the research project. In future research, functioning prototypes should be employed if possible, to enable more consistent and finer-grained analysis of task completion times.

 

10.1.4 Model selection, development and specification

The user, computer and domain models were based on previous work by Hill et al [32, 33]. The choice of user model and model format was based on the premise that Hill et al’s models were based on the Dowell & Long conception, and so contained the appropriate elements necessary to operationalise the conception during the research. The operationalisation of an alternative format for the worksystem and domain models would be possible within the existing methods specified within the research. It would be interesting to consider alternative choices of user, computer and domain models in future research.

 

The process of model development was similar for each SDP, and is now summarised, to inform future research. The domain transformations required to achieve the product goal were specified first, in order to inform development of the domain model. This ensured that the domain model contained only those objects whose attribute values were transformed during the work. The TGS was then specified in two stages. In the first stage, the user and computer behaviours sufficient to achieve the domain transformations specified in the product goal were identified. In the second stage, the user and computer structures, sufficient to support the behaviours identified in stage one, were specified. The user and computer models were then specified, by identification of the minimal set of user and computer structures, sufficient to support the behaviours identified in the TGS.

 

The first iteration of model development did not include user and computer structures in the TGS. However, the lack of user and computer structures in the TGS obviated the explicit relation of the TGS to the user and computer models, and so user and computer structures were then included in the TGS. The requirement for coherence between the models required repeated iteration of the models, as insights identified during development of the user and computer models were used to reverse engineer the TGS, domain model and product goal.

 

User behaviours were assumed to be equivalent in terms of workload (Uc). Whilst these behaviours undoubtedly involve user workload, the relative workload of each may well not be equal. Empirical evidence for differential workload between behaviours could be integrated into the Costs Matrix. Physical costs were enumerated by counting directly observable user behaviours, but abstract costs were inferred from tasks performed by the user, and from verbal protocols. A standardised set of encoding criteria informed identification of user abstract costs, which assumed the cost of encoding each page of information to be equivalent to the cost of mental (structure) activation. The criteria, and the equivalence of costs incurred during behaviours and activation of associated structures, could usefully be explored further in future work.

 

10.2 Specify CDP

Pd for the CDP was specified relative to the Pa of the SDSs tested (i.e. increase Tq, reduce Uc). Whilst it is possible, in principle, to state Pd as absolute values, such absolute values were not employed here. Future EDP development should support address of this issue.

 

User abstract and physical behaviour counts were not included in the CDP TGS tables, even when the CDP TGS contained a table taken directly from the SDP TGS and otherwise unaltered. The decision to remove the behaviour counts was taken because the CDP Costs Matrix contained the mean count for each user behaviour, for all SDPs included in the CDP.

 

10.3 Evaluate CDP

Analytic evaluation of the CDP involved checking that sufficient structures and behaviours were present in the user and computer models, to perform the TGS of each SDP. This assessment supported evaluation of the CDP user and computer models. This process was considered necessary, in order the better to inform the development of the subsequent EDP.

 

10.4 Specify CDS

User abstract and physical behaviour counts were not included in the CDS TGS, even though they were generally consistent between SDSs. Whilst the counts were similar for the CDSs developed here, it was anticipated that this might not necessarily be the case for all CDS systems, and so the counts were calculated using the mean values across SDSs, to enable the CDS TGS to operate at greater levels of generality in future design instances.

 

Specification of the CDS TGS involved re-engineering of the CDP TGS to reduce Uc, whilst maintaining Tq. The empirical testing carried out at this stage enabled the results of the re-engineering process to be evaluated, in terms of Tq. This approach offers some promise for the improvement of current HCI design practices, and will be investigated further in future by the researcher.

 

The specification of discrete user mental structures was based on the user mental behaviours identified during analysis of each task. The process by which behaviours are identified as being instances of separate behaviour types (e.g. calculate, read, etc) is therefore an important factor impacting the final enumeration of user costs, and the contents of the user model. The identification of behaviour types, and the subsequent identification of user mental structures should be investigated further in future research.

10.5 Specify SDSs

The SDS prototypes exhibited features, which were ported from the SDP systems tested. Such inclusions ensured that differences in performance between the SDPs and SDSs were due to differences in the CDP and CDS, rather than to specific features of the e-shops and SDS prototypes.

 

The features ported from the SDP to the SDS are therefore not present in the initial EDP. The initial EDPs could not, therefore, support specification of a novel SDS for which no SDP system exists. This issue of completeness must be investigated further during future applications of the initial EDPs, in order to identify those features which should be included in the EDP, such that the EDP will support specification of a SDS for which no SDP system exists.

 

10.6 Evaluate CDS

Structures were specified for both the user and computer models. However, workload was calculated without reference to structures, suggesting that structures are not a necessary component of the models for the purpose of measuring workload. However, structures offer a more complete characterisation of the worksystem, and would be necessary to characterise costs that are incurred when CDP and CDS structures change (that is, when structure ‘set-up’ costs are not zero).

 

10.6.1 Mapping the CDS to CDP

Whilst the CDP and CDS cannot be tested directly, the differences in performances of their instantiations indicate that the CDS achieved more effective performance than the CDP. Subjective reports of workload (Likert scales of task difficulty completed by users after testing) indicated that the SDS prototypes incurred less costs in general (as perceived by users) than the SDP systems. These subjective ratings of relative workload agreed with the relative costs calculated using the costs matrices, so offering informal support for the validity of the analytically derived calculations of relative workload.

 

10.7 EDP definition method

Two initial EDPs were specified using the method specified earlier, and so the method appears to offer promise for the development of further initial EDPs in future. The method identified the computer structures necessary to support the user and computer behaviours, specified in the CDS TGS, and so offers promise for the validation of this relationship in future research.

 

10.8 Initial EDPs

The initial EDPs offer promise for the specification of a ‘parent’ EDP via abstraction. However, this was not achieved during the timescale of this research. The informal similarities, which indicate what might be specified in an initial EDP at the parent-class level, are twofold:

 

First, the price of items should be clearly displayed throughout the transaction. Second, the total price payable should be made available to the user as early as possible in the transaction, and any user costs incurred in providing the minimal data required to specify the total price are to be considered acceptable (e.g. the user costs involved in entering the shipping option in CDS1 would be acceptable, as the shipping option data enabled the total price to be known earlier).

 

The specification of an initial EDP at the parent-class level would enable its application to solve design problems, which are instances of the parent class, but not of either of the two subclasses identified in this research. Such application would support better specification of the scope of each of the classes. The identification of design knowledge at an appropriate level of generality is a key aspect for the development of performance guarantees and, as such, is an important aim for future research in this area.

 

The next phase of research should address application of the EDPs, to novel SDPs within the scope of each EDP. This is likely to inform the expression of the EDPs, and will offer further insights into the appropriate level of class generality.

 

10.9 Review of requirement for validation, leading to guarantee

The research has specified HCI design knowledge at the class level, so offering promise for validation of the design knowledge, leading to guarantee, due to the completeness of the design solution specification. The design solutions were characterised in terms of their domain transformations, enabling relations between the work, the worksystem and performance to be specified explicitly. The use of Dowell & Long’s conception of HCI was therefore considered to be an appropriate decision, as the explicit and separate specification of the domain model offers promise for validation, leading to performance guarantees, in future research.

 

The method for ascribing guarantees of application, described earlier, involves empirical testing in order to develop and ascribe guarantees. It may be considered feasible to ascribe guarantees when testing carried out by multiple researchers, on multiple SDSs developed via application of the EDPs, yields consistent results. However, given the nature of empirical validation, the validity of the EDP can only be shown to be resistant to falsification, and cannot lead, a priori, to a definitive statement of guarantee.

 

10.10 EDP applicability, and their potential as design support

EDPs are a long-term goal of this research, and the form of their expression remains an issue to be addressed. However, the research has delivered interim products, in the form of initial EDPs. The EDPs comprised both necessary features of CDSs (i.e. the CDS-only computer structures identified) and a complete set of computer structures and behaviours, sufficient to specify a complete SDS. As such, the initial EDPs may be applied to novel SDPs which are within the scope of the respective EDP. Whilst the initial EDPs therefore exhibit utility and fitness for purpose, their expression needs to be developed further in future research.

 

The engineering of user behaviours at the class level was an effective design technique, and is lacking in current best practice design techniques. However, the current method of costs diagnosis requires further research in order to be feasible in practice.

 

11. Future research discussion

The EDPs were defined, but need to be applied and tested, to enable validation. Application of the initial EDPs will allow the format of their expression to be evaluated, and it is likely that this process will inform further development of their expression.

 

Application of the user model to diagnose user costs was work-intensive, and is currently not sufficiently well-specified to support multiple practitioners in achieving consistent results. The user model, and procedures for diagnosis user costs, may be considered fit-for-purpose when multiple analysts achieve consistent results, when analysing the same SDP. In order to achieve this goal of analyses delivering consistent results, the criteria for the diagnosis of mental behaviours should be further investigated. It may be that an alternative user model would also be required, to support a more consistent set of criteria for diagnosing mental costs.

 

The potential savings in development costs offered by the availability of EDPs as design support are significant. The development costs involved in EDP definition, whilst significant, are less than the potential benefits, assuming even modest take-up of the EDPs by commercial practitioners.

 

The current research has focused on developing the methods necessary for EDP specification, and operationalisation of these methods to specify initial EDPs. It is interesting to note that, once validated EDPs have been developed, the specification of EDPs and their application will not be carried out by the same analysts. The EDPs must therefore contain sufficient components to support specification of SDSs. In the research reported here, elements of the SDS may have been informed tacitly by design knowledge held by the researcher, which was not specified explicitly in the initial EDPs.

 

The research reported here involved identification of a parent class of e-commerce transaction systems, which contains two subclasses. Future research should investigate the feasibility of further subclasses of this parent class, in order to better understand the specification of design knowledge at varying levels of generality. In addition, the boundary conditions of the two subclasses should be investigated. Whilst the subclass of physical goods transaction systems, identified in Cycle 1 of this research, may well include a great number of SDPs, the class criteria should be reviewed in light of SDPs which are similar in some respects, but may be solvable using an alternative CDS. For example, SDPs for online (grocery) supermarkets are within the scope of the class of physical goods transaction systems, and as such, SDSs could be specified for these systems using the initial EDP, but an alternative EDP, supporting the design of SDSs with superior performance, could be specified.

 

The EDP conception and definition methods should also be operationalised for other classes of design problem, in order to investigate the generality of the methods across multiple classes of design problem. Operationalisation of the EDP definition methods would also allow the format of EDP expression to be refined. Once such work has been undertaken, it will be possible to gain further insights into the definition of EDPs at varying levels of generality, in a number of distinct domains.

References

1: Dowell, J. & Long, J. B. (1989) Towards a conception for an engineering discipline of human factors. Ergonomics 32, pp. 1513-1536

2: Sutcliffe, A. (2000) On the effective use and reuse of HCI knowledge. ACM Trans. Comput.-Hum. Interact. 7, 2 (Jun. 2000), pp. 197-221.

3: Long, J. B. (1996) Specifying relations between research and the design of human-computer interactions, International Journal of Human-Computer Studies, v.44 n.6, p.875-920, June 1996

4: Cummaford, S. & Long, J. B. (1998) Towards a conception of HCI engineering design principles, in T. R. G. Green, L. Bannon, C. P. Warren & J. Buckley (Eds.) Proceedings of ECCE-9, the Ninth European Conference on Cognitive Ergonomics. EACE, pp. 79-84.

5: Stork, A. & Long, J. B. (1994) A specific planning and control design problem in the home: rationale and a case study. In Proceedings of the International Working Conference on Home-Oriented Informatics, Telematics and Automation. University of Copenhagen, Denmark, pp. 419-428.

6: Stork, A. (1999) Towards Engineering Principles for human-Computer Interaction (Domestic Energy Planning and Control). Unpublished Doctoral Thesis, University College London.

7: Cummaford, S. & Long, J. B. (2002) Solving class design problems: towards developing engineering design principles, in S. Bagnara, S. Pozzi, A. Rizzo & P. Wright (Eds.) Proceedings for ECCE-11 the Eleventh European Conference on Cognitive Ergonomics. EACE, pp.167-174.

8: Cummaford, S. & Long, J. B. (1999) Costs Matrix: systematic comparison of competing design solutions, in S. Brewster, A. Cawsey & G. Cockton (Eds.), Proceedings of Human-Computer Interaction INTERACT ’99 Volume 2, IOS Press, pp.25-26.

9: Cummaford, S. (2000) Validating effective design knowledge for re-use: HCI engineering design principles. In CHI ’00 Extended Abstracts on Human Factors in Computing Systems. ACM Press, New York, NY, pp. 71-72.

10: Brighton Usability Group (2003) The Brighton Usability Patterns Collection. Retrieved on 12 March, 2007 from http://www.cmis.brighton.ac.uk/research/patterns/home.html

11: Brown, C. M. (1999) Human-computer interaction design guidelines. Intellect, Exeter, UK.

12: Fincher, S. (2000) The pattern gallery. Retrieved on 12 March, 2007 from http://www.cs.ukc.ac.uk/people/staff/saf/patterns/gallery.html

13: Lim, K. Y & Long, J. B. (1994) The Muse Method for Usability Engineering. Cambridge University Press, Cambridge, UK

14: Smith, S. L., & Mosier, J. N. (1986) Guidelines for designing user interface software. Mitre Corporation Report MTR9240, Mitre Corporation. Retrieved on 12 March, 2007 from http://hcibib.org/sam

15: Thimbleby, H. (1990) User interface design. New York: ACM Press.

16: van Duyne, D. K., Landay, J. A., & Hong, J. I. (2003) The design of sites. Addison-Wesley, Boston, MA.

17: Bernard, M & Sheshadri, A. (2004) Preliminary examination of global expectations of users’ mental models for e-commerce web layouts. In Usability News 6.2, 2004. Retrieved on 12 March, 2007 from http://psychology.wichita.edu/surl/usabilitynews/62/web_object_international.htm

18: Bernard, M. (2002) Examining user expectations for the location of common e-commerce web objects. In Usability News 4.1, 2002. Retrieved on 12 March, 2007 from http://psychology.wichita.edu/surl/usabilitynews/41/web_object-ecom.htm

19: Bidigare, S. (2000) Information architecture of the shopping cart. Argus Centre for Information Architecture white paper. Retrieved on 12 March, 2007 from http://argus-acia.com/white_papers/shopping_cart_ia.html

20: Chaparro, B. (2001) Top ten mistakes of shopping cart design. In Internetworking 4.1, December 2001. Retrieved on 12 March, 2007 from http://internettg.org/newsletter/dec01/article_chaparro.html

21: Ketchpel, S., Garcia-Molina, H., & Paepcke, A. (1997) Shopping models: A flexible architecture for information commerce. In Proceedings of DL97, Philadelphia PA, USA.

22: Kienan, T. (2001) Boosting transaction usability. Retrieved on 12 March, 2007 from http://www.tauberkienan.com/ecommerce/transactionsystems.html

23: Poong, Y., Zaman, K., & Talha, M. (2006) E-commerce today and tomorrow: a truly generalized and active framework for the definition of electronic commerce. In Proceedings of the 8th international Conference on Electronic Commerce: the New E-Commerce: innovations For Conquering Current Barriers, Obstacles and Limitations To Conducting Successful Business on the internet ICEC ’06, vol. 156. ACM Press, New York, NY, pp. 553-557.

24: Silverman, B. (2001) Implications of buyer decision theory for design of e-commerce websites, International Journal of Human-Computer Studies, v.55 p.815-844.

25: Snow Valley (2005a) IMRG e-Retail standardisation report. Part 1: Transactional navigational elements style & terminology. Retrieved on 12 March, 2007 from http://www.snowvalley.com

26: Snow Valley (2005b) IMRG e-Retail standardisation report. Part 2: The checkout process. Retrieved on 12 March, 2007 from http://www.snowvalley.com

27: Spool, J. (2002). The customer sieve. User Interface Engineering

Retrieved on 12 March, 2007 from

http://world.std.com/~uieweb/Articles/customer_sieve.htm

28: User Interface Engineering (2001) Are the product lists on your site reducing sales? E-commerce white paper Retrieved on 12 March, 2007 from http://www.uie.com/publications/whitepapers/pogosticking.pdf

29: Walsh, I. (2003) Good information architecture increases online sales. Sitepoint 23rd October 2003. Retrieved on 12 March, 2007 from http://www.sitepoint.com/article/increases-online-sales

30: Hallam-Baker, P. (1996) User interface requirements for sale of goods. World Wide Web Consortium technical paper (draft). Retrieved on 12 March, 2007 from http://www.w3.org/ECommerce/interface.html

31: Kalakota, R. & Whinston, A. B. (1996) Frontiers of Electronic Commerce. Addison Wesley Longman Publishing Co., Inc.

32: Hill, B., Long, J. B., Smith, W., & Whitefield, A. (1993) Planning for multiple task work: an analysis of a medical reception worksystem. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’93. ACM Press, New York, NY, pp. 314-320.

33: Hill, B., Long, J. B., Smith, M. W. & Whitefield, A. D. (1995) A model of medical reception – the planning and control of multiple task work. Applied Cognitive Psychology. Vol 9, 81-114.

 

---

 

1

A Long Festschrift: To Pass the Torch Which Way?

Of Cheese and Cheese and Chalk, and Sealing Wax, and Psychology, IT and Kings.

Dr. Dan Diaper

Introduction to Dan Diaper

DDD  SYSTEMS, 26 St. Marks Road,

Bournemouth, Dorset, BH11 8SZ, U.K.

ddiaper@ntlworld.com http://www.dddsystems.co.uk/

Tel. +44 (0)1202 523172 Mob. +44 (0)7840 539957

Introduction to the Paper

Abstract

Professor John Long has influenced the author’s research over more than thirty

years. Task Analysis for Knowledge Descriptions (TAKD), for example, directly

arose from their work, with Peter Johnson, in the early 1980s at the Ergonomics and

HCI Unit (EU) at University College London. Many aspects of the Conception of the

General Design Problem developed at the EU have driven the author’s research,

most recently in general systems analysis and culminating in the Systemic Analysis

Method (SAM); seven criteria are listed where the SAM is better than the, now

retired, TAKD method. Some aspects of the Conception are questioned, however,

notably the nature of design principles and, in some design contexts, how desired

performance can be set so as to be a driver of design, rather than having only an

evaluative function.

Comment 1

Notwithstanding such caveats, the influence of Professor John

Long on the research history reported has been both large and profound.

Keywords: Cognitive Engineering, Human-Computer Interaction, Design, General Systems

Analysis, Task Analysis, Conception of the General Design Problem.

Introduction

In an email, Professor John Long wrote to the author in early 2009, “… your assertion that I had

influences … on your work intrigues me. Obviously, we influenced one another, so I am not

claiming that I had no influence on your research/thinking. But what exactly and how did it come

about – both technically and personally? What’s interesting is that you are in a very unique

position to partition my ‘influences’ between pre- and post-conception. You were in the

Ergonomics Unit before the Conception beast got going. You also knew me in Cambridge and my

applied psychology beginnings with Donald Broadbent … This would be an unusual type of

contribution; but very much in the spirit of a Festschrift (see definition attached)”.

From John’s Wikipedia attachment, “In academia, a Festschrift is a book honoring a respected

academic and presented during his or her lifetime. The term, borrowed from German, could be

translated as celebration publication or celebratory (piece of) writing.”

Cambridge

Around the late 1970s, Cambridge was home to a couple of hundred psychologists, a galaxy of

stars, including people who had written papers that had impressed me as an undergraduate,

particularly in my own area of devotion, cognitive psychology. The shrine in this field, however,

was not the University’s Department of Experimental Psychology where I was studying for my

doctorate, but the Medical Research Council’s Applied Psychology Unit (APU) in Chaucer Road,

whose Director, before my time, Donald Broadbent, was John’s doctoral supervisor. The APU

was my intellectual home, actually a Victorian mansion by today’s standards (and a lot of

caravans outback), and I spent much time there as I lived in a house in Chaucer Road for years.

I remember Dr. John Long at the APU, in seminars, at numerous parties, and once, as I descended

the APU’s main staircase, John was at the bottom,

“Dan! How are you?”

“I’m confused.”

“Ahh, but you have that rich, deep confusion that we should all strive for.”

John had, unwittingly, given me one of the best compliments of my life, there have been few

better since. Unlike most of our colleagues, John and I understood, without need for discussion,

that a contrived confusion was a necessity for getting to grips with the truly complex, and

scientific psychology was, and is, truly, truly complex. That complexity was one source of our

passion for psychology, but then John and I are much alike in many ways, more as two cheeses

than in the adage, “as different as chalk and cheese”. John, and I, alone are likely to question that

saying, “Why chalk and cheese?”, “They’ve a lot of similarities, colour, high in calcium, …”, but

together we explosively explore whatever topic is to hand. Our joy, but I’ve noticed often the

glazed eyes of those within earshot.

John left Cambridge in 1979 to be the Director of the Ergonomics Unit (EU) at University

College London, which later, reflecting his research interests, became the Ergonomics and HCI

Unit. I continued at Cambridge until 1982 when I submitted my dissertation. I was comfortable

in Cambridge, earning a reasonable living supervising undergraduates for half a dozen colleges,

but John Long was one of the brightest people I’d ever met in Cambridge, so, comfortable or not,

when a Research Assistant (RA) post was advertised at the EU I applied; there were only a couple

of other people in the world who I wanted to work with.

The Ergonomics and HCI Unit

With his utter, open honesty, another cheese and cheese trait John and I share, John was explicitly

doubtful that I could make the necessary change from theoretical, scientific psychology to applied

psychology as practised at the EU. Even so, I was sufficiently convincing, only just, I think, that

John took me on.

My first EU project with John was on Peter Johnson’s IT syllabus design project, funded by the

U.K. Government’s Manpower Services Commission. Out of this project, within months of my

starting, came the first version of Task Analysis for Knowledge Descriptions (TAKD).

I was having dinner in a restaurant with John around 2005 when I was surprised to find that he

had forgotten my analysis, that I have also explained to others in a range of contexts, of how we

developed working together in the early 1980s. The rules of the intellectual game were simple,

we would continue to discuss some topic at a high level (i.e. in general, abstract terms) until one

of us challenged a high level proposition in the argument chain. At this point the proposer would

have to immediately unpack the proposition at progressively lower levels of detail until the

proposition was either accepted by the challenger, or both of us agreed to reject or modify the

challenged high level proposal, which we then proceeded to do, before returning to our high level

of discourse.

Comment 2

It was very efficient, but relied on a great deal of intellectual trust between us; we

spoke in a shorthand, and it was often confusing for outsiders. It’s not that this sort of shorthand

between co-researchers is uncommon in the specialist area in which they work, but John and I

applied it to everything, from the pros and cons of his Moto Guzzi motorcycle, to chalk and

cheese, and, if it had cropped up, to the value of sealing wax as a communication security system

(one of us would have asked whether the other meant verification or validation, I’m sure).

Happy days, but the seeds of our intellectual divergence were sown. After my Cambridge years I

was already more comfortable with computers and programming than John. After my broad self

education in IT in my first EU project, my next one at the EU, funded by the U.K. Government’s

Science & Engineering Research Council, was on AI, on Expert/Knowledge Based Systems, in

particular, on natural language processing intelligent user interfaces.

Peter Johnson made the initial move towards transferring TAKD from a syllabus design tool to

more general HCI and UID applications (Johnson, 1985; Johnson et al, 1985), but by the mid-

1980s HCI was a major concern for most of the maximum of a dozen or so researchers in the EU

(John’s view was that if there were more than a dozen, then he wouldn’t be able to be sufficiently

involved in all the EU’s research; many years later in a similar situation I agreed with him, but

typically, more extremely than John; he’s always been the more mature cheese, and this has

nothing to do with age).

John prefers to think of himself as a cognitive engineer, his concern primarily being with people,

work and performance, in systems, which isn’t to deny his considerable knowledge and expertise

in other fields, including IT. I, however, was in ready agreement with Russel Winder who I’d

met soon after we became committee members of the then newly formed British Computer

Society’s HCI Special Interest Group (now the British HCI Group). As a computer scientist,

Russel’s view was that HCI folk didn’t build computer systems, computing people like software

engineers did that. Even before I left the EU in 1986, I was already turning to software

engineering (Diaper, 1987), and although I never stopped being, and developing, as a

psychologist, I stopped teaching psychology in 1989 and moved to a fairly traditional Department

of Computer Science at Liverpool University (“traditional” in the sense that computer science is a

branch of, usually discrete, applied mathematics and Liverpool’s Department was part of its

Faculty of Mathematics).

Before I left the EU, John, with John Dowell, first his masters and then his doctoral student, were

developing their Conception (Long and Dowell, 1989), and which is still best expressed,

according to John, in their Ergonomics paper (Dowell and Long, 1989).

Comment 3

I was with John,

physically and intellectually, in the early stages of the Dowell and Long Conception’s

development. Chalk and cheese, however, John’s focus was more the human factors of HCI

whereas mine was becoming a broader HCI, subsequently (Diaper, 1989a), and oft repeated (e.g.

Diaper, 2002a), defined as “everything to do with people and computers” and which placed a

more equal focus on human and computer aspects.

Comment 4

I am indebted to John Dowell for pointing out

to me that my “everything” seems seriously at odds with the Dowell and Long Conception,

Comment 5

which wanted to define the engineering discipline in advance,

Comment 6

 

whereas my definition might imply some

ad hoc accretion and an acceptance that a unified, integrated discipline of HCI, cognitive

ergonomics, or whatever it is called, won’t be achieved.

Comment 7

 

Indeed, Green (1998) in a commentary

paper on Dowell and Long actually argues that, “The case for a single unified model has not been

made. Instead of seeking for a single unified model that encompasses all aspects of a worksystem,

one might propose a ‘network of limited theories’ (Green 1990), each responsible for its own

ontology and for successfully interfacing with its neighbours (See also Monk et al. 1993 for

similar views).”

Comment 8

 

Both John Long and I, and we were far from alone in the 1980s, believed that what was needed

was an engineering discipline that would contribute to IT systems development for the betterment

of people, the users, owners and those affected by such systems . John has been, and remains

today, steadfast in his commitment to this sort of solution, the development of such an

engineering discipline, although he currently prefers the term cognitive engineering to human

factors, in which knowledge about how systems can be reliably designed would prescribe both

design processes and what was designed.

The Dowell and Long (1989) paper’s abstract is explicit, that the paper first “examines the

potential for human factors to formulate engineering principles. A basic requisite for realising

that potential is a conception of the general design problem addressed by human factors.”

Twenty years later, John continues to reflect on why the Conception of the General Design

Problem, he does prefers it capitalised,

Comment 9

 

failed to be adopted by others, other than primarily by his

EU colleagues and students (see this volume).

Comment 10

 

The simple answer, one much too simple, alone,

for John or I, is that it was all too complicated. John suspects that his Conception required a very,

perhaps too, radical change in perspective. I’m not so sure, suspecting, amongst a number of

factors, that the delivery of the Conception was weak because it was incomplete and lacked

convincing examples

Comment 11

and, as John has pointed out to me, his timescale for developing design

principles was as a very long term objective; too long for the software industries, I think.

Comment 12

The TOM Project

In the late 1980s and early 1990s, the 2 Johns, Long and Dowell, Peter Johnson and I, with people

from Logica and British Aerospace, came together on the U.K. Government’s Engineering and

Physical Sciences Research Council funded Task Orientated Modelling (TOM) project. The

TOM project was to research, in what were the early stages of proposals, the further

computerisation of Air Traffic Control (ATC), particularly, replacing paper with electronic flight

strips. John, Peter and I had different solutions to how to achieve project and discipline progress,

and we had different agenda and interests. John was committed to his human factors engineering

discipline (Long and Dowell, 1989), Peter was already in a department of computer science and I

was moving to one. Peter would build prototype systems using formal methods, and I was into an

attempt to address the “delivery to industry” problem (Diaper, 1989b). We carved the project

into three strands based, it should be stressed, on the Dowell and Long Conception although, in

honesty, these strands were then developed independently within the TOM project, but with

regular exchanges of both ideas and, later in the project, data. The Johns developed the

application domain model (TOMDOM); Peter the device model (TOMDEV); and I the user model

(TOMUSR), which would be based on applying TAKD to ATC personnel and for which we, my

RA Mark Addison and I, developed a software toolkit (LUTAKD), in an attempt to make the far

too complicated TAKD method much easier for others to apply (Diaper and Addison, 1991;

1992). I publicly abandoned TAKD, after starting its development at the EU in 1982, in Diaper

(2001a).

The TOMDOM was a model of the ATC domain of application, about aircraft, airspace, airports,

radar, communications, and so forth. The basic architecture was that the TOMUSR and the

TOMDEV, which models users and devices (flight strips in this project), respectively, would

combine to form the TOMIWS, the Interactive Work System. Work, the safe and expeditious

passage of aircraft, is achieved by this work system making changes to the application domain

and dealing with feedback from the domain.

Comment 13

While the TOM project was successful in many ways, one significant failure was to provide John

with the sort of metrics that he wanted to measure work, and, especially, to provide measurements

of both desired performance (PD) and actual performance (PA). One aspect of design principles

that John has consistently stressed to me over the decades is that such principles must include

such performance metrics, i.e. they are a necessary condition of being Conception design

principles but, of course, this condition is only a small fraction of what else such principles must

or might contain. Being a necessary condition, however, I concentrate some effort below on what

I think are problems with PD and PA because, if these are problematic, then, being necessary, there

would appear to be difficulties with the Conception’s design principles and an analysis of this

might provide one example, technically inspired insight into why Dowell and Long’s Conception

was not more widely adopted.

Comment 14

During the TOM project we lost one of our commercial collaborators, who were going to use the

TOMDEV and TOMUSR, with the software tools Peter and I developed with our RAs, to create

TOMIWS models that would interact with the TOMDOM that our lost collaborators would also

create. This uncompleted strand to the TOM research programme might have produced the

metrics for PD and PA that John desired. I’m not, however, convinced that this would have

happened.

For the TOM project, because ATC is a safety critical system, then with paper flight strips both

PD and PA are high

Comment 15

 

and the existing system is highly redundant so as be virtually error free in the

application domain of real aircraft flying around safely, although PA with respect to

expeditiousness may be compromised, as demonstrated by Dowell (1998). What John Dowell

reports, long after the end of the TOM Project, is a generic ATC model that comprehensively

applied the Conception and which provides an extended demonstration of using it. The research

application involves a simplified ATC paper flight strip system simulation which, using ATC

professionals as subjects, does produce cost metrics for both PA and PD. While there is much to

commend it, particularly its “worksystem” model that I return to below, I continue to have some

concerns about both PD and PA. John Dowell recognises that there is a problem with specifying

PD and cleverly and plausibly argues that it can be set as for a single aircraft’s performance flying

through an otherwise empty airspace. ATC interventions are thus zero, as are PD costs, except, I

think, for a monitoring function, i.e. that the airspace does only contain a single aircraft. He then

is able to calculate values for PA by analysing task performance in the simulations; PA costs for

expeditiousness, but not safety, increase as the number of aircraft in the airspace increase.

In Dowell (1998), however, such costs are expressed as “unit-less” values, whereas the senior

decision makers of ATC’s future will ultimately want quantification that they can understand,

probably money, or, at least, the improved ratio of ATC personnel to aircraft safely and

expeditiously controlled.

Comment 16

 

I stress that it is only my later general systems research that leads me to

this sort of argument and I, and I think most others in HCI, have not seen our work in an

adequately large systems context. The 1998 paper does not deal with new designs, but only

presents data from the simulated current system, but clearly one could take a new design and

apply the theories and methods to evaluate it against the current system. The new one may be

demonstrably better, but the issue is whether it is significantly better isn’t answered, for example,

how many more aircraft could be controlled by ATC personnel and, of course, in the end, can the

financial costs of change be justified for the predicted benefits of change?

Comment 17

A related point was

made in Hockey and Westerman’s (1998) commentary paper on Dowell and Long, where, while

lauding the “explicit involvement of costs as part of the performance/usability metric” and that

these should provide “a more effective basis for the comparison of alternative systems”, they

warn that “it will not be possible to say how much increase in performance quality is equivalent

to how much more required effort or time, or how much more long-term effects of ‘fatigue, stress

and frustration’.”

Comment 18

My view on quantifying PA for an extant system, provided one uses relative metrics, e.g. unit-less

ones, is that it is not a problem for nearly all task analytic sorts of approaches, provided, also,

they are intelligently applied. N.B. “extant” may include simulations of new designs, provided

that simulation fidelity is adequate for what is being investigated (e.g. Diaper, 1990). There is a

potential problem, however, that different methods will sometimes produce estimates of PA that

are so different that different conclusions are reached. Out in the messy world of software

development I doubt this is a problem, but may be so for academic theoreticians concerned with

the properties of methods, metrics, philosophies and associated exotica, which certainly includes

the Dowell and Long Conception and, I’m afraid, much of my own theoretical research.

Comment 19

On the other hand, I think there are many problems with the quantification of PD. It may seem an

exotic point, but I cannot see how it is possible to set a PD that is independent of a radically new

design. For example, the PDs for an automated flight strip system and a fully automated AI ATC

system that had no human controllers, and all their associated capital and infrastructure costs,

must be radically different. While paper and electronic flight strip systems might have identical

PDs, this would only seem so provided that neither the application domain or the boundary of the

work system (see below) are changed, for example, if electronic flight strips are simply slotted

into an otherwise unchanged system. If a more radical design changes the application domain,

say, changing air corridor properties such as minimum vertical and horizontal separation, then

there must be, at least, minor changes to PD to reflect this. A people-less, AI ATC system might

change both work system and application domain far more, for example, there may no longer be

any handover processes as currently occurs between human controllers in different sectors, the AI

system staying with each of its aircraft from departure loading to arrivals unloading. In such a

case PDs related to in-flight handovers would simply be irrelevant, whereas they are crucial with

current systems. I can easily imagine future scenarios of distributed, interacting AIs where what

is now ATC is something so completely different that today’s ATC officers would not be able to

even recognise it. It might be unimaginative to suggest that whatever the technology, in the end,

aircraft have to be controlled and so future ATC systems would still be comprehensible with

today’s knowledge. I retort that what happens if instead of controlling aircraft, what is controlled

are the status designations of dynamically changing, perhaps quite small, volumes of airspace,

which gives each aircraft’s autonomous AI constantly and rapidly updated flight path options?

Such ATC by airspace control is beyond human capabilities in realistically loaded ATC systems,

but there are some tasks that computers are better at than people, one example being volume (pun

intended) micromanagement.

Comment 20

One concern I have is where measured PA exceeds PD.

Comment 21

For example, it was recognised decades

ago (e.g. Sell, 1985) that a diagnostic medical expert system could perform better than any

individual human doctor, because it was built from the expertise of a number of human experts.

Initially, PD can only be set by human expert performance in the absence of an implemented

expert system.

Comment 22

Once implemented and its performance evaluated, PA now exceeds PD, even on

the measures specified in the initial PD, but until it is implemented, we cannot know on which

parameters and by how much PA is greater than PD. Subsequently setting PD to PA is the wrong

way round, PD is simply playing catch up if the design is iterated with further incremental

improvements in the expert system’s performance. In such a situation, PD cannot do the job

intended of it within the Conception.

Comment 23

 

Furthermore, where PA is high, then when performance is

thoroughly investigated then this is likely to add additional parameters to PA, which in

consequence would add qualitatively to PD, as well as the quantitative changes described above.

In summary, when not just analysing a current system, then the problem I have with setting PD is

7

that it will sometimes be dependent on a particular design. Therefore, in such cases, PD cannot

drive design, but can only follow design, having an evaluative rather than a design generative

function.

Comment 24

In the Conception it is necessary that a design principle always includes desired and actual

performance costs, yet there do appear to be problems with specifying and quantifying both PA

and PD if they are to contribute the Conception’s desideratum that design principles can generate,

as opposed to evaluate, designs.

Comment 25

For me this is a core issue, whether one can create designs using

principles or whether imaginative, creative design processes remain as human psychological

functions which are, better than now I hope, supported by principles, and methods, tools, etc., that

help evaluate putative designs.

Comment 26

The Conception beyond TOM

John and I never again worked together on a major project after 1992 and, being very busy people,

failed to keep in regular contact. We’d bump into each other every few years at conferences and,

neither of us ever growing up, we’d pretty much start where we left off. John has chided me,

often when we’ve met, even unto sarcasm, for my failure to understand the Conception and, in

particular, what are its engineering design principles, which when applied would give a better

guarantee of performance than, for example, guidelines. I have admitted my repeated failure, on

the Conception and even more so on the principles, but I have made one more attempt, with

John’s help, for the purpose of writing this paper.

Comment 27

I re-read the Dowell and Long (1989) Ergonomics paper. I still didn’t understand it, in some

respects. My difficulty is understanding what is the key concept, of the General Design Problem.

Comment 28

I even wonder, still, whether it is about design. I want to use “design” as in a software

engineering lifecycle model sense, where I suspect that John’s sense is a more general one, of

everything to do with building something, or in his terms, ‘specifying then implementing’ and in

contrast to the activities of science or craft (Long, 1986).

Comment 29

John does admit that the Conception

has nothing to say about creativity, where designs come from,

Comment 30

yet for decades my design problem

has been Diaper’s Gulf (e.g. 1989c), between requirements specification and design.

Comment 31

Noting that

requirement specifications are not the same as the Conception’s design principles,

Comment 32

I do not see

how a requirements specification can generate a design, except by exclusion, or, sometimes

unfortunately, by copying previous designs.

”Comment

In a software engineering project a perfectly reasonable requirements specification might be that

the software must conform to Microsoft’s “Windows XP style”, so, no matter that it is a poor

piece of modern user interface design, we would expect the print option to be under the file menu,

where it is for historical reasons concerning the creation of a print file. This is design by copying,

but the requirement hasn’t generated the design, which was done decades ago,

Comment 34

and it certainly

doesn’t constrain the add-on solutions of having other ways to access print to overcome the old

design fault, that most user don’t understand why print is there, and many users probably don’t

realise it is there because they use other means to access print, such as via specialised, and

sometimes program specific, icons and buttons. Furthermore, I argue, that whether expressed

positively or negatively, or as declarative statements or as performance targets, non-copying-of old –

designs requirement specifications can only exclude some designs and cannot generate design.

My argument is easiest understood with the more modern performance requirements that are now

used in some international standards (ISO, DIN, etc.). Here, one might have a requirement that

users must receive a response within X seconds of their input. Clearly how this sort of

requirement is met by any particular design, quite deliberately, is not specified. So this sort of

requirement cannot generate design. With a requirement expressed as a declarative statement,

such as, ‘All pointing devices must highlight, select and show options’, then this excludes all

designs where this is not true, but it does not generate a design,

Comment 35

whether it’s left and right mouse

buttons and single or double clicking, or how a graphic tablet stylus works. Furthermore, if I

were interested in interface consistency, I can imagine (design) a mouse with a single button that

mimics a stylus, so, for example, holding down the button would provide menu options rather

than making a selection, or vice versa, a stylus that had a button to mimic the right button

selection functions of a Windows-based PC’s mouse. My point is that such designs were not

generated by the declaratively stated requirement, but only that they are constrained by it and

could be evaluated with respect to it.

Comment 36

John’s response to aspects of the above is in two parts. First, that in areas like civil engineering,

then it is claimed that there are structural engineering design principles that are generative as well

as being evaluative. Second, and I agreed with him, my sort of software engineering requirement

specifications weren’t the same as his design principles, whether general or specific ones.

On the first of these, I am suspicious of the generative claim, I don’t really trust civil engineers as

applied cognitive psychologists, and the distinction is subtle between how designs are actually

produced, i.e. generation, which I think is about the psychology of designers, and the constraints

under which designs are produced, either by copying previous designs or explicitly excluding

some design options. I have come to agree with much of what people such as Prof. Gilbert

Cockton says about design, and, up to a point, designers (e.g. Diaper and Lindgaard, 2008;

Cockton, 2008). Furthermore, I have argued that we are missing, or there are very few of them,

experts about methods (Diaper and Stanton, 2004). These experts would not be specific method

developers, there are lots, even far too many, of those, but experts about the properties of methods

and all sorts of stuff about their application. As a humble example, I have for a long time been

undertaking taxonomic research concerning models that are strictly hierarchical from those that

are heterarchical; the latter have levels of abstraction but, in the mathematical sense, are not true

hierarchies because either nodes are repeated or they have more than one parent node (e.g. Diaper

2001b). I have applied this work widely, to the strict hierarchical decomposition required when

using Data Flow Diagrams or in Checkland’s (1972; 1978; 1981; Patching 1990) Soft Systems

Methodology (Diaper, 2000), to categorisation tasks, from card sorting methods to applications in

AI, and for text and dialogue analysis and to much of my work on task analysis and, later, my

general systems analysis research.

John has, of course, been involved in method development himself, for example MUSE (Method

for Usability Engineering; Lim and Long, 1994), but this work never convinced me that John had

really come to grips with the general problems with method design. No matter how worthy, on

whatever criteria, MUSE might have been itself, including integration with then current software

engineering practices, its adoption was only modest.

Comment 37

 

I moved to software engineering because

my solution was to try to directly modify its practices, adding HCI, human factors, cognitive

engineering, etc. from within, as an alternative to working towards a separate human orientated

engineering discipline that would support software engineering and IT development. In any case,

if I don’t think John has successfully got to grips with such method development issues, and I’m

not claiming I have either, only that I’ve done some research on some bits of the problems, then

I’m not willing to trust the claims of architects and civil engineers about requirement

specifications or design principles being able to generate, as opposed to evaluate, designs. In an

area of engineering with which I am familiar, software engineering, I cannot find examples where

requirements specifications are design generative, i.e. Diaper’s Gulf remains a problem. I’m not

sure that software engineering has design principles in anything close to the Conception’s sense.

John’s second point, that my sort of software engineering requirement specifications weren’t the

same as his design principles, led him to suggest I look at the work of his more recent doctoral

students, who had “made a good fist” of investigating the acquisition of such principles. Dr.

Steve Cummaford kindly provide me with several hundred pages from his unpublished doctoral

dissertation (see: Cummaford and Long, 1998; 1999) in response to my request, via John, that I

needed a simple example of such a design principle that I could think about concretely.

Let me start by saying that if I had been Steve’s Ph.D. examiner, I would have had no problem

with passing him, easily, except that his acronyms are even worse than those John and I, cheese

and cheese again, have inflicted on our readers over the decades. Nor am I going to produce a

critique of his doctoral work, but I do have an extremely crude characterisation, a mere thumbnail

sketch, that I offer as a partial explanation of why I still don’t understand John’s design principles

in detail.

 

Figure 1 is from the dissertation’s Chapter 5: Method for the operationalisation of ‘class first’

(engineering design principle acquisition) strategy. When I looked at how this works, one

example is transaction processing from web pages, i.e. a customer buying products electronically,

then on the left side of Figure 1, one does some task analysis, one could choose from many

different methods which would be more than adequate if intelligently applied. The task analysis

is carried out on a number of tasks, and performance data, objective and subjective, is collected.

The results are abstracted, and the specific and higher level class analyses can iterate. It’s the

same on the right of Figure 1, but this time it is design solutions that are the subject of the task

analyses and abstractions. What I don’t understand is the arrow labelled 4. Steve recruits one set

of design heuristics, which are not the Conception’s design principles, but one could choose many

other sources, and while this is all fine in the messy world of real software design, it does not

address my theoretical concern of how an abstraction based on a current system can generate a

new design solution.

 

Comment 38

Once the system has been designed, it can be evaluated, even a paper

prototype can be evaluated, but the roles are evaluative and not design generative.

Steve actually goes as far in his first chapter’s first paragraph to state that his research “offers the

promise of reducing development costs incurred whilst solving problems for which solutions have

already been specified” (p31, my emphasis) and he admits in the final Chapter 13 (Strategy

assessment and discussion) that “the form of their (engineering design principles) expression

remains an issue to be addressed.” (p321), so my problems remain unaddressed as I still do not

have a simple, well expressed, exemplar design principle with which to work.

Comment 39

 

One issue with Steve’s task analyses is that they are basically a task dialogue model between user

and computer. This can be seen in the analyses where one of these agents acts, alternating from

one to the other. As can be seen in Table 22 from Steve’s Chapter 8 (p123), odd numbered task

lines concern the computer’s behaviour (CB) and the even numbered ones user behaviours (UB).

R U/C Task description Dom trans. UB(abstract) UB(physical) CB (abstract) CB (physical)

 

 

While such task dialogue models are extremely common, and I’ve frequently used them myself,

they present an especial problem in the context of the Conception, beyond the usual problems that

such models are sometimes poor at capturing parallel performance in agents. The problem is that

the Conception proposes that it is the interactive work system that performs work by transforming

the application domain and in Steve’s example the work system is the combined operation of user

and computer, whereas the analysis separates the two agents. Thus, the two agents’ performance

has to be subsequently combined to form an interactive work system model, but it would surely

be more appropriate to perform the task analysis at the interactive work system level in the first

place: it is conceptually neater; will usually provide different insights from a task dialogue level

of analysis; and will probably be more efficient.

Comment 40

In contrast, Dowell (1998) does present an interactive work system model at this level and not at

the level of its components,

Comment 41

 

although I might disagree with the claim made early in the paper

(p746) that “costs may be separately attributed to the agents of a worksystem”, as opposed, as I’d

prefer, to the work system itself.

Comment 42

 

Setting aside this minor cavil, John Dowell uses a blackboard

model of his worksystem that does not separately model its components, so neither controllers or

flight strips are visible in his worksystem level model. Blackboard models were commonly used,

particularly in Andy Whitefield’s work with John Long (e.g. Whitefield, 1989), when I was at the

EU in the early to mid 1980s, and although I’m not a particular fan of them myself, I think John

Dowell is absolutely spot-on to express his worksystem model without it containing things like

ATC officers and flight strips. My own, later approach is in one sense less sophisticated in that it

simply ignores what is within a particular work system boundary, treating the whole work system

as the agent that transforms the application domain.

Comment 43

John Dowell and I agree about the

importance of specifying a work system’s boundary, which is an analyst’s separation of it from its

application domain, although we have minor methodological differences, with John Dowell

suggesting that the boundary, at least in part, is determined by the application domain, whereas I tend to stress its definition by the “communicative relationships” that cross the boundary and

which, for me, are primary data for systems analysis at a particular level of work system.

A more major difference I recall discussing with the Johns on the TOM project was whether the

work system was a part of, a subset, of the application domain or, as in the Conception, separate

from it;

Comment 44

 

 

Benyon (1998) suggests a third alternative, “to see the worksystem as including the

domain” (p153, his emphasis). My view was, and still is, that everything in a model of a system

is the application domain, except that part of the system that is within the work system boundary.

Comment 45

This position has lead to my later systems approaches differing significantly from most of the

other John Long inspired work on this, as it allows simultaneous, multiple, overlapping work

system boundaries, whereas the other work tends to have only a single work system. The next

section elaborates on these issues.

Comment 46

General Systems Analysis with Work Systems and Application Domains

TAKD’s requiem (Diaper, 2001a) cleared my research table on task analysis so that I could

invent something better. My first criterion for a better approach was to address the delivery

problem, in particular, how to make it usable for IT professionals of many types who were not

human factors experts. Boring from within, I had experimented with the Pentanalysis Technique

(Diaper et al., 1998), which extended the widely used Data Flow Diagrams (DFDs) software

engineering modelling technique so that a task analysis would have an output that mapped

directly to DFDs. It works, but I didn’t pursue developing the Pentanalysis Technique, I

published just the one paper on it, because I had other criteria to fulfil.

My second criterion was that I wanted to improve model validity, validity being defined as the

relationship between the assumed real world and a model of it (Diaper, 2004).

Comment 47

Not only would

the approach meet my philosophy of “pragmatic solipsism” (I can’t prove the world exists, but

I’ll assume it does for scientific and engineering purposes), but it must cope with an incertae

sedis world in which things do not fit neatly into an analyst’s hierarchical model, i.e. my new

approach would, where necessary, support a model of levels of abstraction that was, when

appropriate, heterarchical (Diaper, 2001b).

Comment 48

My third criterion was that as a computer scientist, for a number of reasons, I wanted the

capability to produce a formal model, i.e. in mathematics/logic.

Comment 49

I also knew that many people

find mathematics difficult or, for some, impossible. This seems hardly a controversial claim and

is sufficiently accepted that when we set up the journal Interacting with Computers in the late

1980s, we had a policy, which is still extant today in the journal’s advice to authors, that

mathematics, logic and other such stuff, when included in a paper, must also be “clearly described

in plain English.” So I developed Simplified Set Theory (SST) for the “mathematically

challenged”, who with it could do: logic using a graphical method based on Venn diagrams as

opposed to equation manipulation; procedural equation manipulation, following a method based

on subset identity and substitution, rather than using the full set of Set Theory’s algebraic

operators; and the automatic conversion of SST equations into, approximately, English. A

method of using SST was my Simplified Set Theory for Systems Modelling (SST4SM) approach

(Diaper, 2004). Another use of SST was to provide a formal modelling system for Checkland’s

Soft System Methodology’s Conceptual Models (Diaper, 2000) and with this, also for DFDs.

My fourth criterion for a better approach was that it should be a general systems analysis

approach, rather than specifically a task analysis one; I was by this time Professor of Systems

Science and Engineering, I chose my own title, and would always point out when meeting people

for the first time that it was systems, in general, and not only computer systems, that I studied.

Studying complex systems, I eventually returned to the basics of systems theories and almost its

simplest example of a heater with a thermostatic switch, say a bimetallic strip. This has led to a

logical  AND gate model that is multimedia, the  AND gate has both electrical and thermal inputs

and, importantly, the critical entities that form  AND gates are the connections, electrical wires and

energy transfer functions, and not the objects such as heater, thermostat and so forth which are the

first focus of most HCI and software engineering approaches, including most of my own. This

theoretical research provides a basis for my more sophisticated work system and application

domain models, particularly with respect to studying “communicative relationships” across work

system boundaries.

My fifth criterion was that because psychology is so very, very difficult, even John and I struggle

with it after decades, then what was needed was a psychological approach that could be

understood, and, importantly, effectively and efficiently used by non-psychologists, say by

professional IT developers and their managers. My vehicle here was scenarios and starting with

Carroll’s (2000) proposals, I argued that scenarios were just imagined tasks, just task simulations

(Diaper, 2002b) and I subsequently developed the All Thought Is Scenario-based (ATIS)

hypothesis (Diaper, 2002c) which, in a nutshell, says that when we think, what we do is tell

ourselves stories (scenarios); this is how we perceive, recall/recreate memories, plan, etc. The

basic idea can be traced to the psychologist Jerome Bruner (1990) and has, for example, been

investigated in narrative approaches to task analysis (e.g. Rizzo et al., 2003).

My sixth criterion was to develop a means to predict the future, because design, whatever else it

involves, is inherently about predicting the future. This, of course, is impossible. In any case, I

don’t believe in the future, an old slogan of mine is, “The past is as mythical as the future.”, and I

only believe in the present world of now, which I designate as t0, as a pragmatic (in the William

James (1907) sense) assumption. What is possible, however, is that one can predict futures in the

plural (e.g. Diaper and Stanton, 2004; Diaper and Lindgaard, 2008). The supporting ideas come

from Chaos Theory where future states are restricted but that at any time t0, then it is impossible

to predict the state at t0+n. Chaos Theory allows micro-causality (strict determinism) but not

macro-causality. For example, a hinged pendulum will swing like an unhinged one most of the

time, but “can show a wide range of unpredictable, capricious movements” (Peterson, 1998, who

provides a non-mathematical introduction to Chaos Theory in chapter 7).

In my approach, which I’ve presented at seminars but not published since I retired as a full time

academic in 2006 (I’ve been meaning to write a book), the future is not a fog, but will contain

islands of stability, possible futures, which will be described at higher levels of abstraction the

more distant in the future is each island. The intention is to model these islands of the future

using scenarios, since that is how people, including designers, think according to my ATIS

hypothesis. The basic idea is that if a good general systems model at t0 can be developed, then

better future scenarios can be developed than if a relatively poor model of the world at t0 is used

by designers. I am far from understanding all the ramifications of this futures modelling

approach, but I hope to interest John in it one day, because it might provide a means to address

how his design solutions could be generated from his design problems, in contrast to my several

arguments above that requirements cannot generate designs.

Comment 50

My seventh criterion arose from the work I’ve been doing in Computer Supported Cooperative

Work (CSCW) since the late 1980s, and which has been my main application area, providing testbeds

for many aspects of my research. It is a myth that even the older task analysis methods,

such as HTA (Annett et al., 1967; 1971; Annett, 2003; 2004), and TAKD, cannot adequately

model collaborative work. Most task analysis techniques, however, are about how people

perform tasks, usually people doing so using tools. I think there are historical reasons for this,

that task analysis was developed by psychologists, and that until recently the only active agents,

the things that perform tasks, have been people. Erik Hollnagel (2003) has suggested that some

intelligent software can act as autonomous agents, although, as is so typical, I’ve argued for a

more extreme position (Diaper, 2004; 2006; Diaper and Sanger, 2006), that many things, not

necessarily involving an intelligent agent of any sort at all, can act as agents and should so be

modelled. This concept of analysing non-human agents is critical for my extension of the

Conception’s modelling of work systems and application domains.

Comment 51

A work system is the agent that performs work by transforming the application domain, and the

consensus, which I question (e.g. Diaper and Stanton, 2004), but won’t here, is that agents are

purposive, that is, they have goals and they attempt to transform the application domain to

achieve these goals.

Comment 52

The work systems in the TOM project, or those in Steve Cummaford’s

dissertation or in Dowell (1998), consist of a number things, at least one human, but also

computers and other objects. As the agent, however, it is the work system that possess the goals

pertaining to desired application domain change and not any one or more of its constituent

components. This is part of the Dowell and Long (1989) Conception, and the part which has

most inspired me.

Comment 53

When doing a general systems analysis, i.e. my much improved alternative to task analysis, which

I originally called SST4SM and which, with extensions, I have now renamed the SAM (Systemic

Analysis Method; Diaper, 2005), what has to be established, for some particular duration, is the

boundary that separates that part of the system that is functioning as a work system agent, from

the rest of the system, the application domain, which is changed. This is different from most task

analytical approaches where it is people who are agents, and does produce a different sort of

model from the task dialogue one discussed in the previous section. As with John Dowell’s

(1998) blackboard model mentioned above, in the SAM a work system, say consisting of a user

and a computer attempting an e-commerce transaction, would together possess the goals of

changing the application domain by successfully purchasing goods.

Comment 54

Overlapping in time,

however, at a lower level of analysis, shifting the work system boundary so that it is coextensive

with only the user, then in the SAM the user’s computer becomes the application domain and the

user has goals about changing it, so that at the higher level, together, they can change the goodspurchasing

domain.

Comment 55

In these general systems models there can be more than one work system

active simultaneously, although, as always, this is tricky to model, and these may persist for

different durations, but more importantly, for any moment (ti) the system can be described at

different levels by the analyst selecting different possible work system boundaries, each boundary

being identified by the goals of that particular work system, which I infer primarily from the

things that cross a work system boundary. If multiple work system boundaries are overlaid on a

systems model, then unlike with strictly hierarchical techniques, DFDs, HTA, Checkland, etc.,

the model is heterarchical, the boundaries can overlap without theoretical constraint which means

that things, abstract things like mental states and operations, and also software ones, as well as

physical things, can be constituents of different work systems, simultaneously.

Comment 56

Provided one remembers that we are talking about an analyst’s model of the world, and not the

(assumed) real one, then the above claim is not difficult to understand: it’s just as when using

physical reductionism one can have, simultaneously, models of water as a fluid that sloshes

around at our mesoscopic level and water at the lower, molecular level, where it enjoys hydrogen

bonding. My first journal paper (Diaper, 1984), a book review that was a springboard for

introducing some theoretical work on AI and which John gave me to do, argued with this

example that for engineering one needed to choose the appropriate level of model, a molecular

one, for example, being unsuitable for designing a dam. Similarly, to address changes to an

application domain, the appropriate level work system, whatever its constituent things are, needs

to be identified by the work system’s goals. This approach to modelling is described, and related

to software engineering, in Diaper (2004) with an ATC example; further examples, not all

involving computers, are described in Diaper and Sanger (2006).

Passing the Torch

This paper’s title asks in which direction John’s Festschrift will “pass the torch”, from him to me

or vice versa?

Comment 57

Or, as John put it in one of his email’s title field, “Passing the Torch (or Torching

the Past) – take your pick!” If the torch is the Conception, then I have already explained to him

that I am not a safe pair of hands simply because I do not have a strong commitment to the

engineering discipline he espouses as a solution to HCI’s continuing problems. Indeed, I’m not

sure it is ever going to work, and I can foresee the opposite alternative happening, that HCI,

human factors, etc. will disappear into some new form of software engineering (e.g. Cockton,

2008). In my bleaker moments I even see any HCI and CSCW stuff as icing on the cake and I’d

be satisfied if industrial practice got even a little bit closer to what is taught as software

engineering, even though what is taught is pretty simple compared to anything that has cognitive

in its title.

Comment 58

I abandoned TAKD because it was flawed and had reached the stage where the associated

baggage needed to correct the problems with using it, outside of its original purpose of syllabus

design, was getting too big and the costs too high, and I could now do something very much

better. TAKD was just a method, it could be described and I spent much effort trying to do so

(e.g. Diaper, 1989d), but the Dowell and Long Conception is theoretical, intended as a

foundation for an engineering discipline, and, as John has emphasised to me, he thinks requires a more

fundamental change, still, in current HCI perspectives. Could I pass, via this paper, the torch

back to John?

Comment 58

For the brightest of those who’ve worked in the disciplines to which John has contributed, and

why he so thoroughly deserved his festschrift, John has been a King.

Comment 59

 

I have never been more

than the King’s Terrier, although cheese and cheese, John has some of my terrier instincts,

chasing problems hard and, at times, down suspicious holes. I have not always chased the rats

my King might have most desired, but perhaps I have chased enough that John could use my

research as it has become integrated in my current general systems approach, the SAM (neé

SST4SM). I believe that I now have available better methods of analysis and representation, both

informal and formal, than John, and that these can be integrated with existing software

engineering practices, and understood and applied by those working in the IT industry who are

not psychologists or human factors specialists, and which go some small way to an understanding

of the nature of design itself and how it can be made more systematic in its prediction of the

possible futures that might be a design’s consequences.

I have, with explicit acknowledgement, made the Conception’s modelling of work systems and

application domains central to my general systems analysis methods and concepts relating to

work systems as agents, how their boundaries are defined by their goals, and much else that I

comment on above can be found, at least hinted at, in the Dowell and Long (1989) paper. John

still retains his commitment to the Conception for the development of a suitable, human

orientated engineering discipline, an enterprise to which I’m willing to contribute. My methods

make abstraction easy, so if John wishes to continue with deriving class design

problems/solutions from lower level specific ones, and iterating between levels, then what I have

is more powerful, flexible and valid than TAKD’s generification, which we worked out back in

the early 1980s. The Conception was a “basic prerequisite” (Dowell and Long, 1989) for

developing design principles. As my more recent methods integrate with my theorising about

predicting futures, I hope such might provide a means of capturing how design, specifically and,

particularly for John, at higher levels of abstraction, can and does occur.

So, King John, will you like the only English King to share your name, refuse to gild refinéd

gold? Here are my theoretical and methodological rats. Would the theoretical ones allow you to

have a more practical instantiation of the Conception, perhaps one you might choose to now

modify, and one that people could understand and use? I’ve developed powerful methods, which

I believe are usable, and that they can be expressed formally is, I believe, one aspect of my work

that could transform and support the sort of design principles John has wanted for so long. My

humble advice, however, Dear King, is that if you wish design principles, then these should be

about design itself. In summary, I offer the SAM for modelling systems of all sorts and for

modelling the design processes itself, the doing of design, methods, practices, etc., and the ATIS

hypothesis for modelling designers and, also, for predicting futures, which is at the heart of

design and which could start to address, and even improve, design creativity; the SAM and the

ATIS hypothesis are intimately connected since both, using the term broadly, are scenario based.

‘The time has come,’ the Walrus said,

‘To talk of many things:

Of shoes — and ships — and sealing wax —

Of cabbages — and kings —

And why the sea is boiling hot —

And whether pigs have wings.’

 

Lewis Carroll (1872) Through the Looking-Glass and What Alice Found There.

 

Comment 60

Acknowledgements

First, my great gratitude to Professor John Long, who’s idea, as reported in this paper’s

introduction, this paper was; alone I’d have written something different, or written nothing at all.

Second, my thanks to Professor John Long who, while being very clear in email to me what

would be inappropriate and that he could act “not as a reviewer, not as a co-author and not as an

editor”, for all his help and advice in the past and, in particular, no matter how unwitting and

inappropriate, with my early drafts of this paper. The work reported, however, is my own and

any errors, and the confusions that remain concerning the Dowell and Long Conception, are my

responsibility. Third, my considerable thanks to John Dowell for supply further papers and for

his detailed comment on earlier drafts of this paper; and to Steve Cummaford for supplying pages

from his doctoral dissertation.

References

Annett, J. and Duncan, K.D. (1967) Task Analysis and Training Design.  Occupational Psychology, 41,

211-221.

Annett, J., Duncan, K.D., Stammers, R.B. and Gray, M.J. (1971) Task Analysis.  Training Information

Paper No. 6, HMSO: London. Reprinted (1980) Training Information Paper Series. Manpower Services

Commission: London.

Annett, J. (2003) Hierarchical Task Analysis . in Hollnagel, E. (Ed.) Handbook of Cognitive Task Design .

17-36. Lawrence Erlbaum Associates.

Annett, J. (2004) Hierarchical Task Analysis . in Diaper, D. and Stanton, N.A. (Eds.) The Handbook of

Task Analysis for Human-Computer Interaction.  67-82. Lawrence Erlbaum Associates.

Benyon, D. (1998) Cognitive Engineering as the Development of Information Spaces . Ergonomics, 41 (2)

153-155.

Bruner, J. (1990) Acts of Meaning . Harvard University Press.

Carroll, J.M. (2000) Making Use: Scenario-based Design for Human-Computer Interactions.  MIT Press.

Checkland, P. (1972) Towards a Systems-based Methodology for Real-world Problem Solving.  Systems

Engineering, 2, 9-38.

Checkland, P. (1978) The Origins and Nature of ‘Hard’ Systems Thinking.  Applied Systems Analysis, 5.

99-100.

Checkland, P. (1981) Systems Thinking, Systems Practice.  Wiley.

Cockton, G. (2008) Working Spheres or Engagements: Implications for Designing . Interacting with

Computers, 20(2), 279-286.

Cummaford, S. and Long, J. (1998) Towards a Conception of HCI Engineering Design Principles . in

Ninth European Conference on Cognitive Ergonomics (ECCE9),  Limerick, Ireland. 79-84.

17

Cummaford, S. and Long, J. (1999) Costs Matrix: Systematic Comparison of Competing Design Solutions .

in Brewster, S., Cawsey and A. and Cockton, G. (eds), INTERACT 99, Volume II , 25-26. British Computer

Society.

Diaper, D. (1984) An Approach to IKBS Development Based on a Review of `Conceptual Structures:

Information Processing in Mind and Machine’ by J. F. Sowa . Behaviour and Information Technology, 3, 3,

249-255.

Diaper, D. (1987) Designing Systems for People: Beyond User Centred Design.  in Software

Engineering. Proceedings of the Share European Association (SEAS) Anniversary Meeting , 283-302.

Diaper, D. (1989a) The Discipline of Human-Computer Interaction.  Interacting with Computers, 1, 1, 3-5.

Diaper, D. (1989b) Giving HCI Away.  in People and Computers V.  (Eds. Sutcliffe, A. and Macaulay, L.),

109-120. Cambridge University Press.

Diaper, D. (1989c) Bridging the Gulf between Requirements and Design.  In Proceedings of the

Ergonomics Society Conference on Simulation in the Development of User Interfaces.  129-145.

Diaper, D. (1989d) Task Analysis for Knowledge Descriptions (TAKD): The Method and an Example.  in

Task Analysis for Human-Computer Interaction.  (Ed. Diaper, D.), 108-159. Ellis Horwood.

Diaper, D. (1990) Simulation: A Stepping Stone between Requirements and Design.  in Simulation and the

User Interface.  (Eds.) Life, A., Narborough-Hall, C. and Hamilton, W., 59-72. Taylor and Francis.

Diaper, D. and Addison, M. (1991) User Modelling: The Task Oriented Modelling (TOM) Approach to the

Designer’s Model.  in People and Computers VI.  (Eds.) Diaper, D. and Hammond, N., pp. 387-402,

Cambridge University Press.

Diaper, D. and Addison, M. (1992) Task Analysis and Systems Analysis for Software Engineering.

Interacting with Computers, 4, 1, 124-139.

Diaper, D., McKearney, S. and Hurne, J. (1998) Integrating Task and Data Flow Analyses using the

Pentanalysis Technique.  Ergonomics, 41, 11, 1553-1583.

Diaper, D. (2000) Hardening Soft Systems Methodology.  in McDonald, S., Waern, Y. and Cockton,

G.(Eds.) People and Computers XIV . 183-204. Springer.

Diaper, D. (2001a) Task Analysis for Knowledge Descriptions (TAKD): A Requiem for a Method.

Behaviour and Information Technology. 20, 3, 199-212.

Diaper, D. (2001b) The Model Matters: Constructing and Reasoning with Heterarchical Structural Models.

in Kadoda, G. (Ed.) Proceedings of the Psychology of Programming Interest Group 13th . Annual Workshop.

17th .-20th . April, 2001. 191-206.

Diaper, D. (2002a) Human-Computer Interaction.  in Meyers, R.B. (Ed.) The Encyclopedia of Physical

Science and Technology.  3rd . Ed. vol. 7, 393-400. Academic Press.

Diaper, D. (2002b) Scenarios and Task Analysis.  Interacting with Computers, 14, 4, 379-395.

Diaper, D. (2002c) Task Scenarios and Thought.  Interacting with Computers, 14, 5, 629-638.

Diaper, D. (2004) Understanding Task Analysis for Human-Computer Interaction.  in Diaper, D. and

Stanton, N.A. (Eds.) The Handbook of Task Analysis for Human-Computer Interaction.  5-47. Lawrence

Erlbaum Associates.

18

Diaper, D. and Stanton, N.A. (2004) Wishing on a sTAr: The Future of Task Analysis for Human-

Computer Interaction .  in Diaper, D. and Stanton, N.A. (Eds.) The Handbook of Task Analysis for Human-

Computer Interaction.  603-619. Lawrence Erlbaum Associates.

Diaper, D. (2005) In-use Computing Tasks, In-situ Work Evaluation with SAM (the Systemic Analysis

Method) . Paper at the BCS In-Use, In-Situ: Extending Research Field Methods  workshop. Middlesex

University IDC Technical Report IDC-TR-2005-4, 30-35. 27-28 October.

Diaper, D. (2006) Joint Cognitive Task Design . American Journal of Psychology, 119 (2), 338-347.

Diaper, D. and Sanger, C. (2006) Tasks For and Task In Human-Computer Interaction . Interacting with

Computers, 18, 1, 117-138.

Diaper, D. and Lindgaard, G. (2008) West Meets East: Adapting Activity Theory for HCI & CSCW

Applications ? Interacting with Computers, 20 (2), 240-246.

Dowell, J. and Long, J. (1989) Towards a Conception for an Engineering Discipline of Human Factors.

Ergonomics, 32 (11) 1513-1535.

Dowell, J. (1998) Formulating the Cognitive Design Problem of Air Traffic Management.  International

Journal of Human-Computer Interaction, 49, 743-766.

Green, T.R.G. (1990) Limited Theories as a Framework for Human-Computer Interaction . in Ackerman,

D. and Tauber, M.J. (eds.) Mental Models and Human-Computer Interaction . North-Holland.

Green, T.R.G. (1998) The Conception of a Conception . Ergonomics, 41 (2), 143-146.

Hockey, G.R.J. and Westerman, S.J. (1998) Advancing Human Factors Involvement in Engineering

Design: A Bridge Not Enough?  Ergonomics, 41 (2), 147-149.

Hollnagel, E. (2003) Prolegomenon to Cognitive Task Design . in Hollnagel, E. (ed.) Handbook of

Cognitive Task Design . 3-15. Lawrence Erlbaum Associates.

James, W. 1907. Pragmatism . New York.

Johnson. P. (1985) Towards a Task Model of Messaging: An Example of the Application of TAKD to

User Interface Design . in People and Computers: Designing the Interface.  (eds. Johnson, P. and Cook, S.,

46-62. Cambridge University Press.

Johnson, P., Diaper, D and Long, J. (1985) Task Analysis in Interactive Systems Design and Evaluation.

in Analysis, Design and Evaluation of Man-Machine Systems.  (Eds. Johannsen, G., Mancini, G. and

Martensson, L.), 123-127. Pergamon Press.

Lim, K.Y. and Long, J. (1994) The MUSE Method for Usability Engineering.  Cambridge University Press.

Long, J. (1986) People and Computers: Designing for Usability.  in Harrison, M. and Monk, A. (Eds.)

People and Computers: Designing for Usability . 3-23. Cambridge University Press

Long, J. (1989) Cognitive Ergonomics and Human-Computer Interaction: An Introduction.  in Long, J.

and Whitefield, A. (Eds.) Cognitive Ergonomics and Human-Computer Interaction.  4-34. Cambridge

University Press.

Long, J. and Dowell, J. (1989) Conceptions of the Discipline of HCI: Craft, Applied Science, and

Engineering.  in Sutcliffe, A. and Macaulay, L. (Eds.) People and Computers V . 9-34. Cambridge

University Press: Cambridge.

19

Monk, A.F., Nardi, B., Gilbert, N., Mantei, M. and McCarthy, J. (1993) Mixing Oil and Water?

Ethnography versus Experimental Psychology in the Study of Computer-mediated Communication . in

Ashlund, S., Mullet, K. and Henderson, E. (eds.) Inter-CHI’93 Conference Proceedings . 3-6. Addison-

Wesley.

Ormerod, T.C. and Shepherd, A. (2004) Using Task Analysis for Information Requirements Specification:

The Sub-Goal Template (SGT) Method . in Diaper, D. and Stanton, N.A. (Eds.) The Handbook of Task

Analysis for Human-Computer Interaction.  347-365. Lawrence Erlbaum Associates.

Patching, D. (1990) Practical Soft Systems Analysis.  Pitman.

Peterson, I. (1998) The Jungles of Randomness: Mathematics at the Edge of Certainty . Penguin.

Rizzo, A., Marti, P., Moderini, C., Decortis, F. and Rutgers, J. (2003) The Design of POGO World. in Hollnagel, E.

(ed) Hand book of Cognitive Task Design. 577- 602. Lawrence Erlbaum Associates.

Sell, P.S. (1985) Expert Systems – A Practical Introduction . Macmillan Computer Science Series.

Whitefield, A. (1989) Constructing Appropriate Models of Computer Users: The Case of Engineering

Designers.  in Long, J. and Whitefield, A. Cognitive Ergonomics and Human-Computer Interaction . 66-

94. Cambridge University Press.

Adam Stork & John Long

University College London, Gower Street, London, WC1E 6BT, UK
Corresponding author: Adam Stork, 32 Chisenhale Road, Bow, London E3 5QZ.  Tel: 0794 383 7676.  Email: adam@adamstork.com

Abstract

The paper summarises the key points and conclusions from the PhD research by Adam Stork [1] that was supervised by John Long.  The research aimed to develop ‘engineering principles’ as proposed by Dowell and Long [2].

The research was successful in developing early and initial engineering design principles: early, because the principles required further generalisation; and initial, because the principles had not been validated through application.  Hence, the research was considered to have made significant progress towards engineering principles.

Given that progress, the research provided a valuable perspective on a strategy for developing engineering principles.  This perspective led to the development of a version of a structured Method for Usability Engineering (MUSE [3]) for use in research (termed MUSE/R) along with shorter-term research products.  The paper focuses on this perspective on the basis that it provides input to research strategies.  Readers interested in the detail of the specific research strategy and the early initial engineering principles developed are recommended to consult the PhD itself.

1. Introduction

The research was primarily concerned with improving Human-Computer Interaction (HCI) design through the development of HCI design knowledge.  The requirement to develop design knowledge arose from a specific operational problem with design, raised by the industrial sponsor of the research.  The research demonstrated that this specific design problem was more widespread.  Essentially, the general operational problem was a failure to be able to design human-computer systems effectively, as expressed succinctly by Draper [4]:

‘The fundamental fact about HCI today is that neither computer science or cognitive science have any theories to offer that are adequate for predicting how a given design of user interface will perform.’

The primary aim in the research was to address these challenges for HCI, and enable HCI to deliver considerable benefits to human-computer systems.

2. Engineering principles as the technical solution

The technical expression of that general operational problem was identified as the requirement to make HCI knowledge more effective, and specifically to acquire HCI knowledge with a guarantee of application.

Dowell and Long’s (D&L) assessment of the discipline of HCI [2, 5] was applied to identify the technical problem.  Dowell and Long propose the development in the longer term of an engineering discipline, with knowledge that has a guarantee.  They term this knowledge ‘engineering principles’.

The development of engineering principles for HCI was the long-term technical solution addressed by this research.  The technical aim of this research was to make progress towards these engineering principles, through the development of a conception of engineering principles, a strategy for their acquisition, and an assessment of that strategy.

2.1 Conception of (substantive) engineering principles

Part of D&L’s proposal is a conception of the general design problem of an engineering discipline of HCI.  However, the research required and developed a conception of engineering principles, and scoped that conception to ‘substantive’ principles.  Substantive engineering principles ‘prescribe the features and properties of artefacts’ (D&L).  Components of engineering principles were conceptualised: general design problems; general design solutions; specific design problems; specific design solutions; partial design problems; and partial design solutions.
Since human-computer system costs are poorly conceptualised by D&L relative to task quality, the research also proposed an initial conception of human-computer systems and their costs.

2.2 Strategy for acquiring engineering principles

The research proposed a strategy for developing such substantive engineering principles and compared it with alternative strategies.

The strategy adopted took a bottom-up approach with two main steps:

• To identify general relationships between specific design problems and their solutions.  These general relationships would be considered putative, i.e. requiring validation, and termed initial engineering principles.  The identification of general relationships between specific design problems and their solutions requires the operationalisation of specific HCI design problems and their solutions from the conceptions of specific HCI design problems and specific HCI design solutions.
  The research applied best-practice current HCI design (‘SuperCraft’) to develop the specific HCI design solutions.
• To validate initial engineering principles by testing.

The research aimed to assess the proposed strategy by developing examples of initial engineering principles, which have not been validated by application.  They are, however, tested through evaluation (and thereby acquire ‘potential guarantee’).

The research was further scoped to cognitive ‘planning and control’ engineering principles and an initial conception of planning and control was developed based on the relevant literature.

2.3 Two cycles of best-practice development and operationalisation

The research conducted two cycles of best-practice development to solve two sets of energy management user requirements.  The resulting artefact specifications were informally evaluated against the user requirements, both analytically and empirically.  The evaluations were positive, supporting the subsequent operationalisations of the specific design problems and their solutions.

The operationalisations were of the specific design problem and solution conceptions, incorporating the conception of human-computer systems and the conception of planning and control.

2.4 Initial Engineering Principles

Examples of initial engineering principles were acquired by identifying general relationships between specific design problems and their solutions.  A detailed strategy for this acquisition was presented, to clarify this ‘identify’ process for finding general ‘relationships’.  ‘Identify’ was clarified as targeting relationships that are more likely to be general, and six forms of targeting are presented.  ‘Relationships’ are clarified as commonalities.  General relationships, either within or between operationalisations, are initial engineering principles.

The six forms of targeting were:

• Initial engineering principles identified during operationalisation(s).
• Initial assumption assessment from operationalisation(s).
• Inspirational initial engineering principle from operationalisation(s).
• Initial engineering principles from general guidelines.
• Initial engineering principles from guidelines.
• Initial engineering principles from task analysis.

The research developed specific notations for the initial engineering principles to enable their precise representation.

3. Assessment of the research strategy

The assessment of the strategy rests on the status of the acquired initial engineering principles.

3.1 Strategy and conception changes

The research strategy was initially considered to be a bottom-up strategy.  It might be claimed that the strategy is closer to a top-down strategy:

• The human-computer systems and the planning and control conceptions directly influenced the content of the initial engineering principles.
• The detailed strategy based the identification of initial engineering principles on Craft substantive design knowledge.

There remained a contrast with a top-down strategy, however.  Stork, Lambie and Long [6, 7] described a project that attempts the top-down strategy.  They started with an informal statement of Craft substantive design knowledge and then attempted to operationalise it as an initial engineering principle.  Accordingly, Stork and Long [8] proposed that there might be a continuum of strategies between the bottom-up and the top-down strategies, along a continuum of the expected initial generality.  Therefore, the research strategy was closer to the top-down strategy than originally anticipated, although it can still be distinguished from the top-down strategy.  Application of the alternative strategies outlined would have been beneficial to confirm strategy selection.

Identifying initial expected generality in the strategy raised a concern for the conception of engineering principles.  If the specific design problem and its solution conception contains concepts that relate to engineering principle acquisition, then are they required for engineering practice?  Possibly yes, although a specific design problem and its solution conception might need to encompass alternative such general conceptions to match the partial design problem and solution operationalisations (potentially to be applied).  However, it seemed more likely that it would be possible to operationalise partial design problems and solutions directly from the user requirements.  (A similar strategy that operationalised initial expected generality—the Hill et al [9] model of the planning and control of multiple tasks—for design is described in Stork et al [6].)

3.2 Status of initial engineering principles

Initial engineering principles were acquired.  These initial engineering principles had the pre-requisites for acquiring potential guarantee:

• They are conceptualised according to a conception of the discipline of HCI.
• They are operationalisations of conceptions based on that conception.
• They are generalised over or within the two cycles.
• They are tested by successful evaluations of the two cycles.

The generality was a remaining concern for the initial engineering principles.  Particularly, two or fewer cycles were considered poor generality, indicated by the difficulty of selecting appropriate general cycle types.  A further concern was that the expression of the initial engineering principles might not be appropriate for application.  These concerns indicate that the initial engineering principles should be considered ‘early’.

3.3 Validation of initial engineering principles

Validation of the initial engineering principles would involve re-expression as required and testing by application to a design situation.  Methodological engineering principles would be required for application.  The guarantee of engineering principles validated by application would be based on:

• The initial engineering principle guarantee.
• The operationalisations.
• The (known) generality.

Testing would be a challenge, however, since the effect of a particular engineering principle needs to be identified.  The alternatives appear to be:

• To control the designs to include or exclude the engineering principle application.
• To ‘trace’ the engineering principle application and its contribution to effectiveness.  Simulation may support this tracing.

Metrification of the guarantee of engineering principles could be considered at this stage.

3.4 Method and tool support

The research highlighted a requirement for methodological and tool support for the research strategy.  The strategy products could have been integrated with a method (see section 4).  Tool support could usefully support:

• Application of best-practice.
• Operationalisation.
• Detailed strategy: the identification of relationships.
• Initial engineering principle validation.

3.5 Craft and scientific knowledge

The strategy provided a means of potentially incorporating Craft and (Applied) Scientific knowledge into initial engineering principles.  Craft knowledge was incorporated by (Super-)Craft design.  Scientific knowledge was incorporated through the conceptions (for example, a cognitive architecture conception was included in the human-computer systems conception) and the operationalisation.

Dowell [10] claimed that engineering ‘principles could be validated through utility or through explanation by science’.  (Scientific disciplines have a general science problem of the explanation and prediction of phenomena.)  It seems likely that the potential guarantee of the early initial engineering principles was improved by their Craft and Scientific knowledge basis.  However, the status of the Craft and Scientific knowledge for design is not known, therefore determining that improvement is not possible.

3.6 Formal methods

The early initial engineering principles were formal in that they were operationalised to the level of metrics.  These metrics enable mathematical techniques to be employed.

‘Expressed as simply as possible, the goal of Formal Methods is to base the software development process squarely upon a workable set of mathematical techniques.’  Gerhart [11]

Formal methods are an approach that was being followed by Software Engineering (SE) [12, 13] to solve the SE ‘software crisis’ [14].  There had been significant analysis of the issues surrounding formal methods in SE [15, 16] and some in Human Factors (HF) [17, 18].  Stork [17] identified that their use in HF was restricted mainly to the formal description of interfaces (for example [19, 20, 21]).  There were several concerns about formal methods with implications for their use in informal engineering principles: the Fetzer ‘gap’ [15], the complexity of systems; the purposiveness of animates; and the executability of specifications.

Fetzer [15] argued against formal verification. He accepted that there may be a formal path between requirements specification and a solution specification, but claimed that there will never be an unbroken path from the formal solution specification to a solution implementation.  The initial engineering principles distinguished between a general design solution and its artefact specification.

Cantwell-Smith [16] claimed that the increasing complexity of computer programs would prevent them from being proven correct.  The analogy considered in the research was that engineering principles might be too complex to be formally applicable.  However, the initial engineering principles were not complex.  Also, in high risk situations (for example, safety critical systems) complexity may deliver valuable results and so be worth significant effort in formal application.

Becker [22] underlined the value of mathematics: ‘Mathematics arises from man’s attempt to find concepts that allow a wide range of phenomena to be described in similar terms, and thereby understood in a coherent manner.’  He claimed that mathematics cannot be used for the phenomena of behaviours:  ‘It is quite possible that animate behavioural systems are organized in ways (e.g., ultra-high parallelism) that are not compatible with our traditional habits of induction and part-whole analysis.’; and ‘A “behavior” is not a well-defined thing like a numerical quantity; rather it is a selective account of an event that is defined only by certain decisions on the part of us, its describers.’  However, the initial engineering principles were not concerned with a scientifically-correct description of the behaviour, only the utility of the initial engineering principle.

Hoare [23] dismissed the requirement for formal specifications to be executable: ‘To require a specification or design of a program to be executable is hardly less absurd than requiring the specification of a building to be habitable or the blue-prints of a car to be driveable.’  However, tool support may be required to apply engineering principles.  Further, the formalism may need to be altered to enable tool support [24].

The main concern for a formalism ‘is whether it is useful, and I would add usable to that’.  (Stork [17], following [25, 26]).  The known guarantee of engineering principles would ensure usefulness.  Usability of engineering principles relates to the requirement for that known guarantee (e.g. safety critical systems may be worth significant effort).

4. MUSE for Research (MUSE/R)

A MUSE application involves the construction of products that have a well-defined scope, process, and notation.  Similarly, the application of the research strategy involved the construction of products that have a well-defined scope and notation.  The scope and notation of the MUSE products and the research products were related to each other to propose a version of MUSE to support research similar to the research.  The research proposed and outlined such a version of MUSE, termed MUSE for Research (MUSE/R).

4.1 Scope and notation

The research strategy had four main products: specific current design operationalisations; specific design problem operationalisations; specific design solution operationalisations; and initial engineering principles.  The scope of the products is expressed in the product name.  The notations for the specific current design and specific design solution were covered by: domain, structure, and behaviour diagrams; and quality, structure changes, and behaviour formulae values.  The notations for the problem and initial engineering principles were logical/mathematical expressions.
MUSE was considered to have potential for supporting the research strategy because:

• Most of the products are operationalisations of specific design situations: either systems being analysed or the system being designed.
• The products are explicit with a well-defined and explicit scope.

The scope, processes, and notations of the MUSE products were considered to need to be enhanced to accomodate the research conceptions and to support the research strategy.

MUSE has three phases: the information elicitation and analysis (IEA) phase; the conceptual design (CD) phase; and the detailed design (DD) phase.  The first stages of the IEA phase involve the analysis of extant systems: including operationalisations of the ‘tasks’ and ‘domains of discourse’ of the systems; and generalisations of the tasks.  For MUSE/R, these stages were re-scoped to operationalise: the specific structures and behaviours of the extant worksystems; their structural and behavioural costs; their domain; the quality of their work; and appropriate generalisations of these operationalisations.  Thus, the first stages of the IEA phase operationalise specific current designs according to the research conception.
The final stages of the IEA phase and the first stages of the CD phase in MUSE involve the specification of a human factors statement of the user needs.  For MUSE/R, these stages were re-scoped to operationalise the specific design problem.  The final stages of the CD phase and the DD phase in MUSE involve the specification of the interaction artefact and the documentation of the design rationale.  For MUSE/R, these stages were re-scoped to operationalise the specific design solution and the previously-acquired HF knowledge applied to develop the specific design solution.

The notations for the re-scoping were those employed by the research strategy, which are similar or additional to the MUSE notations no longer employed.

HCI design knowledge, including MUSE/R, was considered to also have potential for application to more general research strategies, including validation of HCI knowledge.  Stork and Long [27] outlined the more general case for HCI research and development.

4.2 Process

The redefinition of scope of MUSE/R suggested the process changes.  Figure 1 shows the overview of the processes of MUSE/R.  The redefinition demonstrated potential for reduction in research effort relative to this research.

Figure 1.  MUSE/R Overview

4.3 Support for design

MUSE/R needs to support HCI practice at least as well as MUSE.  The accomodation of the research conceptions suggested that the MUSE/R products will be more complete and coherent (relative to the general design problem conception of HCI) than the MUSE products, suggesting improved support for design.  (Linked with claims for non-Engineering application of the D&L general design problem conception.)

Görner [28] claimed that ‘a significant relationship [exists] between the amount of goal characteristics analysed and the quality of resulting design solution’ (see also Chung et al [29] who employ a similar diagram for design).  MUSE/R emphasised the operationalisation of task quality relative to MUSE, so potentially increasing ‘the amount of goal characteristics analysed’.  Further, Gomaa et al [30] claimed that domain model analysis supports design (including different views of the domain, as implied by the specific design conceptions).  Ege and Stary [31] successfully employed a combined task and object approach to designing interfaces.

MUSE/R has been applied to the design of an in-car newspaper-information device.  The scope, process, and notation of the products were defined further and MUSE/R applied informally.  A different research strategy was employed: the Hill et al [9] model of planning and control of multiple tasks was operationalised for the products, to attempt informal validation of the model for design.  A successful design was produced.

4.4 Further research on MUSE/R

Although an initial application of MUSE/R was conducted, the scope, process, and notation of the products needed to be defined further to support application.  Further resarch and design case-studies need to be performed to assess the support MUSE/R offers for research and design.  Further, the apparently strong relationship between research and design in HCI needs to be investigated further.

The evaluation component of the strategy could be incorporated into the MUSE/R method (and remain consistent with design [32]).  Tool support designed for MUSE/R would be essential, since both the design and research parts of this research would have been significantly assisted by designed tool support.  Automatic generation of prototype interfaces would be advantageous (for example, the DIANE method provides automatic help generation [33]).

5. Conclusions

Research is the acquisition of knowledge to support a purpose.  The research has acquired: a conception of engineering principles, a strategy for the acquisition of engineering principles, examples of early initial engineering principles, a positive assessment of the strategy at this juncture, and next steps for the acquisition of engineering principles.  The research has made significant progress towards solving the operational problem of the inability to design human-computer systems effectively.

In addition, shorter-term benefits can be identified.

Recognising the targets of the research was an important aim in appropriately focussing the research and acquiring knowledge towards solving the operational problem.

5.1 Technical solution

The research acquired early initial engineering principles to support the effective design of human-computer systems.  The acquisition of these early initial engineering principles supported the research strategy and took HCI substantially towards engineering principles.

5.2 Operational solution

Taking HCI towards engineering principles offered the potential for more effective HCI practice, as required to solve the operational problem, which was the inability to design effectively human-computer systems.

5.3 Strategy

The explicit strategy and implementation for the research, and its discussion, provided a basis for other HCI researchers to consider their acquisition of HCI knowledge.  The MUSE/R method provided a basis for other researchers (and designers) to apply the strategy.

5.4 Informal assessment of Dowell and Long

The D&L consideration of the discipline of HCI as an Engineering discipline and their conception of the general design problem of HCI enabled the research to acquire early initial engineering principles.  The research informally supports D&L’s consideration and conception.

5.5 Shorter term benefits

Several shorter-term research goals have been met by the research.

A version of MUSE was developed, MUSE/R, that supports more complete, coherent, and consistent specification of the design problem and solution.  It is expected that MUSE/R has the potential to improve HCI practice in the medium- and longer-terms.
Conceptions and operationalisations of design problems and solutions are available to assist practitioners in the short term to be better able to identify their design problems and solutions, and to be better able to assess that the solutions solve their design problems.

A short term goal of MUSE assessment had been achieved (Stork et al [34]).  MUSE applications have been delivered that can be used to support MUSE training and application.  Two example artefacts to HCI user requirements have been developed.  Guidelines from the development of the artefacts have been acquired.

6. References

[1] Stork, A. 1999.  Towards Engineering Principles for Human-Computer Interaction (Domestic Energy Planning and Control).  PhD thesis, University of London.

[2] Dowell, J. and Long, J. (1989).  Towards a conception for an engineering discipline of human factors. Ergonomics, November 1989, 32(11), pp. 1513-1535.

[3] Lim, K. Y., and Long, J. (1994).  The MUSE Method for Usability Engineering.  Cambridge University Press.

[4] Draper, S. (1991).  User Interface Performance Measurement.  JCI Summer School August 1991.

[5] Long J. & Dowell J. (1989).  Conceptions for the discipline of HCI: Craft, Applied Science and Engineering.  In: A. Sutcliffe & L. Macaulay (eds).  Proceedings of the Fifth Conference of the BCS HCI SIG, Nottingham, England, 5-8 September 1989, Cambridge: Cambridge University Press.  pp. 9-32.

[6] Stork, A., Lambie, T., and Long, J. (1998).  Cognitive Engineering Coordination in Emergency Management Training.  HCI ’98 Conference Companion, J. May, J. Siddiqi, and J. Wilkinson (eds), pp. 36-37.

[7] Lambie, T., Stork, A., and Long, J. (1998).  The Coordination Mechanism and Cooperative Work.  Proc. of the Ninth European, Conference on Cognitive Ergonomics, T. Green, L. Bannon, C. Warren, and J. Buckley (eds), pp. 163-166.

[8] Stork, A., and Long, J. (1998).  Strategies for Developing Substantive Engineering Principles.  HCI ’98 Conference Companion, J. May, J. Siddiqi, and J. Wilkinson (eds), pp. 36-37.

[9] Hill, B., Long, J., Smith, W., and Whitefield, A. (1995).  A Model of Medical Reception—the Planning and Control of Multiple Task Work.  Applied Cognitive Psychology, vol. 9, pp. S81-S114.

[10] Dowell, J. (1993).  Cognitive Engineering and the Rationalisation of the Flight Strip.  PhD thesis, University College, London.

[11] Gerhart, S. (1991).  Formal Methods: An International Perspective.  13 Int. Conf. on Software Engineering, Austin Texas, pp. 36-37.

[12] Hoare, C. (1969).  An Axiomatic Basis for Computer Programming.  Communications of the ACM, vol. 12, no 10, pp. 576-583.

[13] Gerhart, S. (1990).  Applications of Formal methods: Developing Virtuoso Software.  IEEE Software, September 1990, pp. 7-10.

[14] Pressman, R. S. (1982).  Software Engineering: a practitioners approach.  McGraw-Hill.

[15] Fetzer, J. (1988).  Program Verification: The Very Idea.  Communications of the ACM, vol. 31 no. 9 pp. 1048-1063.

[16] Cantwell-Smith, B. (1985).  The Limits of Correctness.  Paper prepared for the Symposium on Unintentional Nuclear War, Fifth Congress of the International Physicians for the Prevention of Nuclear War, Budapest, Hungary.

[17] Stork, A. (1992).  A Formal Description of Worksystem Behaviours and Interactions.  MSc (Ergonomics) Thesis.  University of London.

[18] Bauer, B. (1995).  Proving the Correctness of Formal User Interface Specifications.  Design, Specification and Verification of Interactive Systems ’95.  P. Palanque and R. Bastide (eds).  Springer.  pp. 224-241.

[19] Alexander, H. (1985).  Two Techniques for Specifying and Prototyping Interactive Software.  University of Stirling internal report.

[20] Alexander, H. (1987).  Executable Specifications as an Aid to Dialogue Design.  Formal Methods and HCI 1987, IEE Computing & Control Division Colloquium.

[21] Anderson, S. (1987).  Analysis and Synthesis in HCI.  Formal Methods and HCI 1987, IEE Computing & Control Division Colloquium.

[22] Becker, J. (1975).  Reflections on the Formal Description of Behavior.  In: Representation and Understanding, Bobrow and Collins (eds), Academic Press, pp. 83-102.

[23] Hoare, C. (1989).  Formal Methods in Computer System Design.  Computer Physics Communications 57, pp. 206-210.

[24] Breuer, P.T. and Bowen, J.P.  (1994). Towards correct executable semantics for Z. In Proc. Z User Meeting, J.P. Bowen and J.A. Hall (Editors), Springer Verlag Workshops in Computing, pages 185–209. Springer Verlag, June 1994. Cambridge, UK.

[25] Milner, R. (1989).  Communication and Concurrency.  Prentice-Hall.

[26] Dix, A. (1991). Formal Methods for Interactive Systems.  Academic Press, Cambridge.

[27] Stork, A. and Long, J. (1997).  Structured Methods for Human Factors Research and Development.  HCI International ‘97.

[28] Görner, R. (1994).  Zur Psychologischen Ayalyse von Konstrukteur- und Entwurfstätigkeiten.  Die Handlungsregulationstheorie.  Von der Praxis einer Theorie, B. Bergmann and P. Richter (eds).  Hogrefe, pp. 233-241.

[29] Chung, L., Nixon, B., and Yu, E. (1995).  Using Non-Functional Requirements to Systematically Support Change.  Proc. of the 2nd IEEE International Symposium on Requirements Engineering, pp. 132-139.

[30] Gomaa, H., Kerschberg, L., and Sugumaran, V. (1992).  A Knowledge-Based Approach to Generating Target System Specifications from a Domain Model.  Information Processing 92, vol. 1, pp. 252-258.

[31] Ege, R., and Stary, C. (1992).  Designing Maintainable, Reusable Interfaces.  IEEE Software, November 1992, pp. 24-32.

[32] Hacker, W. (1997).  Improving Engineering Design—Contributions of Cognitive Ergonomics.  Ergonomics, vol. 40, no. 10, pp. 1088-1096.

[33] Barthet, M-F.  (1993). L’intégration de l’ergonomie aux méthodes informatiques de conception de logiciels. In: Congresso Latino Americano de Ergonomia, II, 1993, Florianópolis. Anais… Florianópolis: ABERGO: FUNDACENTRO, 1993. p. 87-111.

[34] Stork, A., Middlemass, J., and Long, J. (1995).  Applying a Structured Method for Usability Engineering to Domestic Energy Management User Requirements: A Successful Case-study. BCS HCI 1995.  pp. 367-385.

 

 

 

Tony Lambie

t.lambie@ucl.ac.uk

Effective Work and the Autonomy of Design

The Ergonomics Unit was John Long’s creation, and the seminal ideas of the Unit are best summed up, I think, as ‘effective work and autonomy of design’.  John was an outstanding teacher equipped with these burgeoning ideas.  They were contentious ideas too – the best kind; at first blush, simple ideas but with hidden dangers.  We knew they were burgeoning because everyone who came to the ‘Unit’ took them on.  Not only were they challenging and fruitful, but they imposed discipline, with the added value of John’s supervision; excellent ideas upon, and around which, to build PhDs.

My thesis was on the status of these ideas as elements of a distinct engineering discipline, and how that discipline could be entirely autonomous of science and yet not come adrift from it epistemologically.  But I was also interested in how John Dowell (another PhD student) along with John Long, had formalised the ‘Conception’ of those two ideas, among others: an engineering ontology for HCI.  Projects I have been involved in have required effective collaboration and I have wondered how that ontology could be augmented to accommodate multiple agents at the interface.

The following is a brief exposition of an attempt to adapt those ideas to that end.  It is a revised theoretical framework, part of a consultancy project on which I worked with John Long, but whose agreement with it should not be assumed, though I use the first person plural pronoun.  However, I hope he might be sympathetic to the general direction of my views, and their good intentions.

1. Views of Coordination and Work

1.1 Introduction

Computer Supported Cooperative Work (CSCW) has been dominated by views which have an anthropological or sociological origin.  These views are appealing because they address a less constrained, but arguably more realistic, set of problems than those traditionally tackled by human factors specialists.   Socially-oriented practitioners choose to approach the complex process of group behaviour in this way because they do not consider the paradigm of a model or framework appropriate for such a protean phenomenon.  It is this reluctance, we believe, which is largely responsible for the difficulty faced when attempting to integrate conventional systems development work with an understanding of the part played by the joint activity of those carrying out the work.

The challenge is to promote the idea of a framework or model to aid design reasoning, without imposing constraints: in short, to manage the design problem of cooperative work without practising reductionism; and at the same time, to maintain a consistent and systematic overall view so that a common basis might be established between the different stages from the understanding of the problem to the setting out of a solution.  This common framework or model might support a language of representation for use by diverse designers to enhance mutual understanding throughout the different stages of the problem formulation and solution.

2.1  Some Frameworks or Models

Schmidt’s conceptual framework of cooperative work [1] is a comprehensive review of the aspects of collective behaviour at work, and covers its gross and obvious as well as its fine and subtle features, and Schmidt, indeed, has something interesting to say about the core notion of cooperation which we believe is associated with the idea of teamwork.  However, just as we shall not address classes of coordination, such as ‘augmentative’, ‘integrative’ etc., nor the work organisation nor the modes of cooperation (all dealt with in Schmidt’s paper), except incidentally, so we are not concerned with whether the cooperation is between experts and experts, or experts and non-experts, as Falzon’s [2] paper is.  Likewise, whether design is not contingently a cooperative activity but essentially and necessarily a collective one, as Bødker [3] asserts (“Design is a collective activity….”), need not detain us at this point.

All these writers recognise collectivity or potential collectivity of work, but none offers a framework which underpins the workers’ interaction, nor do they try to forge a connection between the systematic (and established) activity of the software engineer and this multiple and mutual interaction.  Falzon, for example, deals with it as an instance of dialogue governed by the general principles of dialogue, and it is of course useful for that.  The mechanics of the dialogue of any cooperative activity must be understood and their representation is going to be a key component in any specification or diagnosis of group work effectiveness.  However, as Brehmer [4] points out, “cooperation can be achieved without communication”.  That is to say, we can look below explicit dialogues for the principles of cooperative behaviour, and, arguably, we can go further as Schmidt [1] does when he writes, “conceptualisation is in principle a coordination of viewpoints”: i.e., that individual and collaborative cognitive behaviour is fundamentally inseparable in group work.  It is in this region that we shall look for a ‘device’ which will help us to reason about cooperative work design problems.

Storrs [5] clearly deals with some sort of group work scenario but his paper constitutes, as he himself describes it, an ontology and, thus, is more descriptive and classificatory than ours, which will also attempt to account for, or provide a rationale for, the multiparty interaction – in the interests of the systematic specification of teamwork design, i.e., it comprises a prescriptive element.

Others, such as Rasmussen [6] and Vicente [7], who aspire to some form of engineering, deal with group interactions to carry out work, and provide frameworks of a kind.  However, our conceptualisation (not to be confused with Schmidt’s term [1]) is different from theirs, and will extend it by re-interpreting the role of agents vis à vis one another and the complex of tasks which make up the work.

Our conceptualisation of cognitive coordination is an addition to, or an elaboration of, a conception of design peculiar to the Ergonomics & HCI Unit at University College London (see Dowell & Long [8] & [9]) – a version of cognitive engineering.  So we start out with an account of the background against which the conceptualisation has taken place.  We shall not provide a separate account of the Conception of a Cognitive Engineering discipline [8] & [9], but will introduce the features relevant to the interaction and the domain of work, and the way in which it might be re-conceived in part to permit multiple agents.

2. The Conceptualisation

2.1 Performance and Engineering

There are two main issues to be addressed when one considers the part the agents play in any work domain: firstly, how they relate to that work domain, i.e., how the interface should be represented, at least for the designer; and, secondly, how they relate to one another in the work environment.  Perhaps the essential feature of cognitive engineering, as we understand it, is that it is a cognitive design activity which takes place against a background of scarce resources, as other kinds of engineering do.  In order that a design problem can be defined and solved, a clear understanding is required of what constitutes performance. For this to be possible, agents doing work are, to one degree or another, conceived as set over and against the work domain, in order to facilitate reasoning about the design problem, e.g., to consider the merits of different specifications of agents’ behaviour to achieve the same work result, thereby answering the requirements, and enabling an assessment of the performance’s effectiveness.  It is true that there is a dynamic and reflexive relationship which develops between the work being done and the work to be done, and that the agent contributes to this developing situation, i.e., the workers transform the object of their work, and consequently have to deal with a new object, sometimes with unanticipated properties. However, in order to get something done to advance the problem, i.e., to facilitate design reasoning, a decision has to be taken as to where the dividing line is between the worker, or workers, and the domain of work.

Thus a distinction is made between the work to be done and the means of getting that work done; the design problem being how to improve the means in order to get the work done more effectively.  Enhancements to both must be relative to some criteria.  These criteria, pertinent to each, are the resource costs of implementing the means, in the case of the agents’ behaviour, and the quality of what is done, in the case of the domain.  That is to say, the doing of the work implies commitment of time and effort and, for the agents, the aim is to make these acceptable in the context of the requirements and the constraints of the setting.  The work has to be performed as desired.  The simplest way of putting this is to say that the costs of the work and its quality have to be weighed in the balance; and it is that balance which constrains the process of finding a design solution.

To put the point conversely, unless we have a clear idea of the agents’ behaviour and the domain of work, we cannot accurately and consistently measure the  costs of their behaviour relative to the quality of work and, therefore, our design solution, or solutions, would be baseless, i.e., we would have no way of measuring performance.

Figure 1 Interactive Work System & Domain

As Dowell & Long (Dowell & Long, [8] & [9] have it, the conception of the design problem involves the fundamental duality of agents’ behaviour and the transformation of the domain contents, its objects and attributes.  These are, respectively, the Interactive Work System (IWS) and the Domain of Work (the Domain) (Figure 1).
This duality is the basis for the measure of performance; and is why we can consider such a design problem specification to be, potentially, an engineering one.  It is important, however, not to conclude that this separation, which is well-defined in order to permit such a calculation, leads to a radical detachment of the IWS from the Domain.

2.2 Autonomous Design, Human Factors, & the User’s View of the Work

As Dowell & Long [9] have also indicated, referring to Simon [10] and Neisser [11], it makes no sense to consider one in isolation of the other.  They write, “If the worksystem is well adapted to its domain, it will reflect the goals, regularities and complexities in the domain”.  Thus, what we are considering is a conception which not only permits a clear specification of a design solution determined by the calculation alluded to above but also, in spite of that rational basis, is arrived at through an understanding of the values of the work to be done.  It is not, in any sense, a solution dictated by factors, or knowledge, external to the design problem, such as that, for example, drawn directly from the psychology of cognition or perception, though such knowledge might be the inspiration for a ‘conjectured’ (Popper [12]; see also below) design model to be defined as engineering knowledge through systematic design practices. The solution is at once domain-driven and IWS-constrained (in a way which awaits the particular knowledge of the domain and user behaviour).

In order that there is a means of connecting these limiting forces during the design problem-solving (i.e., the forces of motivation, or goal-fulfilment, on the one hand, and the constraints, representing the costs of so doing, on the other), there has to be a semantic connection between the IWS and the domain.  This semantic connection is that of the intentions of the agents and the goals in the domain.  These are linked, in the sense that the intentions result, when implemented meaningfully, in the transformation of states of the domain of work.  In general terms, intentions in the work system are continuous with the goals in the domain, in the sense that objects in the domain accommodate those intentions (the affordances of the objects, as elements of ‘states of affairs’). An intention is not an intermediate goal to be fulfilled before the domain goal is specified or fulfilled.
However, though they are one in this sense, they are not one and the same, because the structures which support the behaviour of the worksystem and those which underlie the possible transformations in the domain are different depending on the work done.  In one case, they may underlie the behaviour, and in other, support the work done.  In any one interactive work system/domain of work they will relate the one to the other.  The ‘reflections’ which Dowell & Long [10] (quoted above) allude to, in the well-designed system, amount to a congruence between these distinct structures.  It is this double aspect of semantic continuity and syntactic disjunction which allows us to maintain the intimate connection between the users and the work, while rigorously separating them so as to define, and tackle, the design problem in a systematic manner.

The relationship, then, between the IWS and the Domain can be articulated

• so that performance may be measured, and
• so that a specification of the semantics of the design problem may be made and the solution may be addressed.

2.3 Human Factors and Joint Cognitive Behaviour

We have written throughout the last two subsections of the agents’ (plural) interaction with the objects in the domain, but in the Conception the agent is usually associated with the single user, as in a human-computer interaction.  The relationship between several agents might produce odd results if we begin to consider each other as objects in the domain of work and not in the interactive worksystem.

However, the relationship between the agents in the IWS is not one of means-to-ends in a typical design problem, though people do use others in this way and one can envisage a design of a Machiavellian kind of problem. Indeed, this may be feature of the conception’s application to collaborative design problems which CSCW researchers think intractable.  The agents must then be ends-in-themselves or they would be objects in the Domain, coming and going in and out of it, muddying the specification.  When we refer to agents and the work they do, we mean that they are essentially all active and autonomous agents.  So, in what might this relationship of cooperation consist?

It is the intention of this paper to introduce a further concept consistent with the framework, one which underpins this relationship.  In other words, the paper will introduce a structure, or mechanism, which supports joint behaviour in the IWS, permitting the inclusion of social aspects in both the definition of the design problem and the specification of the design solution.

The mechanism is intended to fulfil a cognitive and a social function at one and the same time.  However, our approach is still able to aspire to the title of engineering because of its emphasis on performance as the accomplishment of effective work of a desired quality at the expense of acceptable costs. It is often assumed that such an approach is reductive and mechanistic, and is unable to accommodate adequately the subtler concepts of cognition.  It might be thought, for example, that such an approach would work if it is limited to the simplest case, but that ‘true’ CSCW would be beyond it, because in the class of socio-technical systems the focus is at the social end of the spectrum, which features strong social influences less constrained by technical knowledge, and composed of the co-workers’ diverse sets of values or perspectives (Vicente [7] points this out but fails, we believer, to deal with just those values and perspectives).  The development of our conceptual framework for cooperative work should offer just this possibility.

3. Other Engineering Approaches

There are other workers in the field of human factors who also espouse an engineering approach, notably, when one considers the question of the behaviour of collective systems: researchers in cognitive systems engineering (CSE), and others comprising Rasmussen, Vicente, Woods, Hollnagel, Roth, etc.  Their work is very rich and provides many means of representation for cognitive design purposes which assist design problem expression.  They, for example, offer levels of description of agents’ behaviour in terms of skills, rules and knowledge, and as such their means of representation of this behaviour is consistent with another group of human factors workers, those espousing activity theory (AT) (see Draper [13]).  However, as we have pointed out above

• these distinctions, in the first place – and with respect to, CSE, for example – concern individual agents, and do not relate those representations in social terms
• in the second place, and with respect to AT, although the group shares the cognition, the AT theorists can only advance design knowledge to a limited and uncertain extent without an explicit framework to support this collective cognition
• and in the third place, none of the above groups has an explicit and objective means of measuring performance, the pre-requisite of effective design products, in the sense pertaining to our cognitive engineering approach, outlined above.

Vicente and the CSE workers, in addition, have a concept of task which is left less well specified, apparently in order to leave room for evolutionary design development, i.e., not to limit the designer’s freedom nor that of the worker implicated in the designed system.  However, we believe that it is possible to specify the task clearly as long as one is not confined to the conventional level of work (or technical) tasks; that is to say, if the notion of task is deepened to comprise the cognitive process, and is understood as shared in a more fundamental way (this should become clearer in our conceptualisation of cognitive coordination).

Vicente [7] argues for a review of the concept of task analysis, and we take account of this difficult concept.  However, we feel that there is no need to make too much of Vicente’s distinction between “instruction-based” task analysis and the “constraint-based” form, but simply to recognise that task analysis must always be qualified by the limitations of domain knowledge.  In other words, we hold to the position that a normative (i.e., a systematic and prescriptive) approach to design is required, but do not conclude, as Vicente does, that this implies dictated and rigid design solutions.  The reason we are in a position to do this is that we take the view that such rigidity, which we agree is a danger, follows from applying, for example, psychological knowledge and failing to recognise the particularity of the design problem; as well as from assuming that the way things are done determines the way things should be done. We make no such assumptions and pursue engineering, not scientific, knowledge of workers’ behaviour,.  Thus, though our output is prescriptive it is not a prescription determined by the way things are done in some domain of work nor the scientific findings of psychology which may or may not be appropriately scoped for the design problem under investigation.  This tendency to lean heavily on psychology, we believe, stems from the perceived need for science to underwrite the guarantee for good design knowledge.

It is taken for granted that secure factual or descriptive knowledge is warranted by science, but as Popper has pointed out, even science needs to establish its knowledge through its practices: the hypotheses are bold speculations (Popper [12]).  As we are trying to secure engineering knowledge we must, analogously, establish it through engineering practices, which are distinct from those of science and have different goals: diagnosis and prescription in the case of engineering, and explanation and prediction in that of science.  To begin with, then, and analogous with Popper’s description of the acquisition of scientific knowledge, we start with a ‘conjecture’– the conceptualisation – but we do so with the intention of integrating it within a larger framework of cognitive engineering and, crucially, with the intention of operationalising, and ultimately testing and generalising some solution to the design problem.

After the conceptualisation, the exposition of the rationale for the conceptual structure or framework, there follows what counts as the next step in the acquisition of early design knowledge: seeing how it assists the analysis of a particular design problem.  We might call this an intermediate stage between that of conceptualisation and that of operationalisation properly speaking: a stage where the concept benefits from feedback provided by its employment in the diagnosis of design issues – in this case, coordination issues.

The operationalisation has a converse and an obverse: the framework needs operationalising as part of the process of validation; and the properties, in this case of teamwork, need it too so that the design problem can be made more tractable, in such a way that reasoning about a design solution can be facilitated .

4. Conceptualisation of Cognitive and Social Coordination

Shaw & Fox [14] consider three types of system which might be thought of as sharing the task of problem-solving: collaborative reasoning systems; distributed problem-solving systems; and connectionist systems.  Among the considerations which these types of group reasoning share (i.e., a component of a common framework for such systems) is what the authors call the ‘coordination mechanism’ (a term which we shall adopt, enhance, and integrate into the above conceptualisation of the design problem): “since each problem-solving agent only possesses a local view and incomplete information, it must coordinate with other agents to achieve globally coherent and efficient solutions”.   They believe that “the design of coordination can be viewed from three different perspectives: the information content, the exercise of control, and the coordination mechanisms”.  They go on to state that “coordination can be achieved through passing different types of information among the agents, such as data, new facts just generated, partial solutions/plans, preferences and constraints”.  We may think, then, of these as the objects or contents of the coordination process.  The authors suggest that “the initiative to coordinate may result from a variety of means of control: it may be self directed, externally directed, mutually directed, or a combination of them, e.g., coordination by synchronisation, coordination by negotiation etc.  In short, therefore, the information etc. which is passed could be thought of as the content of the process, the “means of control” might be understood as the direction and force of the passing of that content.

If we look at Schmidt’s framework [1] we find him asserting, as we have seen, that “conceptualisation is, in principle, a coordination of viewpoints”, that “conceptual thinking is a cooperative effort”.  We come then, in his view, enabled for coordination; and there is no inherent difficulty for us in coordinating: we start with a primitive basis for mutual knowledge, without which we could not communicate, i.e., it allows us to coordinate our attempts to communicate; unlike Shaw & Fox’s agents, which only possess “a local view”.  It is part of what it means to have a conceptualisation of the world that it is shared.  In contrast, the artificially intelligent agents under discussion by Shaw & Fox require an explicit input under something like the categories of ‘object’ and ‘means of control’ – what they call the “coordination mechanism” – in order for them to be coordinated (Fig.2).  Like Schmidt, Hutchins [15] writes, “the distribution of labour can only be negotiated if the distribution of knowledge and ability is at least partially redundant”.  That is to say, cooperative behaviour presupposes a common or shared view – a fundamental coordination, such as Schmidt writes about.

What we may now consider is how this natural, and fundamental, coordination which Schmidt writes might be characterised in a more formal and explicit manner by reference to the idea of a ‘coordination mechanism’.  If this is possible we could be better equipped to articulate problems concerning the design of group behaviour.
What we are looking for then is a means of amalgamating the analysis of Shaw & Fox, which exposes the elements of a coordination mechanism but minus the assumption of mutual knowledge.  There is a convenient concept associated with linguistic behaviour which has striking similarities in structure and in function, we shall argue, to the coordination mechanism as exposed by Shaw & Fox [14]: that of the Speech Act (Austin, [16]; Searle, [17]).

The Speech Act is a structure with two components, the ‘illocutionary’ and the ‘propositional’, where the illocutionary might be the act of commanding or asserting something etc., and the propositional content might be the representation of some state of affairs commanded or asserted.  Together, such components might be exemplified by the imperative ‘put the cat on the mat’, which is composed of the mode of expression – a command – and the state of affairs envisaged as the object to the command, the cat’s being on the mat.   Thus even an utterance that the cat is on the mat carries a mode of expression – the stating of a fact for a purpose plus the bare state of affairs concerning the cat; perhaps thereby informing someone of that fact to some end.  Even though this assertion “The cat is on the mat.” seems indistinguishable from an exposition of its content (i.e., the propositional component) it is, nevertheless, not merely representational of a state of affairs.  It is a speech act, and comes with the intention of having an effect.  Speech Act theory claims, as the name suggests, that speech should be understood as another form of action.

Figure 2 CM & the Speech Act

These components then can be described in terms of the content or state of affairs – the cat being on the mat (unasserted), and the exercise of control – some communicative act over the content or state of affairs: in other words, an analogue of the coordination mechanism as described by Shaw & Fox [14] (see Fig.2).  In the first case – the statement of a fact – the exercise of control is in the form of an assertion of the state of affairs expressed in the proposition, the cat’s being on the mat, and can be true or false; in the second, it might be in the form of a command that the state of affairs should exist and that someone should effect it, or, say, in the form of a warning, recommending some urgent response.  Both may be tasks to be effected or requirements to be fulfilled.

We can recall the injunction by Dowell & Long [9]: “If the worksystem is well adapted to its domain, it will reflect the goals, regularities and complexities in the domain”.  This expresses the rationale for uncovering links between the complex of tasks and the structures of the worksystem.  If we take another example of an utterance in the form of a warning “The bull is charging”: one of the component of the CM (enhanced variety of coordination mechanism) is determined by perceived goals (Do we try to identify the breed of the bull, wonder at its strength, or run for our life?); the other refers to the state of affairs (The bull is at A and charging towards B, and you are at B)

Two things should be emphasised, and the one is linked to the other.  The first is that these utterances are expressed for a purpose in the normal way of things.  A statement of facts is made, for example, to satisfy someone’s curiosity, perhaps allowing him or her to make or modify some plan of action: to use the vacuum cleaner now or later.  A command, or a warning are perhaps more patently also links in a chain of actions.  They too are actions, and they are goal-oriented.  The second thing which needs to be taken note of is that, as such, they have to be coordinated, like other group actions, for communication to take place at all; and that, as Clark [18] puts it, “language use and joint activity are inseparable”.

So, what we are suggesting is that it may make sense to suppose a more fundamental structure – the CM – which is goal-oriented, might underpin models such as distributed cognition, may support all of what Clark means by ‘joint activity’ (including the use of language) and comprise coordination in CSCW, i.e., working or behaving together effectively.

The difference alluded to above between the artificially intelligent agents and their human counterparts is that, in the case of the AI agents in [14] “each problem-solving agent only possesses local view”.  By contrast, human agents often share information knowledge publicly and transparently, and necessarily share knowledge of conventions and values, as a basis for mutual understanding: jointly, these allow them to coordinate – including what is required for communication – on the basis of recognising objective states of affairs and of employing mutually understood means of controlling them.

Thus, to bring together the observations of Hutchins and Schmidt: cognition is already distributed, at least to some degree, for conceptualisation to take place, and the CM is the formal structure which can account for this distribution in the familiar terms of goal-oriented acts; at the same time, it brings together the terminology and conceptual apparatus of distributed cognition (DC) and distributed artificial intelligence (DAI). (Figure 3)

Figure 3 DC+DAI=CM for Language & other Joint Activity

To  distinguish this CM from the speech act structure we shall refer to the element which exercises ‘control’ (in Shaw & Fox terminology [14]) as the ‘attitude’; and the object, data or representation of a state of affairs (designated ‘information’ by Shaw & Fox [14]) will be referred to as the ‘content’.  In order to characterise the mutual knowledge underlying this fundamental communicative process, which supports both social behaviour and linguistic communication, we can adapt a famous account of non-natural meaning (in this case, the understanding of the intentions of others) by Grice [19] as follows: he writes, “‘The speaker meant something by x’ is (roughly) equivalent to the speaker intended the utterance of x to produce some effect in an audience by means of the recognition of this intention”.  If we substitute ‘speaker’, ‘utterance’ and ‘audience’ for ‘actor/agent’, ‘execution’, and ‘spectator’, respectively, we get: The actor/agent meant something by x is (roughly) equivalent to the actor/agent intended the execution of x to produce some effect in a spectator by means of the recognition of this intention.  We can see, therefore, how it might be possible to embrace indifferently both language and more general social activity with the more fundamental and general concept of the CM as an expression of a mutually understood state of affairs or ‘content’ governed by a meaningful ‘attitude’ in language, or in behaviour more generally.

5. Operationalisation of the CM

The foregoing is the case for employing a particular conceptual framework in certain situations of design involving group work.

• The employment itself of such a conceptual framework in these specific situations is what we call its operationalisation – with its feedback and feed-forward as suggested above.
• Series of such operationalisations to solve design problems amount to a validation of the concepts.

However, this is not a scientific endeavour and we are not trying to prove how things are but rather we are bringing frameworks and processes to bear on design problems, and if they have utility then to that extent they realise their meaning, and their value.  If a conceptual framework, for example, assists the language of design allowing people in the field to communicate their ideas more easily, then this amounts to some success.  The operationalisation, and a series of the same, amount to the engineering practices, which will constitute cognitive engineering knowledge.

We are arguing that the conceptual language currently at our disposal for cognitive engineering is inadequate for the purpose of tackling the design problems of group behaviour, and are attempting to correct this state of affairs, while maintaining this approach within a cogent discipline framework.  The conceptual framework we offer should be considered as ‘early design knowledge’ which is in need of the operationalisation which results from addressing a particular design problem.  At the current stage of the development of these ideas even operationalisation, strictly speaking, is problematic.

6. Conclusion

What we have tried to do is provide a way of understanding how agents might be conceived as working together in some domain or other, being equal members of an interactive worksystem, maintaining their independence but sharing their understanding of the task and working effectively to perform it.  The CM is part of the structure which underlies collaborative activity, and allows the designer to specify the design problem of such collaboration for effective work by identifying the attitudes and contents of the CM.  It permits a designer, for example to analyse dialogue and action into descriptive items reflecting states of affairs in the Domain and normative items reflecting goals of the work and intentions of agents in the IWS.  It does this without resulting in a reductive account, and is consistent, we believe, with the ontology and related methodology of the original HCI Conception.

References

[1] K. Schmidt, Cooperative Work: a Conceptual Framework, J. Rasmussen, B. Brehmer & J. Leplat (Eds.), Distributed Decision Making. England: J Wiley & Sons 1991.

[2] P. Falzon, Cooperative Dialogues, in: J. Rasmussen, B. Brehmer & J. Leplat, Distributed Decision Making J Wiley & Sons, 1991.

[3] Bødker, S. (1998). Understanding Representation in Design. Human-Computer Interaction, 13, 107-205

[4] B. Brehmer, Some Notes on the Literature, in: J. Rasmussen, B. Brehmer, J. Leplat (Eds.), Distributed Decision Making, J. Wiley & Sons, England, 1991 pp. 3-14.

[5] Storrs, G. A Conceptualisation of Multiparty Interaction., Interacting with Computers, 6, ii, (1994) 173-189.

[6] J. Rasmussen, Modelling Distributed Decision Making, in: Distributed Decision Making J. Rasmussen, B. Brehmer J. Leplat, England: J. Wiley & Sons, 1991.

[7] K. Vicente, Cognitive Work Analysis: Toward Safe, Productive, Healthy Computer-based Work, Lawrence Erlbaum Associates, 1999.

[8] J. Dowell, and J. Long, Towards a Conception for an Engineering Discipline of Human Factors. Ergonomics, 32, 11(1989) 1513-1535.

[9] J. Dowell, and J. Long, Conception of the Cognitive Engineering Design Problem (Peer Review Paper), Ergonomics, 41, 2 (1998) 126-139.

[10] H. A. Simon, The Sciences of the Artificial MIT Press, 1969.

[11] U. Neisser, From Direct Perception to Conceptual Structure, in: U. Neisser, Concepts and Conceptual Developments: Ecological and Intellectual Factors in Categorisatio, England: Cambridge University Press, 1987.

[12] K. Popper, Conjectures and Refutations: the Growth of Scientific Knowledge, Routledge, London & New York, 1963.

[13] S. W. Draper, Review of Designing Interaction: Psychology at the Human-Computer Interface, in: EACE News, Views and Reports, Int. Journal of Man-Machine Studies, 37 (1992) 812-821

[14] M. J. Shaw, M. S. Fox, Distributed artificial intelligence for group decision support: integration of problem solving, coordination & learning, Decision Support Systems, 9, , (1993) 349 – 367.

[15] E. Hutchins, The technology of team navigation, in: Galegher, Kraut & Egido (Eds.), Intellectual Teamwork, Lawrence Erlbaum, Hillsdale, N.J., 1990, pp. 191-220.

[16] J. L. Austin, How to Do Thing with Words. Oxford University Press, 1962.

[17] J. Searle, Speech Acts, Cambridge University Press, 1969.

[18] H. H. Clark, Using Language, England, Cambridge University Press, 1996.

[19] Grice, H. P. (1959). Meaning. Philosophical Review, 66, 377-388