Evaluation of existing Human Factors tools and methods

What is this report about?
A HF methods review was conducted in order to identify the most suitable methods for use during the C4i design lifecycle. Initially, a literature review was conducted in order to create a comprehensive database of existing HF methodologies. The literature review was based upon a survey of standard ergonomics textbooks, relevant scientific journals and existing HF method reviews. The result of this initial literature review was a database of over 200 HF methods and techniques.

What problem does the report address?
The methods review highlighted a distinct lack of effective integration of HF methods. Further, in order to ensure comprehensiveness, a need for an integrated approach to be used in the analysis of C4i systems was identified. Consequently, the HF methods matrix was constructed. The methods matrix was constructed in order to allow practitioners to identify possible combinations of methods to suit specific analysis needs. The matrix was also designed to inform the HF practitioner which methods act as an input into one another. As such, the matrix specifies which of the methods can be used effectively in conjunction with one another (i.e. which methods can be integrated) and also which of the methods act as input into other methods.

What is the benefit of this work?
The integration of Human Factors (HF) methods in this way is an attractive prospect that offers a number of advantages. Firstly, the analysis becomes much more comprehensive as multiple aspects of the system or task under analysis can be analysed, such as operator activity, workload, situation awareness, error, and performance. Secondly, the analysis becomes more exhaustive as each of the integrated methods act as back up checks for one another. Any aspect of system performance not identified (or missed) by one method may be picked up by another method within the framework.

Who should take note of it?
Anyone interested in Human Factors design and assessment methods

What is the report's status?
Completed

What are the main issues addressed in the report?
HF techniques were subjected to a review. A screening process was employed in order to remove any unsuitable methods. Methods were classed as unsuitable if they were unavailable, software based, inapplicable, or if they were very similar to another method already selected for review. As a result of the method screening procedure, 96 HF design and evaluation methods were selected for the methods review. Each method was then analysed using a set of criteria designed not only to establish which of the techniques were the most suitable for use in the design and evaluation of C4i systems, but also to aid the HF practitioner in the selection and use of the appropriate method(s).

What are the findings?
The main conclusions were:

• Identification of methods that could be used in combination
• Identification of methods that served as the input to other methods
• EAST methodology for analysis of C2, C3, and C4 activity

What is recommended?
The event analysis of systemic teamwork (EAST) methodology ( Baber and Stanton 2004) is an example of an integrated HF methods approach extracted from the HF methods matrix. EAST was developed for the analysis of C4i activity. EAST uses a combination of methods to form a framework for analysing C4i activity, including hierarchical task analysis (Annett 2004), co-ordination demands analysis (Burke 2004), communications usage diagram (Watts and Monk 2000), social network analysis (Driskell and Mullen 2004), operation sequence diagrams, critical decision method (Klein and Armstrong 2004) and propositional networks.

Why bother?
EAST has been applied in a number of domains to describe command and control activity, including the fire service (Baber et al 2004), naval warfare (Stewart et al, 2005), aviation (Stewart et al, 2004), energy distribution (Salmon et al 2004), air traffic control (Walker et al 2005) and rail (Gibson et al 2005) domains.