Generalising from Novices to Experts in Military Studies
Abstract
This report completes a suite of papers delivered during 2006 that were concerned with design issues as they related to so-called ‘Command Wall systems’. This report is about whether the results gained from novices in previous studies can be generalised to an expert (i.e. military) population.
A considerable corpus of Human Factors research, military and otherwise, is conducted on novices despite their findings being generalised to expert user groups. This report deals with the extent to which novices are ‘different’ or ‘similar’ to experts, with particular regard to Command Planning studies.
This study was based on the same Command Planning task(s) used in previous research (HFI DTC 2006, a & b). It uses two groups of participants: The first group was selected from a cohort of Brunel University undergraduates (matching the profile of participants in the previous studies). The second group were serving military personnel based at the British Army’s Land Warfare Training Centre in Warminster. A simplified command planning task, based on the Combat Estimate, was undertaken by both groups. Psychological variables such as situational awareness (SA) and mental workload, as well as task performance, were measured.
Whilst experts and novices are clearly not identical there appears to be enough commonality (with some important caveats in place) for them to be used interchangeably in Command Planning studies. The findings show:
- Novices may be somewhat quicker than experts but experts are more accurate.
- Differences in the ‘quantity’ or ‘extent’ of probe recall performance and workload are in evidence (experts and novices differ in the relative levels of their measured performance) yet within that the ‘pattern’ or ‘type’ of results obtained is broadly concordant (despite differences in the level of performance, the pattern of findings between experts and novices is actually similar), however,
- the critical difference appears to lie in structural and elemental differences in the participant’s situation model (which we would expect to differ between experts and novices).
We conclude that novices can be used to good effect in this experimental context but it is important to be alert to critical differences that do exist between the two groups (e.g. Situational Awareness content). We recommend that this issue is revisited periodically should the context to which experts and novice participants are used change.
Although experts are frequently interchanged for novices in military research the potential exists for differences to be so profound that the results are rendered virtually meaningless. An important safeguard against this eventuality (and risk) is to conduct analyses such as this and to continue to do so when warranted.
contact us