Stuart will present his recent work on studying usability testing to the lab.
How UX practitioners Produce ‘Findings’ in Usability Testing
Usability testing (often called ‘user testing’, or just ‘testing’) is routinely employed in the technology industry to shape product, service, and system design outcomes. Forms of testing include classical ‘lab’ setups, where stakeholders watch participants performing tasks with an artefact under test (e.g., prototypes), through to remotely administered tests, or ‘rapid iterative’ variants (RITE) where the artefact is fixed ‘on the fly’ in response to findings. Curiously, although evaluation methods like usability testing have long been a core interest of academic HCI research, (UX) practitioners’ appropriations of such methods have largely been overlooked or treated as an impoverished relation rather than a distinct domain of practical action. Beyond HCI, usability labs have received little attention (except Woolgar (1991)).
I present an ethnomethodological study that starts to uncover how ‘findings’ from usability testing are produced as a matter of practitioners’ work—i.e., how the shaping of the object (finding) becomes an “increasingly definite thing” (Garfinkel, Lynch & Livingston 1981). This study seeks to articulate what constitutes usability testing’s practical ‘production work’ and thus begin to nuance conventional views of usability findings as straightforwardly artefact-intrinsic, i.e., ‘there to be found’ and ‘read off’ by competent evaluators. In order to do this, and informed by a broader ethnography, I use video recordings of usability labs at a UX design consultancy where various stakeholders and participants engage in usability tests with a prototype website. As exhibits of this production work, I explore how stakeholders collaboratively locate ‘troubles’ in the test’s unfolding, and then surface these as candidate findings, some of which ultimately may be still be ‘passed over’ and dissipated in a range of ways. As part of this I unpack how troubles and solutions are formulated as a matter of this production work, and how this turns on, variously, the topicalisation of prototyping itself, the organisation of test protocol, and the contingencies brought to this work by stakeholders’ particular orientations.
Although there is considerable ethnomethodological tradition based in exploring the work of scientific laboratory settings (Lynch, 1993) as sites of “discovering work” (Sormani, González-Martínez & Bovet, 2011), the usability ‘lab’ presents a conceptually distinct setting where artefacts come to be constituted very differently. The implications of the study also suggest respecifications of current understandings of usability testing in HCI and its conceptualisation of UX practitioners’ work.