Wednesday, 22 September 2021
Giving ‘don’t know’, ‘not sure’ or similar responses on questionnaires and surveys could be overcome by a new method of gathering data that captures valuable information about uncertainty and flexibility.
Researchers from the University of Nottingham have been working to address uncertainty in questionnaires across a variety of problems with support from the UK EPSRC and the National Cyber Security Centre (NCSC). They have developed an interval-valued response method as an alternative to the commonly used questionnaire approaches. Their research ‘Capturing richer information: On establishing the validity of an interval-valued survey response mode’ has been published in Behavior Research Methods.
Filling out surveys, questionnaires and reviews is a part of everyday life. These are used as a way of collecting feedback and data for a range of applications, but current models are inflexible in the responses they can collect, often using scales of 1-5, likely to unlikely, or agree to disagree, where the respondent can only choose one score.
Christian Wagner, Professor of Computer Science, has been leading the research. He explains: “Current methods for collecting quantitative or qualitative data each have their own limitations and we are exploring the viability of alternatives designed to maximise data capture for effort invested. Specifically, we have focused on an issue we believe has received relatively little attention given its potential significance – the efficient capture of uncertainty as perceived by an individual respondent. We have all been faced with a conventional scale and been unsure which option to choose, whether the question is badly worded or ambiguous, or whether we don’t know the exact answer or indeed feel that multiple options apply. Our research offers an alternative response system that takes this uncertainty into account, providing better quality data for little to no added effort. This has a lot of benefits, and it could even be used to improve data privacy in the future.”
The new method uses ellipses to allow respondents to choose a range when answering a question, in an intuitive and accessible way. So instead of choosing one discrete point along the scale, the respondent can draw an interval around all options that apply. The team have tested these interval-valued questionnaires in a variety of areas; from vulnerability assessments in cyber security, to consumer perceptions of products, patient understanding in healthcare, and environmental management.
Data is used to inform decision making across a wide range of areas, and our research shows that there is an efficient way to capture better quality data without requiring much more time or money – and that people can effectively use this approach to communicate their uncertainty.
The research is led by the Lab for Uncertainty in Data and Decision Making (LUCID), at the University of Nottingham’s School of Computer Science, and is an inter-disciplinary collaboration with partners including Carnegie Mellon University’s School of Social and Decision Sciences.
his most recent output really highlights the potential of cross-disciplinary research, linking both Computer Science and Psychology in this case. In parallel to ongoing research focussing on the analysis of the resulting data, we are already working with stakeholders in other sectors, such as consumer goods manufacturers to apply this to their business and we are keen to expand into other areas to explore how beneficial this method can be.
Story credits
More information is available from Christian Wagner and Zack Ellerby at the University of Nottingham on christian.wagner@nottingham.ac.uk, zack.ellerby@nottingham.ac.uk
Notes to editors:
About the University of Nottingham
Ranked 32 in Europe and 16th in the UK by the QS World University Rankings: Europe 2024, the University of Nottingham is a founding member of the Russell Group of research-intensive universities. Studying at the University of Nottingham is a life-changing experience, and we pride ourselves on unlocking the potential of our students. We have a pioneering spirit, expressed in the vision of our founder Sir Jesse Boot, which has seen us lead the way in establishing campuses in China and Malaysia - part of a globally connected network of education, research and industrial engagement.
Nottingham was crowned Sports University of the Year by The Times and Sunday Times Good University Guide 2024 – the third time it has been given the honour since 2018 – and by the Daily Mail University Guide 2024.
The university is among the best universities in the UK for the strength of our research, positioned seventh for research power in the UK according to REF 2021. The birthplace of discoveries such as MRI and ibuprofen, our innovations transform lives and tackle global problems such as sustainable food supplies, ending modern slavery, developing greener transport, and reducing reliance on fossil fuels.
The university is a major employer and industry partner - locally and globally - and our graduates are the second most targeted by the UK's top employers, according to The Graduate Market in 2022 report by High Fliers Research.
We lead the Universities for Nottingham initiative, in partnership with Nottingham Trent University, a pioneering collaboration between the city’s two world-class institutions to improve levels of prosperity, opportunity, sustainability, health and wellbeing for residents in the city and region we are proud to call home.
More news…