The development of a questionnaire to evaluate a DVD examination for pre-registration student nurses
Background:
QAA emphasise the responsibility that higher education institutions have to design, approve, supervise and review assessments in order to maintain consistent and rigorous assessment practice. Previously in the DN3 Module of the Diploma of Nursing, the Objective Structured Clinical Examination (OSCE) incorporated a test of students' communication skills in assessment. This examination required students to assess/communicate and deliver care to a member of staff posing as a patient/client. Whilst generally successful, a number of issues were raised by staff through curriculum meetings and students through SEMs and Student representatives e.g. non-standardisation across school, the potential distraction of using teaching staff to play the role of patient. Academics within the school developed a new DVD examination in order to remove these potential biases. This new assessment requires students to watch a DVD of a nursing scenario and answer questions based it. The aim of this study was to evaluate the students' experience of this new exam through the development of a tailored questionnaire.
Method:
A likert type scale was developed via a student focus group (n=8) to engage students in proposing items to be included in the questionnaire and provide an in-depth understanding of students attitudes towards the experience of the DVD exam. Following this, the content and wording of the questionnaire were scrutinised by members of the Communication Special Interest Group (COMSIG) to make sure that all the information the school would need about the student experience could be encapsulated though the administration of the questionnaire. The questionnaire was hand delivered to 603 students during Nursing Concepts and Skills for Practice modules across the 5 school of nursing centres (Derby, Nottingham, Mansfield, Lincoln) and returned via module leaders for analysis.
Results:
222 completed questionnaires were returned for analysis. Principal components analysis (PCA) was used to explore the inter-relationship between the variables on the questionnaire. PCA revealed the presence of two subscale components being ‘practical aspects of the examination' and ‘personal experience of the examination'. Content validity was established through the detailed examination of the questionnaire by a number of experts and reliability in the form of internal consistency demonstrated a Cronbach's alpha coefficient of 0.901. Therefore the evaluation questionnaire is a psychometrically sound tool that reliably used to measure student satisfaction of the video trigger examination.
Exploratory analysis of student responses gathered via the questionnaire revealed a mean score of 2.47 and a standard deviation of 0.56. Students were asked to rate their experiences on a scale of 1-5 where 1 is positive, 5 is negative and 3 is neutral. Therefore this result suggests that the students who completed the questionnaire were positive about the DVD examination in which they took part.
Conclusion:
A number of recommendations arise from the study including providing additional preparation in the form of group tutorials and printed handouts to better prepare students for the examination. Given that this evaluation was for one cohort, further surveys need to be conducted in order to continue to refine the examination and improve the student experience. As the questionnaire is psychometrically sound, it can now be used to audit this assessment and provide ongoing information in relation to student satisfaction.