Feasibility study of implementing e-assessment: lessons to be learned
John Andresen (Department of Chemical & Environmental Engineering), Dragos Axinte (Department of Mechanical, Materials & Manufacturing Engineering), David Hann (Department of Mechanical, Materials & Manufacturing Engineering), Mark Haw (Department of Chemical & Environmental Engineering), Karen Steel (Department of Chemical & Environmental Engineering), Katy Voisey (Department of Mechanical, Materials & Manufacturing Engineering).
The project evaluates the suitability of existing e-teaching environments to fulfil students’ learning needs, academics’ requirement to handle versatile “teaching/ assessment tools” and “sine qua non” constraints in teaching engineering subjects.
Dealing with “precise quantities” the engineering subjects are, theoretically, suitable for employing e-assessment/learning methods. E-assessment automates marking, hence reducing the marking load on academics. Moreover, e-assessment enables monitoring of large numbers of students while providing academics with necessary feedback to evaluate students’ progress and their needs for attaining better module understanding.
To achieve the project goals the following aspects have been considered: literature survey; online preliminary surveys on student/staff’s previous experience with e-assessment/learning; implementation of e-assessment in two academic modules; statistical analysis of the results; conclusions and recommendations.
The literature shows that combinations of e-assessment methods (e.g. multiple choice/matching questions, peer e-assessment) should be employed; transcribing the existing questions into e-assessment systems should be avoided. Additionally, security (e.g. digital certificates) and technical (e.g. diagrammatic representations) issues have to be addressed. Moreover, uncertainties in total compatibilities with the future e-learning environments were also considered as limiting factors in adopting these methods. IS upfront support to modify and implement e-learning platforms to fit engineering curricula would be necessary.
The preliminary online surveys within the Schools showed that both students (>60%) and academics (>80%) feel that e-learning tools would be overall beneficial. Dissatisfactions with the current environments were expressed by students (25%) while academics predicted difficulties with software, responsive technical support and plagiarism avoidance. A staff “wish list” of any future e-learning package would include: randomisation (73%); input of equations by students (67%); “responsive” guidance using students’ previous answers (87%).
To assess the feasibility of introducing e-assessment in engineering, such methods have been included in one module from each School. While students showed interest in e-assessments, little use was made of them prior the exams. Setting questions in present e-environment was time consuming unless simple questions would be set. Elements of engineering practice (diagrams, equations) were not possible. Linking and randomising the questions was difficult while significant work was necessary to “translate” past exam questions into e-assessment environment. Moreover, it seemed that students could be distracted by other items on the computer (internet, e-mails). In one case, the two systems, i.e. “classical” tutorial sheets with numerical answers and e-assessments (WebCT based), were run successively over two academic semesters. It was found that 77% of students preferred the old system because it allowed them to work at their own pace. However, 98% of the students said they would use the ‘new system’ if their marks amounted to 10% of their final mark.
The study revealed that: due to their technical limitations the existing e-environments are difficult to be used for e-assessment in engineering areas; facilities to enable easy diagrammatic representations, drawing/ sketching, formula writing, question linkage and randomisation should be developed; the existing e-environments are more suitable for multiple-choice type questions which might be applicable for e-assessment in more “soft-engineering” subjects.