Staff Survey on Online Assessment

Use of online summative assessment has increased rapidly over the past five years. In the previous academic year (2018-19) the Faculty ran 90 online exams (53 in the exam period, 37 mid semester). This activity involved over 13,000 student sittings (N.B. each student may have taken more than one online exam in this period).

In order to better understand the perception of this activity and to monitor the service that the eLearning team provides, all 55 academics who had run an online exam in the past two years were invited to complete a short survey. Thirty-five responses were received (a 63.64% response rate).

Survey Results

Experience of Online Assessments

63% of participants said they had run online assessments “many times” over several years, 17% said that academic year 2018-19 was their first time using online assessment. The remaining 20% said they had run online assessments “a couple of times” over several years.

77% of participants had run an online exam in the formal exam period, and 54% had experience of running online exams during the teaching semester.

77% of Participants had run a Blackboard based online exam. 23% have run a Hybrid exam (part online – part paper). 29% of participants have run a Software based online exam.

The Blackboard assessment tool?

It is encouraging to see that the vast majority of academic staff have some confidence in creating questions for Blackboard assessments. The eLearning team is looking at ways we can further support this. We would encourage any staff who have questions about creating online questions to contact the eLearning team directly. Several staff added comments within the survey and we have been able to address these issues directly – we appreciate that some aspects of Blackboard are not obvious so please do ask. One responder said they would like to receive further training and support but completed the survey anonymously.

It’s also good to note that two-thirds of FSE academics using online assessment feel that the Blackboard tool set is adequate for their needs. The survey highlighted several common features that academics would like to see implemented in Blackboard:

  • Better functionality to Import/Export online Exams from/into Word/Excel documents – we are currently working on a process where import via Word is possible
  • Better functionality for equations – ability to cut and paste from Word, or import Latex – there is some support for Latex and MathML within Blackboard.  If you would like to know more, please get in touch
  • Ability for students to draw diagrams/graphs/schematics within question answers – This is a very common request; we have passed this onto Blackboard
  • Set questions with sub questions – Another common request that we have raised with Blackboard

Blackboard Analytics

 

The eLearning team has been generating Exam Reports after online exams. These reports analyse student performance on a question-by-question basis, and can be a useful guide to how effective your assessment has been. Exam reports are created by the Item Analysis function in Blackboard; any academic can run such analyses on their tests. The eLearning Team has started sending the reports to academic leads to promote this function. We know that some academics already use this functionality. Some departments in our Faculty use these reports to demonstrate good assessment practice to external moderators,  so we feel they are being used well.

Effective and Efficient ways of assessing Students?

Very encouraging to see that over 90% of staff using online assessment see it as an effective and efficient way to assess students. Which probably explains the sharp increase in uptake of online assessments across the faculty.

Confidence in the Online Assessment Processes

 

 

While it is clear that the majority of academic staff have confidence in the services supporting online assessment, these results show that more work can be done in this area. The eLearning team, relevant colleagues from IT Services, and the Exams office, have been working very hard in the past 12 months to improve processes to make better use of the resources and expertise available to support this activity. As we all know, the University is going through a period of considerable change and there are plenty of challenges ahead, but we are working very hard to ensure that online assessment is successful from the point of view of academics and support staff, and that we deliver the best student experience possible.

There is a concern that 10% of academic staff do not have confidence in the eLearning team to support online exams. The survey respondents who expressed this lack of confidence had completed the survey anonymously.  I would welcome further details in this regard,  so that we can work to mitigate the perceived problems. If anyone would like to send me further information, please email me directly: Daniel.jagger@manchester.ac.uk

Results of Staff Survey on Online Assessment
Tagged on: