Important information for the academic session Staff guide to the use of DMU core technologies for anonymous marking in eAssessment Turnitin Guidance for Students for Handbooks

Guidelines for e-Assessment @ DMU

1. Overview

These guidelines are designed to support staff in the process of developing on-line assessments, or e-assessments, and should be implemented with the guidance of Enhancing Learning through Technology (ELT) Project Officers, as appropriate to the curriculum and programme context.

e-Assessment “is defined as the use of electronic systems for the development, operation and delivery of accredited qualification assessment or the collection of performance evidence, which contributes to the awarding of a unit or an accredited qualification” (QCA, 2007, p. 5). These systems may include: Blackboard tests; Blackboard discussion boards, blogs or wikis; multimedia tools; Turnitin; and external, web-based tools, such as blogs or wikis, used to prepare portfolios. In embedding these systems, the use of e-assessment has technical, administrative and operational implications.

This document sets out guidelines for the use of e-assessment at DMU. All staff involved in the design, presentation, setup and invigilation of diagnostic, formative and summative e-assessment are advised to adopt them. Please note that these guidelines should be read alongside the University Assessment Policy and Procedures, which highlight that assessment should be: transparent; equitable; valid; reliable; just; and sustainable.

2. Validity and Reliability

• Programme teams should ensure that e-assessment produces a valid and reliable measure of a candidate’s achievement of specific learning outcomes. The choice of assessment method should be independent of the technology on which it may be based.
• Questions should be carefully designed with due regard for the material being assessed and the level-of-study.
• Where on-line, summative assessment is planned, there should be formative opportunities for students to understand both the process and content of this type of assessment, and gain the appropriate feedback.
• Students should be made aware of: the number and type of questions to be used; the scoring logic for individual items and the overall assessment; the contribution of this assessment to the overall module; logistical issues like timings, restraints on navigation, and permitted resources; and, any personal identification they will require.

3. Technology/Security

• Only DMU hosted technologies should be used for the delivery of online tests. If staff have the need to use a non-DMU hosted technology to deliver an online test this must be discussed in advance with their Faculty Learning Technologist. The use of non-DMU technology will need to be formally approved by the institution and this process may take a significant amount of time.
• Where students are sitting phase tests either on- or off-campus, programme teams should ensure:
  • appropriate security against plagiarism, collusion or copying;
  • enough lab capacity to cope with hardware or software failures (10% spare capacity is recommended). Timetabling enough time and space for assessments in labs is crucial; and
  • that on-line, assessed work can be properly attributed to individual students.
• These same issues apply where programme teams are assessing students based upon outcomes produced through non-DMU, web-based technologies.
• Where students are sitting phase tests either on- or off-campus, programme teams should consult with Faculty ELT POs and ITMS, to assure themselves of the robustness of the local technology that supports an assessment.
• On the use of Turnitin on mobile technologies:
  • Given that sensitive data related to student assessment/records are stored on or accessible from devices like iPads, the turnitin app should not be installed on personal devices and used to mark student work as this use may compromise GDPR.
  • the iPad application for Turnitin was enabled by iParadigms, who manage the Turnitin application. It cannot be adapted in any way by ITMS.
• Teams must address the secure transfer of work to external examiners. Access to DMU systems, like Blackboard and Turnitin, can be arranged to give secure access for external examiners to module-level work for moderation purposes.
• Where scores or grades are communicated to students, the Blackboard Grade Center provides secure, personalised access to indicative marks. MyDMU provides secure, personalised access to validated marks.
• Where programme teams are assessing students based upon outcomes produced through ‘approved’ non-DMU, web-based technologies they should:
  • induct students into the mechanics of the specific tool, and how it will be used for assessment;
  • clarify issues around plagiarism, collusion, IPR and copyright with their students; and

4. Operation of Assessment

• Anonymous Marking (AM) for all summatively-assessed coursework is the norm across DMU. This can be managed via a paper-based cover-sheet or curriculum teams may wish to opt-out of AM after talking to the Chair of their PMB. There is no requirement for staff to manage AM through e-assessment. However, there are three on-line solutions available for staff, which are detailed at: https://celt.our.dmu.ac.uk/anonymous-marking-for-eassessment-the-staff-guide
• Both Blackboard and online tools such as blogs and wikis, can engage students in the assessment process though personal ownership of feedback, and self-and peer-assessment. However, clear links should be made between assessment modes (diagnostic, formative and summative), the technologies that support them and expectations of students.
• Where staff use Blackboard or an approved non-DMU, web-hosted application to deliver summative assessment, the following issues should be addressed for both on- and off-campus students.
  • Students should be given opportunities to familiarise themselves with the interface to be used.
  • The format of the test should be clearly explained to students, with a specific focus upon how to save answers, restrictions surrounding the test (time, multiple attempts etc.), and how to submit.
  • Deployment of tests using assessment pools, and the randomisation of questions during tests, may reduce the risk of collusion, but this increases the need for staff to be assured that all questions, and the order in which they appear to students, are of a comparable level.
  •  When using Blackboard to deliver online tests, browser windows cannot be locked down or web-access restricted, students should be made aware of the penalties for searching for answers in closed-book exams. Appropriate strategies for invigilation and ensuring that exam conditions are maintained are critical. The invigilation team should include a senior academic with knowledge of both the assessment and the assessment software.
  • The time limit allowed for the examination may need to be flexible to compensate for any delays (technical, emergency, etc). Additional time may be permitted for some students, e.g. those using assistive technology. Therefore, IT suites should normally be booked for 30 minutes before an assessment and at least 15 minutes after the last assessment is scheduled to be finished.
• An identical assessment may be run in sequence to make best use of facilities and staff availability. In this instance, candidates in the first session are not permitted to leave the examination room, until everyone has finished and the next group moves into the room while the initial group(s) leave(s).
• Programme teams should develop a contingency in the event that either DMU or external, web-based e-assessment systems fail at the point of delivery. This may include, for instance, postponement or paper-based delivery. For summative assessment this should be agreed with the Faculty Head of Quality
• Programme teams should explain to students the procedures for managing late e-submission of work, where e-submission is the sole format or is being used alongside paper submission. Teams will also need to have a clear procedure for managing submission where technical systems fail (see Appendix 1).
• As soon as possible after a CAA assessment has finished (within 60 minutes), a copy of the results must be made and transferred to a second, secure hard-drive.
• Where collaborative partners are involved in e-assessment, programme teams should provide their staff with clear guidance on format, operation and IT requirements.
• Where distance learning students are in the process of applying and enrolling on DMU programmes, they should be made aware of the minimum hardware, software and connectivity requirements for their assessment.
• Where e-portfolios are being developed, programme teams should ensure that students can save evidence in a variety of media, in a secure, personalised system. For instance, this might include the Blackboard-based wiki or blog. Note that where evidence is stored on Blackboard, it will be backed-up and maintained for a maximum of 6 years, in-line with the University’s regulation on programme enrolment.
• Online summative assessments should be managed via secure hardware, rather than mobile devices. In particular, taking summative assessments on tablets over a wireless connection is high-risk and should be avoided. Where staff wish to make formative assessments available to students using their own technology the following apply.
• Mobile tests are available to students on iOS and Android devices. However not all operating systems on these devices may be compatible. Students should not be forced to take a test on their own personal mobile device.
  • Timed tests are not available via mobile devices
  • Mobile tests are not available via mobile web browsers, e.g. Safari on the iPad
  • Any test that contains non-mobile-compatible question types will not be available for students to take via mobile devices.
  • Tests that are not compatible with mobile devices will appear in the course map with a message directing students to take the test through a web browser.

5. Electronic submission and digital receipting

• Where staff use Blackboard-Turnitin for electronic submission the following options are available:
  • Assignment: where users “view/complete” an assignment, an electronic receipt is presented and the submission is stamped as “completed” in the assignment window. This is not date/time stamped;
  • Email submission through Blackboard generates a date/time-stamped receipt, which can be printed; and
  • Turnitin: where users “view/complete” a TII assignment, an email receipt is presented and the submission is date -stamped in the relevant TII window.
• Where teams use non-DMU, web-hosted applications to prepare for or deliver assessment, submission and receipting procedures should be clearly explained to students.

6. Anonymous submission

• Anonymous Marking (AM) for all summatively-assessed coursework is the norm across DMU. This can be managed via a paper-based cover-sheet or curriculum teams may wish to opt-out of AM after talking to the Chair of their PMB. There is no requirement for staff to manage AM through e-assessment. However, there are three main on-line solutions available for staff: Turnitin, Blackboard Assignments or Blackboard Tests, which are detailed at: https://celt.our.dmu.ac.uk/anonymous-marking-for-eassessment-the-staff-guide
• Anonymity at the point of marking scripts can only be provided where e-submissions are made to a third party who is not involved in marking. It is not possible at present for Blackboard, TII or email to provide true anonymity although, submission using p-numbers alone is possible.

7. Accessibility

• Programme teams must have policies and procedures in place to ensure that disabled learners are not treated less favourably than non-disabled learners when implementing e-assessment, in terms of its format, operation and IT requirements. Faculty Disability Co-ordinators and Student and Academic Services are key sources of information on this issue.
• Where appropriate, and when requested, a non-computer based assessment may be made available to a disabled candidate.

8. Health and safety

• PCs, chairs and the general assessment environment must conform to standard safe environmental guidelines (clear screen, adjustable screen, adjustable chair, good lighting and ventilation).
• Candidates should be advised to take ‘micro-breaks’ to relax their muscles and eyes i.e. stretch, look up at the ceiling.
• If the assessment lasts for longer than 90 minutes, then there must be a provision for a break away from the PC. This may require supervision of candidates during the break, although facilities such as the provision of separate tests or blocks of questions may be used.

9. Further guidance

Staff seeking further guidance or professional development should contact their Faculty ELTPO.

Options if Turnitin fails

The following options are available.

1. Give extended time for resubmission – this is at the prerogative of the tutors
2. Paper based submissions (also enables anonymous marking)
3. Blackboard assignments (also enables anonymous marking)
4. Individual wiki/blogs or Blackboard Journals could be utilised to provide a private dialog for submission/feedback (does not enable anonymous marking)
5. Blackboard tests: this will only work if question settings are set to allow ‘Add images, files, and web links to answers’ (in theory this will enable anonymous marking)

Note: in each of the above, there will be an impact on students. There may be more impact to staff and students in routes 3, 4 & 5, if tutors are unfamiliar with how to use these features.

Information about each of these routes is available via Faculty ELT Project Officers. However, the contingency for the potential unavailability of an institutional e-submission route will be locally-implemented, rather than institutional.

Faculty Communication from CELT and ITMS

Once it has been recognised that there is a service disruption to Turnitin, the following DMU-wide communication will follow.

• ITMS, as DMU’s System Administrator, inform CELT Team of disruption (NOTE: CELT monitor Turnitin social media as default).
• ITMS to raise ‘support help ticket’ with Turnitin, if necessary.
• Head of CELT or CELT nominee issues all staff Blackboard user email. This agreed form of words to staff will include: the issue; the impact on assignment submission (including anonymous marking).
• Faculties will need to decide about alternative submission routes or if a deferral of submission times/dates is to be agreed (see 1 above). This might be taken globally or by Programme Management Boards. Academic Practice Officers will be notified about downtime by the Faculty ELT Project Officer.

The following message will be placed on Blackboard and MyDMU for students.

Intermittent Turnitin Submission Issue

If you are submitting assignments to Turnitin you may experience problems in uploading your work. Please make a note of, or print-off, any message you receive from Turnitin and re-attempt the upload at a later time. All academic staff users of Turnitin have been informed of this intermittent issue. Students should still submit work to Faculty Offices/Advice Centres if directed to do so by their module teams, and should upload their work to Turnitin at the earliest possible opportunity.

This will be managed by CELT and ITMS.

Module/programme teams are responsible advising their students if there are any assignment submission changes as a result of the service disruption (i.e. extended date submission for electronic submission, with hard copy submissions are expected through faculty offices).

The Faculty ELT Project Officer will discuss updates/system risks with APOs and Chairs of FLTCs, as well as Heads of Study. The Head of CELT will communicate updates to staff users of Blackboard/Turnitin. The CELT Project Officer will update Library and Learning Services staff. When the system is fully functional as defined by a communication from Turnitin, CELT Head or CELT nominee will send an all users email.

To agree procedures where the failure of systems impacts processes like anonymous marking;

FLTCs and APOs need to consider ‘Alternative on-line submission routes as contingencies’ and their implications for anonymous marking, given that options beyond Turnitin are limited.

Programme and Module Leaders might usefully consider procedures or alternatives with the Faulty ELT Project Officer, including clear communication with students.

Faculty APOs need to understand alternative assignment submission procedures for anonymous marking, so that they are aware of circumstances surrounding any change.

NOTE: the failure of systems might impact staff workloads and agreed turnaround times.