Dr Marie Bassford, Senior Lecturer, Faculty of Technology, started to teach a new first year Physics Fundamentals module in 2014/15. Having taught only second and third year students for a number of years; Marie saw the opportunity to develop a new first year module as a vehicle to re-engage with the use of phase tests for assessment.
In the past, Marie would have made use of a paper based optical mark reading system as the platform for delivering phase tests but with her learning technologist background, Marie sought to identify a more automated approach that could be re-used year on year.
Marie delivers Physics Fundamentals with two colleagues and following an exploratory conversation with the ELT Project Officer, Marie and the team decided to move forward with the Blackboard Learn Test tool for administering phase tests via the Virtual Learning Environment.
Each of the module team members were tasked with producing questions for the phase test. These questions were linked to the learning outcomes and were all multiple choice type questions with one correct answer. The phase test itself was to be 25 questions in total but with the input of three colleagues a bank of 58 questions was produced.
The ELT Project Officer helped Marie to develop her skills and understanding of the Test tool and Marie ensured that the questions were authored in Blackboard Learn.
The over production of questions was purposeful as in conversation with the ELT Project Officer, Marie had decided to create a large pool of questions and have Blackboard Learn serve a random 25 questions to each student from the pool. There were a number of reasons for this approach:
- Having each student answer a randomly selected set of 25 questions helps to minimise copying in the test environment;
- Building a pool of questions enables Marie and the team to add to this pool each year; and
- The pool can be re-used year on year with minimal effort by including it in the annual Course Copy.
There was an amount of effort required to create the initial pool of 58 questions but the three colleagues teaching Physics Fundamentals spread this work across the module team and this pool will now roll over each year. Marie is confident that although there will inevitably be updates and amendments to the question pool; in the long run, the time that has been invested will be recouped.
Due to the multiple choice nature of the questions, Blackboard Learn marks each test upon completion and there are options for students to see their provisional grade instantly upon completion of the test along with any generic/automated feedback. When compared to the work required to print and scan optically mark read submissions the use of the Test tool minimises the effort required to deliver the test and grades once the initial pool is created and the test is deployed on the module shell.
The students who engaged with the phase test in December 2014 were generally positive about the experience. One student did question the use of randomised question sets but as the questions that each student received had all been carefully written to support one or more learning outcome this query was swiftly dealt with.
Conversely, one student actively told Marie that she thought it was “clever how the questions were randomised” and that she understood that it made it fairer to assess that way when taking the test together, side by side at PCs.
During the test, there was one issue in that a superscript character that had been used in one of the questions did not display correctly but this was quickly dealt with on the day and Marie has now re-formatted the question text to ensure that it displays fully to the students.
Marie’s next Physics Fundamentals phase test in is April 2015 and she will be using this question pool with the addition of further questions for the second phase test and Marie will continue to use the Test tool this year and in years to come as part of the Physics Fundamentals assessment components.
Thank you to Dr Marie Bassford and the Physics Fundamentals module team for enabling the documentation of their use of online phase tests. If you wish to learn more about how to replicate this practice at DMU, please contact your local ELT Project Officer.