‘UNIwise were really supportive and communicative and addressed our needs’ - the Imperial College Business School’s experience of implementing WISEflow.
Imperial College Business School in London, UK is consistently ranked among the world’s top business schools offering MBA, MSc and Executive programmes, as well as joint honours undergraduate degrees in collaboration with Imperial College departments.
Imperial College Business School ran a pilot project in 2016 and expanded to 70+ exams with 1600 students in 2020.
Context
Imperial College Business School had started a global MBA programme in 2015; digital assessment capability was required in order to offer a fully online experience. WISEflow was chosen in 2016 after a procurement process. The initial pilot project ran in the 2016/2017 academic year with the MSc in Climate Chance, Management and Finance. The exam took place on campus using the lockdown browser, with approximately 40 students, and the project was a success.
Expansion of the pilot as Bring Your Own Device (BYOD) was pushed back until a dedicated project lead could be funded. Emma Stoakes joined the project in 2019, and a successful summer school exam, featuring all multiple-choice questions (MCQs) and 70 students, took place in 2019
What follows is an account of the WISEflow project at ICBS by Emma Stoakes, Exams and Assessment Manager & Digital Assessment Project Lead (Business School).
Emma Stoakes, Exams and
Assessment Manager &
Digital Assessment Project
Lead, Imperial College
London Business School
“We were planning to expand to 14 BYOD exams across a number of MSc cohorts in the April 2020 exam session, developing our use to include written answers, use of appendices, fill-in-the-blanks questions, case studies and in-platform marking. The exam and assessment team had attended bespoke training on user roles in preparation for the Spring exams, and we ran in-person training sessions for students, who were also given help with any lockdown browser issues. They got access to a bespoke demo test that showed them how to use commonly used question types, and we planned drop-in sessions for anyone who had any further issues. We were also going to expand our invigilator training. Everything seemed to be going well. And then lockdown hit.
“Students went home. We were in the middle of a Finance suite exam period and three MCQ tests were due to take place the week after lock-down was announced. Our main exam period was coming up in April. In other words, we needed a plan.
“Because Finance was mid-exam, there wasn’t time for any WISEflow training, so we used our VLE. This was pretty easy to set up, but scoring and marking was fairly laborious, and there were many other administrative checks that needed to be carried out. Following the success of our Summer School Pilot, we decided to run the planned MCQ tests using the WISEflow platform, as our first ever remote timed assessments, since I was confident that students would find MCQs on WISEflow intuitive. The weekend before those tests, we ran WISEflow training sessions, including producing videos on accessing and using the platform. During the tests, we easily extended the exam time for any students who had issues; in the end, everyone who sat the exam the next week was able to complete it. Furthermore, when we looked at their results, there was no obvious indication of cheating or collusion.
“The UNIwise Support team was amazing and helped us with anything that looked a little iffy.”
“For the Spring exam period, most of the students sitting exams had already been trained on WISEflow. We decided against using the lockdown browser, because we wouldn’t have any invigilation or control over additional device use, and because this option also gave us the flexibility for students to handwrite graphs and equations and upload them as appendices. As a result, all exams could be moved onto WISEflow, even those that were maths-heavy. Students were told that these would be closed-book exams and that the expectation was that students would behave with academic integrity; we notified them that samples would be run through Turnitin. We also gave all students a remote exam allowance of 30 minutes to cover upload time and any other issues. Furthermore, we decided to use the instant message functionality to inform students when the standard exam period had ended and encourage them to start uploading and checking their work. We set up help desk email accounts for queries and a WISEflow information page on our VLE. We also ran some town hall meetings in place of the training sessions, where students could pre-submit questions. We trained our undergraduate students on WISEflow remotely, as they hadn’t had this opportunity on campus before lockdown. We also decided that marking would be completed in WISEflow, so training guidance was produced to help faculty, including a faculty WISEflow page.
“On reflection, once the exams were over, we could see that students were well prepared to use WISEflow. The instant messaging functionality was really useful and worked well to communicate with students. The UNIwise Support team was amazing and helped us with anything that looked a little iffy. Faculty appreciated not having to read handwriting. Overall, it was very successful! There were, of course, some issues, given the short timescale we had to sort this out. Like everyone else, we had no control over the remote scenario – students’ connectivity, VPNs or adblockers. We also struggled to help some students when we couldn’t see their screens. We had a lot of post-exam emails from students being anxious and asking us to check that their uploads had been received. It was a baptism of fire for some! The student interface is great, and the notepad, highlighter and accessibility tools for dyslexia are really good. From a student point of view, everything’s quite easy. What was more challenging was trying to anticipate what would work for the markers. The exam staff created the exams for faculty and made decisions when setting up the exams that led to a mixed marking experience for faculty. We did also see quite a high amount of collusion and plagiarism in comparison to on-campus assessment, and college regulations at the time were a bit lacking in guidance for how to handle that. However, many of these issues were common to all assessment platforms across the college, and, overall, WISEflow had fared really well given the circumstances.
“With our next batch of exams in June 2020, we moved most of the exams that had been on our VLE onto WISEflow; we will be keeping them on the plat- form going forward until full return to campus, when discussions concerning BYOD, remote and paper-based will be held.”
“We’d planned to run 13 exams on WISEflow, but have now conducted 74 on the platform, which is much more than we anticipated, and accounts for over 8,000 submissions, not taking into account all of the summer resits we had.
“We sought feedback from students so we could make improvements for our new student cohort. We now have a new, more polished introduction video about WISEflow; we’ve moved the training to be closer to exams and been clear with students about what to expect. We’re training students on how to reference sources in exams and are using the ‘front cover’ of the exam to get students to agree to our plagiarism statement. With faculty, our Associate Dean of Education Policy is providing guidance on how to write exams for remote sitting. We’ve also set up a faculty working group, and we’ve worked with them to produce some new assessment set-up guidance and an exam template that has common settings options included in it, so that they can easily indicate their preferences to the exams staff at the same time as providing the exam content.
“We love the UNIwise Support team! They always have our back. They’re also good at prioritising issues and inquiries and are great at thinking outside the box.”
“For those looking to run exams on WISEflow, I’d say that you have to consider all stages and take time to play around with the system. I’d set up dummy flows and run through them as a student, as a marker – all stages, all roles. You need to have an understanding of the complexities of the system, what it can do and what options you have – and then make sure that your decisions work for your users as best as possible.
“It was a steep learning curve, but I can’t believe it’s been less than a year since we did the summer school exams. We’ve gone from doing one MCQ with 70 people to running 74 exams with 1,600 students, in different formats, in less than a year. We’ve had to be really agile.
“It’s also been really brilliant having the UNIwise Support team on hand. We love the Zendesk guys! They’re really good at looking into issues for you and never make us feel like we’re being a pain. The Support team always has our back. They’re also good at prioritising issues and inquiries and asking the development team for help if necessary, and are great at thinking outside the box.
This academic year, we’re aiming to continue to embed WISEflow. We’re working collaboratively with our faculty working group to improve the author and marker experience. We’ll continue to share development suggestions with UNIwise and hopefully start to work with a London WISEflow user group to share our experience and best practices and identify common development needs.”
All images courtesy of Imperial College London Business School.