After four years of WISEflow, Professor Mariann Rand-Weaver, Vice-Provost at Brunel University London, reflects on digital assessment and its future.

Written by Professor Mariann Rand-Weaver, Vice-Provost (Education), Brunel University London

As we on-board the last College to using WISEflow we reach a milestone that is worth celebrating. We have successfully negotiated the journey from proof of principle in 2015/16 to institution-wide adoption in 2019/20, and WISEflow is now the University’s assessment platform for summative coursework assignments (completed in students’ own time to an extended deadline) and exams (onsite, time-limited, invigilated assessments). We have learnt a lot in the last four years – the most important lesson being that there is no substitute for experience.

Growth

We have grown total submissions from 642 in 2015/16 to 51,258 in 2018/19, and the number of exams from one for 156 students to 79 exams for over 2700 students in the same period (Figure 1). This means that over 20% of students experienced a Bring Your Own Device (BYOD) digital exam during the last academic year.

This achievement can be attributed to the enthusiasm and willingness of academic staff, professional staff and senior management to engage in a cross-institutional collaboration in order to demonstrate how technology can improve practice for students and staff, and successfully deliver high-stakes exams. We are therefore rightly proud to have been recognised with an Advance HE Collaborative Award for Teaching Excellence (CATE) in August 2019; this national award is given to teams that have had demonstrable impact on learning and teaching.

Figure 1. Infographic showing growth of BYOD exams at Brunel University London

Reflections

This milestone is also an opportunity to take stock and to consider three key questions:

• Are we content with digital versions of pen and paper exams?

• How can technology be integrated in the assessment, and not merely used as a means of documenting answers?

• How can pedagogy, rather than efficiency, become the driver for exploiting the use of technology in assessment?

Preparing young people for successful careers means preparing them for a digital future, and technology can help us to meaningfully assess skills, capabilities and competencies that are required to succeed in 21st century workplaces.

What limits adoption of digital assessment?

It is perhaps surprising that the Higher Education sector globally is not further along this journey, given that digital exams have been used for over 20 years. It may be partly due to the drivers – for example, a sole focus on administrative efficiencies. The technical and security aspects may be sufficient disrupters in themselves, and once addressed there is little appetite for further disruption. Academic staff are often averse to the perceived risks associated with more adventurous assessments; after all, they are subject to student evaluations. Whilst all these reasons may play a part, the truth is probably that few of us now using digital exams have planned beyond implementation and scaling up, as these tasks are fairly momentous in themselves.

Arild Raaheim and colleagues (2019) provided insight to what the barriers are to innovative use of digital exams in Norway, and interestingly the top responses related to a lack of knowledge of how assessments could make use of a digital format, and of alternatives to essay questions. This raises questions about the feasibility of expecting academic staff, most of whom will not be from a digitally native generation, to have the expertise required to develop post-paper exams. A recent report by JISC on use of technology in teaching suggested that digital literacy among staff was lower than perhaps we might have expected it to be; it is likely that it would have been even lower if assessments were considered.

Perhaps we should be satisfied with digitised pen-and-paper exams; after all we are reaping the benefits of improved administration and reduced paper use. There are no uncomfortable demands on academic staff who can continue to set the same question types as in the past, and no challenges to current practice – we can stay in our comfort zone whilst getting some benefits of digitisation.

Is this enough?

But is such a position tenable? We are already in a connected, digital world, where information is easily available and it is the ability to know what to do with this information, to see new connections and to transform it into new knowledge that will be valued and attractive to employers. With so much information at our fingertips, memory tests are no longer a valued concept. The Times Higher Education published an article earlier this year which called for more authentic assessments - clearly possible in a digital exams environment.

At Brunel we have ambitions to use digital exams to assess in ways that are not possible with pen-and-paper exams; to set sophisticated tasks that will challenge students to demonstrate what they can do and apply knowledge to authentic tasks that are aligned to the skills and capabilities our students will need for successful careers.

We are just starting on the post-paper exams journey, where the use of technology is integral to the question to the extent that it is no longer possible to switch to pen and paper should there be a technical issue during the exam. Whilst this is a leap of faith (there is no easy back-up), it is also a natural progression once a degree of familiarity with the software is achieved. For us, the white-listing feature in WISEflow that allows access to defined documents or applications has helped push boundaries; for example our students have been asked to analyse and visualise data in a spreadsheet containing over 1000 rows of information; or design a prototype game based on a client brief. These examples demonstrate how it is possible to create examination tasks that assess creativity, an attribute highly valued by employers, as there will be a range of ways in which such assessments can be tackled.

Looking to the future

We have come a long way in the last four years, but we are just beginning to tackle the challenge of integrating pedagogy and technology to derive real value for our students. The academic trailblazers are important for the next step of our digital assessment journey, as they are showing us what is possible, and demonstrate that innovative practice can be within reach of all staff. We are now looking outwards to a greater extent than before, and are finding inspiration from the other side of the world (literally!) – the Australian government-funded e-Exams project (2016) aims to transform exams in Australian Universities through the use of BYOD digital exams, and their reports and resources are valuable for those just starting out.

BYOD digital exams have the potential to transform assessment and lever curriculum change if institutions provide the academic leadership, investment in training and opportunities to share good practice that will be required. Whilst the onus is firmly on institutions to drive this forward, a collaborative approach with suppliers has the potential to create an assessment revolution - the footprint left by the thousands of digital submissions could be a rich source of information on how assessment can be used for learning as well as of learning. Let’s make the future happen!