Assessment


Assessing digital stories

 

 

An important consideration in the application of digital storytelling is whether it will be used formatively or summatively.  For most academic staff in higher education, digital storytelling is still a new technique and one that challenges the traditional paradigms of the written word (Coventry, 2008).  These challenges are illustrated by the following quote, from research into digital storytelling use at the University of Gloucestershire:

 

“What the stories have done is to make us reconsider the kinds of abilities that we are assessing because our approach to these kinds of modules has been within a paradigm that is about vocationalism and vocational skills and professionalism and mainstream academic skills...  What they demonstrate through the stories are a whole other range of abilities ... but they’re so different from the norm...  It’s made us think more deeply than we would otherwise have done about what we’re doing.  It has created some great challenges ... we haven’t resolved all these tensions yet but we’re treating them as good and positive.”  [Sport Lecturer, University of Gloucesteshire]

 

Digital stories can, as the above quote demonstrates, present a new challenge for assessment purposes.  Whether they are to be used for formative or summative assessment will impact on the guidance, structure and support that are in place for both students and academic staff.  Even with such guidance, digital stories introduce elements that have been traditionally alien to many discipline areas in higher education.  They allow student greater emotional engagement (Oppermann, 2008) and with developing technologies an increasing opportunity for creativity. 

 

In relation to creativity and assessment, there are three approaches to note:

 

  1. Assessing for creativity: where the emphasis is on designing assessment to encourage creativity
  2. Assessing creativity: where the focus is on assessing creativity
  3. Assessing creatively: where the focus is on being creative in the means of assessment.

 

 

Digital storytelling could be used to support any one of these approaches and even more than one at the same time.  Elton (2007) recognises the challenges these could present and argues that assessing creativity requires ‘connoisseurs’.  Creativity, Elton (2007) argues, needs a portfolio of evidence which encompasses the range of the students' work.  To assess portfolios - and Ohler (2008) has described a digital story as a ‘portfolio unto itself’ - requires an interpretivist approach that draws upon the professional expertise of the assessor. Hence the need for connoisseurs, experts in the discipline area.

 

Formative or summative?

Sanders (2009: 18) observes that there is a bias towards text-based assessment within education and questions whether digital storytelling is likely to be acceptable for summative assessment, proposing that it may be more suitable for formative assessment purposes. The affordance of iteration noted by Burn (2007) supports the developmental process that is fundamental to digital storytelling creation.  The iterations involved in developing the story, especially as a social process, are explicit with McDrury & Alterio’s Model of Reflective Learning Through Storytelling (McDrury & Alterio, 2003).  Digital storytelling does also provide a technique that makes composition strategies visible in new ways (Oppermann, 2008: 179).  The social aspect is particularly important where used formatively, allowing opportunities for both peer- and self-reflection. 

 

The problem with this assessment perspective, which focuses on the development of a story as a vehicle for reflective and process learning, is that it could be seen to place less importance of the story as an artefact.  This is the process versus product debate within the use of digital storytelling.  Where the story becomes a vehicle for ‘making learning explicit’ to assessors, there is a risk that the actual manifestation of the story can fall by the wayside.  This was highlghted by Simon Turner, a lecturer in Media at the University of Gloucestershire, who also worked with the Capture Wales project (http://www.bbc.co.uk/wales/audiovideo/sites/galleries/pages/capturewales.shtml), in Boase's (2008) overview of digital storytelling for reflection and engagement.  In this Turner argues that ‘people are far more inclined to use a tool such as this if it can be perceived as having value, and the first time they will come across it is by watching other people’s work’ (Turner, pers. comm.).  This argues that seeing the final product as only a means to an end, that is, assessment, will lead to a diminishment of the potential of the story to affect its audience beyond the assessors.  This debate could be seen to reflect potential tensions resulting from an expanding use of digital storytelling that is starting to operate outside the strict boundaries of the original CDS tradition approach.  However, the growth in digital storytelling has been in part due to the accessibility of the stories and that they can be easily shared, reviewed and reflected upon.  There is a need for a baseline in terms of technical quality, stories that are badly produced are unlikely to become effective pedagogic resources for others.  This baseline will though vary depending on how they are being used and who is producing them.  It is important to consider such factors when evaluating or assessing digital stories and these factors are considered in the example assessment frameworks in 'Assessment criteria for digital Storytelling'.

 

Despite Sanders' reservations, digital storytelling is being used for summative assessment within higher education.  The table below provides examples, with links to papers and case studies, where digital storytelling has been used for assessment.  Not always in disciplines which would not necessarily be considered ‘creative’, and the assessment of creative products has required the revision of assessment criteria in some cases (Gravestock & Jenkins, 2009).  It should be noted that not all digital stories have to be assessed, but in some situations they might be used as a replacement for a reflective-based assignment, such as an essay (see Case Study 2).

 

As with any assessment task, when assessing digital stories it is important to consider what the purpose of the assessment is and to ensure that the criteria match this purpose; however, there may be some additional aspects that need to be considered which relate to the use of the technology, such as use of voice or use of appropriate images to support the story.  The Section on Assessment criteria contains some example frameworks and criteria Assessment criteria for digital storytellingthat may be helpful as a starting point for developing criteria for assessing digital stories. 

 

 

Examples of digital storytelling

 

Reflection

Discipline area  
Dietetics

Postgraduate Diploma in Dietetics, Leeds Metropolitan University

Available on pp 40-42 of Sanders (2009).

Education and ICT

MA in Education and ICT, University of Leeds

Available on pp 36-40 of Sanders (2009).

Medicine

Bachelor of Medicine and Bachelor of Surgery, University of Leeds

Available on pp 29-31 of Sanders (2009).

Performance Arts

BA Theatre and Performance, University of Leeds

Available on pp 32-36 of Sanders (2009).

Sport Using digital stories for individual student reflection, University of Gloucestershire.  Case study 2.
Science (Ecology, Evolution & Genetics)

First year science students at University of British Columbia, asked to create a digital story to reflect upon the personal signifiance of their learning on this course (Prud'homme Genereux & Thompson, 2008).

 

Example digital story available from UBC Coordinated Science Program web site.

 

 

Discipline area  
Education

Olney, I.W., Herrington, J. & Verenkina, I. (2009) Digital story telling using iPods, in Herrington, J., Mantei, A., Olney, I.W. & Ferry, B. (Eds) New technologies, new pedagogies: mobile learning in higher education, Faculty of Education, University of Wollongong.  Available from <http://ro.uow.edu.au/edupapers/79> Accessed 17th September 2009.

 

Describes an authentic learning task requiring education students to create a picture-story book for children, in a digital form with sound and visual effects.

   

 

 

Formative Assessment

 

Peer Assessment

Discipline area  
Landscape Design Use of digital stories as part of studio culture and for personal reflection.  Case study 1