Students Staff

5 June 2014

A Collaborative Lecture Annotation System

Filed under: Ideas, Issue 3 — inpractice @ 3.37 pm

Tom Foulsham (Psychology) describes the development of an innovative new piece of software that allows students to bookmark and annotate online lecture material and compare their observations with peers.

For full PDF version click here

The idea for the Collaborative Lecture Annotation System (CLAS) began when I was working in the Psychology department at the University of British Columbia in 2009. A group of us were conducting experiments on how people pay attention to complex stimuli such as videos. Whilst working with my colleagues, Alan Kingstone and Evan Risko, we became curious about how this might apply to lectures. We began to ask ourselves how students know when to pay attention. Equally we were interested in how people might differ in what they chose. We wondered if students might become distracted by something that was particularly memorable or funny but miss information that the lecturer felt was important. Additionally, we wanted to know if students were performing badly because they hadn’t yet learned how to sustain their attention throughout a lecture or select the right information.

We were also aware that students were keen to access their lectures online. Indeed, where this option wasn’t available they were recording the lectures for themselves on their mobile devices. There was however very little information about what they actually did when they listened again. Our suspicion was that students might simply replay the whole lecture again in the hope that the information would somehow filter through. There is a lot of research in psychology that shows memorising material passively is not very effective, so this strategy is unlikely to be successful. We felt it would be far more beneficial to students if they were able to pick out key items of information from the recorded material. The metaphor we were using at that time was highlighting a text; we were interested in how the act of highlighting might help the student recall information later on.

Out of these early discussions, we came up with the idea for the Collaborative Lecture Annotation System (CLAS). In its first iteration it was a web based tool that allowed students to watch an online lecture – like those on YouTube or on iTunes University, for example – and bookmark the most important parts at the press of a button, the video equivalent of highlighting a text. In the later version we included the option to add a note about why the user thought that part of the lecture was important. They could also see everyone else’s bookmarks and annotations, and were presented with a “consensus map” which identified the points in time that most people thought were important. The idea of these functions was to allow students to engage more actively with recorded lectures through making notes and personalising the content. Students could then use these tags to find the lecture highlights later on, perhaps when revising. Peer learning was also an important aspect of the tool. Although a student’s individual notes are important, if they found that they differed from the group consensus in their identification of key points they could compare their conception of what was important, and why, with that of their classmates. This might guide them towards an important piece of information they missed. Of course, it might also mean that everyone else was wrong and they were right, either way it would allow them to return to that part of the lecture and reflect on it. With more advanced modules, or on modules where there is more room for interpretation, this could be an interesting question in its own right and might prompt further explorations of the material.

In the first phase of the project controlled experiments were carried out using existing online lectures. Volunteers came in for two sessions. In the first session they just watched a video. In the second session, a week later, they had fifteen minutes to revise followed by a test. There were several variants of the experiment which were designed to test whether the CLAS software was useful. Half the volunteers had access to CLAS and the other half didn’t. Initial data suggested that participants remembered the material better when they used the tool. Interestingly most participants revised their interpretations of what was important after seeing what others had tagged, and found moving between their own notes and those of their peers useful. We also carried out experiments where the lecture was in audio format only. Some points were highlighted across both video and audio formats, perhaps because a key word or key concept had been introduced, but other key issues were missed in the audio format, maybe because it was shown on a slide or was emphasised more through the lecturer’s body language.

During this academic year, I recorded my own lecture and made this available to the students along with the CLAS tool. The uptake has been somewhat disappointing. This may reflect the fact that the module ran in the autumn term, whereas the tool might be more useful in the run up to summer exams. It may also be the case that students who were present in the live lecture already had adequate notes. However, it may be that the integration of the tool into the module needs to be rethought, with some different strategies for getting students involved earlier which I will be considering for next year. Ultimately, it will be important to evaluate the context in which the tool is used, as well as the tool itself; is it useful in face-to-face courses, or is it better suited to remote learning? Another possibility is to use it as part of a ‘flipped classroom’ approach, whereby the students watch a recorded lecture in advance and then use face-to-face time to reflect on the highlights or the parts that were missed.

Despite it being early in development we feel that CLAS has many potential benefits for both students and lecturers. For the students it can help them highlight the key points in a lecture and share their perceptions with colleagues. This contributes to more active and collaborative learning. There are also potential benefits for the lecturer. We as teachers have an idea about what is important in our lectures, but we have no way of knowing what our students are taking away. This system can tell us where our own perceptions don’t align with our students. It also provides opportunities to consider why certain things were not picked up and perhaps experiment with different ways of signposting key points to see if that makes a difference, or go over that information again in the following lecture. I have also discovered, through this piece of research, the unexpected value of being filmed whilst lecturing. It can be painful to watch but it does provide a good sense of how you’re doing and how you have improved from year to year.

The next step would be to consider whether CLAS could be used in a live lecture. This might be particularly valuable for students in very large lectures where high levels of peer interaction aren’t possible. It would allow the lecturer, perhaps during a break, to see where people were paying attention, where they weren’t, and providing the opportunity to address those gaps immediately. It would also be useful to look at how CLAS might integrate with other classroom technologies such as Listen Again or EVS (clickers). In the meantime, the CLAS technology is being developed as open-source software for use with online video and audio material. I would be very happy to hear from anyone interested in trialling it at Essex in their own teaching.

 

Tom Foulsham

Department of Psychology

University of Essex

E foulsham@essex.ac.uk

 

Leave a Reply

The University of Essex will moderate comments and there will be a delay before any posts appear.