Designing student learning by promoting formative assessment

In her paper at Learning in Law Annual Conference 2008 Alison Bone (University of Brighton) presented the findings of a pilot study exploring the use of optional assessments as a ‘feed-forward’ device to enable students to submit better work for future assessment.

According to the 2006 National Student Survey students are generally happy with the teaching on their course, but they are less satisfied with assessment and feedback. To date qualitative research examining how students value feedback has been on a comparatively small scale (Higgins et al, 2002; Orsmond et al, 2005), although it has long been acknowledged that how students are assessed influences how they learn (Gibbs, 1999; Sadler, 2003).

A small scale survey involving law students carried out across the UK in 2005 investigated the provision of formative assessment opportunities for students to enable them to receive feedback. This found that comparatively few were given ‘pure’ formative assessment – ie assessment which did not count in any way towards their final grade – although the majority felt it would have been of benefit to them (Bone, 2006). As a result of these findings it was decided there would be value in conducting more focused research into the take-up of ‘pure’ formative assessment in an undergraduate law programme. This paper reports on the findings of a pilot study undertaken during 2006-07 and sets the scene for data being collected during the 2007-08 academic year.

The pilot study, involving a small number of students, highlighted time pressure as one of the key reasons students will provide as to why they do not engage with such assessment. Students’ views on the feedback provided were not always complimentary, although the majority appreciated the comments made and acted on them, feeling that their work improved as a consequence. The current study is exploring in more depth the rationale behind the choice to participate or not in optional assessments, and views as to the usefulness of the exercise.

Formative assessment: the context

Research into formative assessment and its role in student learning has been conducted over several years, at least from a theoretical perspective (Ramaprasad, 1983; Sadler, 1989; Wiliam & Black 1996). Much of the discussion about the value of feedback focuses on the importance of the student acting on the feedback given as opposed to simply noting the mark awarded (Sadler, 1989), so there tends to be an underlying assumption that most of such feedback accompanies a piece of work which has been graded. It is difficult to identify studies where students have only been given qualitative feedback or where the grade awarded has not counted towards their final mark. This latter element is particularly significant, as it introduces the element of choice which is pivotal to this research project.

It has also been argued that from an assessment perspective the student needs to learn how the tutor assesses (Taras, 2006 p367, quoting from Sadler 1983 p121) so that both the tutor and the student need to (a) have a concept of the standard being aimed for and (b) compare the actual level of performance with the standard. In the case of the student there is of course a third requirement – that they take the appropriate action to close the gap between the two. The ability to do this will depend on how useful the feedback is – many authors comment that students receive a wide variety of remarks from their tutors – some detailed, some vague, some authoritarian, some empathetic (Connors & Lunsford, 1993; Creme & Lee 1997). Student interviews carried out by Higgins, Hartley & Skelton (2002 p55) reveal examples of feedback that is not at all helpful:

But some of it was like “this line is immature” which wasn’t particularly useful in any way…and the worst of it, the problem was that she didn’t specify what was wrong with it, she just said “this line isn’t right”, “this is wrong”, “this is very good”, “this introduction is unstructured”, but she didn’t say how it had become unstructured.

As will be seen from the findings of the pilot study below, most students do not seem to mind if feedback is relatively short as long as it is focused and gives clear guidance. Knight (2006) makes the useful distinction between two types of feedback – feed-back, which comments on the quality of the work, and feed-forward, which consists of information intended to help the student do better on a similar type of task in the future (p446). If a student is being summatively assessed by a piece of coursework then arguably only the first form of feedback is required, whereas feed-forward is essential if there is a likelihood that similar assessment tasks will be set in the future.

The main problem seems to be getting students to engage with formative assessment which does not ‘count’. There are differences across disciplines – for example Weaver (2006) discovered that art and design students are more likely to accept and value feedback on work in progress’ than business studies students. Recent guidance provided by Bloxham & Ford (2007) is limited to suggestions involving peer assessment – they too recognise that unless the work submitted has some percentage of the final grade attached to it, students are unlikely to do it.

The pilot study

A pilot study involving 55 students on a final year law module (employment law) at the University of Brighton was conducted in 2006-07. The students met in two different workshop groups on consecutive days. The invitation to submit a sample of work for feedback (and of course feed-forward) was extended to both groups orally and by means of a handout, which could be passed to students not attending on a particular day. Details were also posted on the module homepage on the university intranet. It is not known how many of the 55 students in total received this invitation, although the registers indicate 48 students attended classes and received it directly. The benefits of completing this optional assessment were explained in class.

The students on this module are assessed in two ways – an examination at the end of the academic year worth 70% of the final mark, and a piece of coursework worth 30%, due to be submitted at the beginning of February 2007. Although the majority of students taking this module were law students for whom this was a core module (they had no choice but to take it), there were around a dozen students who were taking it as an elective. This latter group had studied only one or two (usually business-based) law modules in the earlier years of their courses.

The coursework for the module consists of a report on recent employment law developments. Many of the students admitted in their feedback on the module at the end of the course that this was the first time they had been asked to write a report or to make an evaluation based on their own choice of material. In November 2008 all students were invited to submit up to 500 words online by way of a sample of the 2,500-word piece of coursework on which they would later be summatively assessed. They were told that it had to be submitted a week before the end of term and that they would be given brief feedback (again online) covering both style and content. They were informed that this sample would not be given a mark, nor would it be noted who had or had not submitted.

After the full coursework had been handed in, marked and returned (March 2007) the whole group were invited to complete individual questionnaires about why they had (or had not) decided to submit a sample, how they used any feedback they had received and their perceptions of the value of this formative assessment. These questionnaires were distributed in the lecture and completed during it to improve the response rate. This enabled quantifiable data to be generated. (Focus groups were not used in this pilot study, so there is little qualitative material except as set out in answers to the questionnaire.)

During 2007-08 100 students studying a second year law module and 90 students studying a third year module have been given the same option, ie to submit a piece of work online as a sample so that they may receive lecturer feedback on both style and content. Again this sample will not be graded. Questionnaires will be distributed after the (summative) coursework has been marked and returned, and focus groups of students will be invited to discuss the findings.

Pilot study findings

Only 13 students (around 24% of the whole group) decided to take advantage of the offer and submit a sample of work by the three week deadline. Six were taking the module as an elective – ie they were not law students. Feedback was provided within three days of submission by e-mail. This was very brief typically, consisting of 4-5 sentences, and usually emphasised the need (if they had failed to do so) to be evaluative rather than descriptive, with a short explanation of what was required. Only 34 students completed the questionnaire – a response rate of 62%.

Of the 34 who completed the questionnaire 22 (64%) said they had not submitted a sample piece of work in order to receive feedback. It is known of course that a higher percentage actually did not do so, but not how many of these simply did not know about it (nobody completing the questionnaire said they were unaware of the invitation).

The main reason given for deciding not to submit a sample was lack of time – 21/22 (95%) gave this reason. There were a couple of comments:

I suppose I could have found time but I felt it would be better to wait until [my understanding of the topic] had developed a bit.
 
I had other coursework deadlines and research to do.

Two students said that they did not think it would help with the final piece of work (this was given by one student as an additional reason to the lack of time). One said they had only completed it after the deadline and therefore did not submit it.

Students who had not submitted a sample were also asked if, now they had had their final work back, they thought they could have improved their mark if they had submitted the sample and received comments on it. 11/22 (50%) responded positively, and most gave comments:

It would have given me an idea of how to really tackle the task and what to focus on.
 
I would have had a clearer understanding of the style required…the content needed for a high mark. The practice would have helped too.

Similar comments were received from others – about both structure and content. 8/11 (36%) however responded negatively – they did not think getting feedback would have helped them. There was a number of different reasons given, not all of which were necessarily ‘anti-feedback’ – for example: “I was happy with the mark I got”, “I achieved a first class mark without completing the formative assessment”. There were two rather pointed comments however:

A couple of people who did it said the feedback wasn’t very detailed and so did not help them with the assignment.
 
Nobody was happy with the feedback. Some were marked down for points that were not commented on in the 500 word sample.

Three students were unsure about whether doing the sample would have helped – one because they achieved a really high mark anyway, whereas another philosophised “possibly – every little helps”. The third said “how can I know as I did not do it”!

Of the 13 who did the sample 12 completed the questionnaire. Eight (67%) said that doing the sample was helpful for a variety of reasons – not all of which were to do with feedback – for example: “it saved time later on”. One was unsure, but said it may have helped with time management. Students were also specifically asked whether the feedback they were given was helpful. Seven (58%) were very positive, saying they were reassured by the comments received.

It confirmed that my writing style was correct for the work required which was helpful since it was a style that I had not written before. [The feedback] recommended that I look carefully at the law rather than focusing too much on business.

All the questionnaires were anonymous, but it seems likely that this comment was made by a business studies undergraduate.

The other five students did not think the feedback was helpful for a variety of reasons, most of which seemed to be related to their final mark, although one said the feedback was “vague” and the other that it was “minimal”.

Others said “my errors were not pointed out in the sample although they were on the final assignment” and “it was helpful to write [the sample] but the feedback from [the final] assignment criticised the way I had put the feedback to use”.

After the summative coursework was returned a generic feedback sheet was circulated, pointing out what was generally done well and also common errors. It was noted by one student that it may have been helpful to have such a sheet provided to all students based on the sample coursework, regardless of whether they had done it or not. This will be done in the current academic year.

The deadline has now passed for the 100 second year law students to submit a sample of work for comment in the 2007-08 study. Although the benefits of doing the work were explained by the lecturer at the start of the course when over 80% of the group were present, only two students submitted work for feed-forward comments. Reasons for non-submission will be explored by questionnaire after the summative coursework has been marked and returned.

Extract from employment law assignment brief

Hand-out date: w/c 30 October 2006
Hand-in date: 6 February 2007
Hand-back date: w/c 5 March 2007
 
The assignment task is to compile a report highlighting key employment law developments that occur between 1 November 2006 and 31 January 2007. Publications/information appearing between these dates may refer back to earlier decisions/developments, but the emphasis must be on current developments.
 
The report can take a broad approach highlighting all key developments in areas covered in the first semester (status, contracts of employment, individual rights, family rights, discrimination and equal pay), or specialise in one area only, for example discrimination. Similarly the report can be a broad overview of a particular topic or concentrate on (say) four or five key cases/developments.
 
Whichever approach is adopted it is very important that you explain carefully why you have selected the cases or other materials, ie why they are deemed worthy of inclusion. This is a level 3 assignment, and although credit will be given for your research skills (see the learning outcomes below), the most important aspect of this assignment is the demonstration of your critical judgement in deciding which materials to include.
 
‘Materials’ can be given as wide an interpretation as you wish, and may include newspaper articles, journal articles etc as well as formally recognised law reports and law websites, but if you use non-legal sources remember to give them the appropriate treatment – this is a law assignment so please don’t write like a tabloid journalist!

References

---

• Bloxham S & Boyd P (2007) Developing effective assessment in higher education: a practical guide Buckingham: Open University Press
• Bone A (2006) The impact of formative assessment on student learning (UKCLE funded research project)
• Connors R & Lunsford A (1993) ‘Teachers’ rhetorical comments on student papers’ College Composition and Communication 44:200-223
• Creme P & Lea M (1997) Writing at university Buckingham: Open University Press
• Gibbs G (1999) ‘Using assessment strategically to change the ways students learn’ in S Brown & A Glasner (eds) Assessment matters in higher education: choosing and using diverse approaches Buckingham: Open University Press
• Higgins R, Hartley P & Skelton A (2002) ‘The conscientious consumer: reconsidering the role of assessment feedback in student learning’ Studies in Higher Education 27(1):53-64
• Knight P (2006) ‘The local practices of assessment’ Assessment & Evaluation in Higher Education 31:435–452
• Orsmond P, Merry S & Reiling K (2005) ‘Biology students’ utilization of tutors’ formative feedback: a qualitative interview study’ Assessment & Evaluation in Higher Education 30(4):369-386
• Ramaprasad A (1983) ‘On the definition of feedback’ Behavioural Science 28:4-13
• Sadler D (1983) ‘Evaluation and the improvement of student learning’ Journal of Higher Education 54:60-79
• Sadler D (1989) ‘Formative assessment and the design of instructional systems’ Instructional Science 18:145-165
• Taras M (2006) ‘Do unto others or not: equity in feedback for undergraduates’ Assessment & Evaluation in Higher Education 31(3):365–377
• Weaver M (2006) ‘Do students value feedback? Student perceptions of tutors’ written responses’ Assessment & Evaluation in Higher Education 31(3):379–394
• Wiliam D & Black P (1996) ‘Meanings and consequences: a basis for distinguishing formative and summative functions of assessment?’ British Educational Research Journal 22(5):537–548

Last Modified: 9 July 2010