CHAPTER 4: ASSESSING WRITING

Assessing Writing

 

Most teachers find that it is relatively easy to write subjective test item prompts as contrasted to objective ones. The difficulty lies in clearly specifying the task for the student so that grading is fair and equitable to all students. Some teachers find that the best approach is to write a sample answer and then analyze the elements of that answer. Alternatively, it is useful to ask a colleague to write a sample answer and critique the prompt. Writing good subjective items is an interactive, negotiated process.

The F/SL literature generally addresses two types of writing: free writing and guided writing. The former requires students to read a prompt that poses a situation and write a planned response based on a combination of background knowledge and knowledge learned from the course. Guided writing, however, requires students to manipulate content that is provided in the prompt, usually in the form of a chart or diagram.

Guided Writing
 

Guided writing is a bridge between objective and subjective formats. This task requires teachers to be very clear about what they expect students to do. Decide in advance whether mechanical issues like spelling, punctuation and capitalization matter when the task focuses on comprehension. Some important points to keep in mind for guided writing are:

• Be clear about the expected form and length of response (one paragraph, a 250-word essay, a letter etc.).

• If you want particular information included, clearly specify it in the prompt (i.e. three causes and effects, two supporting details etc.)

• Similarly, specify the discourse pattern(s) the students are expected to use (i.e. compare and contrast, cause and effect, description etc.)

• Since guided writing depends on the students manipulation of the information provided, be sure to ask them to provide something beyond the prompt such as an opinion, an inference, or a prediction.

• Be amenable to revising the anticipated answer even as you grade.

• Be clear about the expected form and length of response (one paragraph, a 250-word essay, a letter etc.).

• If you want particular information included, clearly specify it in the prompt (i.e. three causes and effects, two supporting details etc.)

• Similarly, specify the discourse pattern(s) the students are expected to use (i.e. compare and contrast, cause and effect, description etc.)

• Since guided writing depends on the students manipulation of the information provided, be sure to ask them to provide something beyond the prompt such as an opinion, an inference, or a prediction.

• Be amenable to revising the anticipated answer even as you grade.

Coombe/Hubley 20

Fundamentals of Language Assessment

Free Writing

All of the above suggestions are particularly germane to free writing. The goal for teachers is to elicit comparable products from students of different ability levels.

• The use of multiple raters is especially important in evaluating free writing. Agree on grading criteria in advance and calibrate before the actual grading session.

• Decide on whether to use holistic, analytical or a combination of the two as a rating scale for marking.

• If using a band scale, adjust it to the task.

• Acquaint students with the marking scheme in advance by using it for teaching, grading homework and providing feedback.

• Subliminally teach good writing strategies by providing students with enough space for an outline, a draft and the finished product.

• In ES/FL classrooms, be aware of cultural differences and sensitivities among students. Avoid contentious issues that might offend or disadvantage students.

Writing Assessment Scales

The F/SL assessment literature generally recognises two different types of writing scales for assessing student written proficiency: holistic marking and analytical marking.

Holistic Marking Scales

Holistic marking is where the scorer “records a single impression of the impact of the performance as a whole” McNamara (2000:43). In short, holistic marking is based on the marker's total impression of the essay as a whole. Holistic marking is variously termed as impressionistic, global or integrative marking.

Experts in holistic marking scales recommend that this type of marking is quick and reliable if 3 to 4 people mark each script. The general rule of thumb for holistic marking is to mark for two hours and then take a rest grading no more than 20 scripts per hour. Holistic marking is most successful using scales of a limited range (i.e. from 0-6).

FL/SL educators have identified a number of advantages to this type of marking. First, it is reliable if done under no time constraints and if teachers receive adequate training. Also, this type of marking is generally perceived to be quicker than other types of writing assessment and enables a large number of scripts to be scored in a short period of time. Thirdly, since overall writing ability is assessed.

Coombe/Hubley 21

Students are not disadvantaged by one lower component such as poor grammar bringing down a score.

Several disadvantages of holistic marking have also been identified. First of all, this type of marking can be unreliable if marking is done under short time constraints and with inexperienced, untrained teachers (Heaton, 1990). Secondly, Cohen (1994) has cautioned that longer essays often tend to receive higher marks. Testers point out that by reducing a score to one figure tends to reduce the reliability of the overall mark. The most serious problem associated with holistic marking is the inability of this type of marking to provide feedback to those involved. More specifically, when marks are gathered through a holistic marking scale, no information or washback on how those marks were awarded appears. Thus, testers often find it difficult to justify the rationale for the mark. Hamp-Lyons (1990) has stated that holistic marking is severely limited in that it does not provide a profile of the student's writing ability.

Analytical Marking Scales

Analytical marking is where “raters provide separate assessments for each of a number of aspects of performance” (Hamp-Lyons, 1991). In other words, raters mark selected aspects of a piece of writing and assign point values to quantifiable criteria (Coombe & Evans, 2001). In the literature, analytical marking has been termed discrete point marking and focused holistic marking.

Analytical marking scales are generally more effective with inexperienced teachers. These scales are more reliable for scales with a larger point range.

A number of advantages have been identified with analytical marking. Firstly, unlike holistic marking, analytical writing scales provide teachers with a "profile" of their students' strengths and weaknesses in the area of writing. Additionally, this type of marking is very reliable if done with a population of inexperienced teachers who have had little training and grade under short time constraints (Heaton, 1990).

 Finally, training raters is easier because the scales are more explicit and detailed.

Just as there are advantages to analytical marking, educators point out a number of disadvantages associated with using this type of scale. Analytical marking is perceived to be more time consuming because it requires teachers to rate various aspects of a student's essay. It also necessitates a set of specific criteria to be written and for markers to be trained and attend frequent moderation or calibration sessions. These moderation sessions are to insure that inter-marker differences are reduced which thereby increase the validity. Also, because teachers look at specific areas in a given essay, the most common being content, organization, grammar, mechanics and vocabulary, marks are often lower than for their holistically-marked counterparts. Another disadvantage is that that analytical marking scales remove the integrative nature of writing assessment.