CHAPTER 7: STUDENT TEST-TAKING STRATEGIES

Student Test-Taking Strategies

TEST WHAT YOU TEACH AND HOW YOU TEACH

In today's universities, grades are substantially determined by test results. So much importance is placed on students' test results that often just the word "test" makes students afraid. The best way for students to overcome this feeling of fear or nervousness is to prepare themselves with test-taking strategies. This process should begin during the first week of each semester and continue throughout the school year. The key to successful test taking lies in a student's ability to use time wisely and to develop practical study habits.

Nowadays, effective test-taking strategies are synonymous with effective learning strategies. This section is intended to provide suggestions for long-term successful learning techniques and test-taking strategies, not quick "tricks". There is nothing that can replace the development of good study skills.

The following steps will help students approach tests with confidence:

• Make a semester study plan.

• Come to class regularly.

• Use good review techniques.

• Organize pre-exam hours wisely.

• Plan out how to take the exam.

• Use strategies appropriate to the skill area.

• Learn from each exam experience.

Make A Semester Study Plan

Students need to plan their study time for each week of their courses. They should make schedules for themselves and revise these schedules when necessary.

These schedules should:

• BE REALISTIC. Keep a balance between classes and studying. Block out space for study time, class time, family time and recreation time.

• INCLUDE A STUDY PLACE. Finding a good place to study will help students get started; don't forget to have all the materials needed (i.e. pens, paper, textbooks, highlighter pens etc.).

• INCLUDE A DAILY STUDY TIME. Students forget things almost at once after learning them, so they should immediately review materials learned in class.

. Students should go over the main points from each class and/or textbooks for a few minutes each night. Encourage students to do homework assignments during this time as a good way to remember important points made in class.

. Come To Class Regularly

In order for language learning to take place, students need to come to class on a regular basis. It is not surprising to note that poor attendance correlates highly with poor test results.

. Teachers need to point out early in the semester what constitutes legitimate reasons to be absent and stress the advantages of regular attendance.

. Use Good Review Techniques

If students make a semester study plan and follow it, preparing for exams should really be a matter of reviewing materials. Research shows that the time spent reviewing should be no more than 15 minutes for weekly quizzes, 2 to 3 hours for a midterm exam, and 5 to 8 hours for a final exam.

When reviewing for a test, students should do the following:

• PLAN REVIEW SESSIONS. Look at the course outline, notes and textbooks. What are the major topics? Make a list of them. How much time was spent on each of these topics in class? Did the teacher note that some topics were more important than others? If so, these should be emphasized in review sessions.

• TAKE THE PRACTICE EXAM. By taking the practice exam students will have an idea of the tasks/activities that they will encounter on the real exam. They will also know the point allocation for each section. This information can help them plan their time wisely.

• REVIEW WITH FRIENDS. Another way of studying for an exam is to create a "study group". By studying with friends there is the advantage of sharing information with others who are reviewing the same material.

An important factor in test taking is exam planning. Students should arrive early at the designated exam room and find a seat. All books and personal effects (with the exception of student ID cards and writing materials) should be left at the front of the room. Students should come prepared with several pens or pencils and an eraser.  As soon as the exams have been distributed and students have been told to start the exam, the student should write his/her name and ID number on all pages of the test paper.

If one section is given first, such as the listening portion of English exams, the student should focus attention on this section. With any section of the exam, the student is well-advised to do an overview of the questions, their values, and the tasks required. At this point, students should determine if the exam must be done in order (i.e. listening first) or if they can skip around between sections. The latter is not possible on some standardized exams where students must complete one section before moving on to the next.

An important consideration in effective test taking is time management. When exams are written, review time is usually factored into the overall exam design. Students should be encouraged to allocate their time proportional to the value of each exam section and to allow time to review their work. Teachers when proctoring can assist students with time management by alerting them to time remaining in the exam. Computer based tests (such as the new TOEFL) often show a countdown of the remaining time. Students should be made aware of this feature during the practice exams.

Learn From Each Exam Experience.

Each test should be a learning experience. Go over test results with students. Teachers should note specific students' strengths and weaknesses. The analyses that teachers receive right after computer-based exams provide teachers with invaluable information in a timely manner. Teachers should use this information to send students to student support services for remediation.

  

CHAPTER 6: ASSESSING SPEAKING

Assessing Speaking

 

Always keeping the cornerstones of good assessment in mind, why do we want to test speaking? In a general English program, speaking is an important channel of communication in daily life. We want to simulate real-life situations in which students engage in conversation, ask and answer questions, and give information. In an academic English program, the emphasis might be on participating in class discussions and debates or giving academic presentations. In a Business English course, students might develop telephone skills, interact in a number of common situations involving meetings, travel, and sales as well as make reports. Whatever the teaching focus, valid assessment should reflect the course objectives and the eventual target language.

Speaking is a productive language skill like writing and thus shares many issues such as whether to grade holistically or analytically. However, unlike writing, speaking is more ephemeral unless measures are taken to record student performance. Yet the presence of recording equipment can inhibit students and often recording is not practical or feasible. To score reliably, it is often necessary to have two teachers assess together. When this happens, one is the interlocutor who interacts with the speaker(s) while the other teacher, the assessor, tracks the student’s performance.

According to Bygate’s categories, Weir (1993) oral skills into are divided in two main groups: speaking skills that are part of a repertoire of routines for exchanging information or interacting, and improvisational skills such as negotiating meaning and managing the interaction. The routine skills are largely associated with language functions and the spoken language required in certain situations. By contrast, the improvisational skills are more general and may be brought into play at any time for clarification, to keep a conversation flowing, to change topics or to take turns. In circumstances when presentation skills form an important component of a program, naturally they should be assessed. However, avoid situations where a student simply memorizes a prepared speech.

It is good practice to start the speaking assessment with a simple task that puts students at ease so they perform better. Often this takes the form of asking the students for some personal information.

Interview: can be teacher to student or student to student. Teacher to student is more reliable when the questions are scripted.

Description of a photograph or item: Students describe what they see.

Narration: This is often an elaboration of a description. The student is given a series of pictures or cartoon strip for the major events in a story.

Information gap activity: One student has information the other lacks and vice versa. Students have to exchange information to see how it fits together.

Negotiation task: Students work together on a task where they may have different opinions. They have to reach a conclusion in a limited period of time.

Roleplays: Students are given cue cards with information about their “character” and the setting. Some students find it difficult to project themselves into an imaginary situation and this lack of “acting ability” may affect reliability.

Oral presentations: Strive to make them impromtu instead of rehearsed.

Recommendations for Speaking Assessment

Decide with your colleagues which speaking subskills are most important and adopt a grading scale that fits your program. Whether you adopt a holistic or analytical approach to grading, create a recording form that enables you to track students’ production and later give feedback for improvement.

Think about these factors: fluency vs. accuracy, appropriate responses (indicating comprehension), pronunciation, accent and intonation, use of repair strategies.

Train teachers in scoring and practice together until there is a high rate of inter-rater reliablity. Use moderation sessions with high-stakes exams.

Keep skill contamination in mind. Don’t give students lengthy written instructions which must be read and understood before speaking.

Remember that larger samples of language are more reliable. Make sure that students speak long enough on a variety of tasks.

Choose tasks that generate positive washback for teaching and learning!

CHAPTER 5: ASSESSING LISTENING

Assessing Assessing Listening

The assessment of listening abilities is one of the least understood, least developed and yet one of the most important areas of language testing and assessment (Alderson & Bachman, 2001). In fact, Nunan (2002) calls listening comprehension “the poor cousin amongst the various language skills” because it is the most neglected skill area. As teachers we recognize the importance of teaching and then assessing the listening skills of our students, but - for a number of reasons - we are often unable to do this effectively.

One reason for this neglect is the availability of culturally appropriate listening materials suitable for EF/SL contexts. The biggest challenges for teaching and assessing listening comprehension center around the production of listening materials. Indeed, listening comprehension is often avoided because of the time, effort and expense required to develop, rehearse, record and produce high quality audio tapes or CDs.

Approaches to Listening Assessment

Buck (2001) has identified three major approaches to the assessment of listening abilities: discrete point, integrative and communicative approaches.

The discrete-point approach

It became popular during the early 1960’s with the advent of the Audiolingual Method. This approach identified and isolated listening into separate elements. Some of the question types that were utilized in this approach included phonemic discrimination, paraphrase recognition and response evaluation. An example of phonemic discrimination is assessing students by their ability to distinguish minimal pairs like ship/sheep. Paraphrase recognition is a format that required students to listen to a statement and then select the option closest in meaning to the statement. Response evaluation is an objective format that presents students with questions and then four response options. The underlying rationale for the discrete-point approach stemmed from two beliefs. First, it was important to be able to isolate one element of language from a continuous stream of speech. Secondly, spoken language is the same as written language, only it is presented orally.

The integrative approach

It started in the early 1970s called for integrative testing. The underlying rationale for this approach is best explained by Oller (1979:37) who stated “whereas discrete items attempt to test knowledge of language one bit at a time, integrative tests attempt to assess a learner’s capacity to use many bits at the same time.” Proponents of the integrative approach to listening assessment believed that the whole of language is greater than the sum of its parts. Common question types in this approach were dictation and cloze.

The third approach, the communicative approach


It arosed at approximately the same time as the integrative approach as a result of the Communicative Language Teaching movement. In this approach, the listener must be able to comprehend the message and then use it in context. Communicative question formats must be authentic in nature.

Issues in Listening Assessment

A number of issues make the assessment of listening different from the assessment of other skills. Buck (2001) has identified several issues that need to be taken into account. They are: setting, rubric, input, voiceovers, test structure, formats, timing, scoring and finding texts. Each is briefly described below and recommendations are offered.

Setting

The physical characteristics of the test setting or venue can affect the validity and/or reliability of the test. Exam rooms must have good acoustics and minimal background noise. Equipment used in test administrations should be well maintained and checked out beforehand. In addition, an AV technician should be available for any potential problems during the administration.

Rubric

Context is extremely important in the assessment of listening comprehension as test takers don’t have access to the text as they do in reading. Context can be written into the rubric which enhances the authenticity of the task. Instructions to students should be in the students’ L1 whenever possible. However, in many teaching situations, L1 instructions are not allowed. When L2 instructions are used, they should be written at one level of difficulty lower than the actual test. Clear examples should be provided for students and point values for questions should be included in the rubrics.

Input

Input should have a communicative purpose. In other words, the listener must have a reason for listening. Background or prior knowledge needs to be taken into account. There is a considerable body of research that suggests that background knowledge affects comprehension and test performance. In a testing situation, we must take care to ensure that students are not able to answer questions based on their background knowledge rather than on their comprehension.

Voiceovers

Anyone recording a segment for a listening test should receive training and practice beforehand. In large-scale testing, it is advisable to use a mixture of genders, accents and dialects. To be fair for all students, listening voiceovers should match the demographics of the teacher population. Other issues are the use of non-native speakers for voiceovers and the speed of delivery. Our belief is that non-native speakers of English constitute the majority of English speaking people in the world. Whoever is used for listening test voiceovers, whether native or non-native speakers, should speak clearly and enunciate

Coombe/Hubley 28

The speed of delivery

The speed of a listening test should be consistent with the level of the students and the materials used for instruction. If your institution espouses a communicative approach, then the speed of delivery for listening assessments should be native or near native delivery. The delivery of the test should be standard for all test takers. If live readers are used, they should practice reading the script before the test and standardize with other readers.

Test Structure

The way a test is structured depends largely on who constructs it. There are generally two schools of thought on this: British and the American perspectives. British exam boards generally grade input from easy to difficult in a test and mix formats within a section. This means that the easier sections come first with the more difficult sections later. American exam boards, on the other hand, usually grade question difficulty within each section of an exam and follow the 30/40/30 rule. This rule states that 30% of the questions within a test or test section are of an easy level of difficulty; 40% of the questions represent mid range levels of difficulty; and the remaining 30% of the questions are of an advanced level of difficulty. American exam boards usually use one format within each section. The structure you use should be consistent with external benchmarks you use in your program. It is advisable to start the test with an ‘easy’ question. This will lower students’ test anxiety by relaxing them at the outset of the test.

Within a listening test, it is important to test as wide a range of skills as possible. Questions should also be ordered as they are heard in the passage. Questions should always be well-spaced out in the passage for good content coverage. It is recommended that no content from the first 15-20 seconds of the recording be tested to allow students to adjust to the listening. Many teachers only include test content which is easy to test, such as dates and numbers. Include some paraphrased content to challenge students.

Formats

Perhaps the most important piece of advice here is that students should never be exposed to a new format in a testing situation. If new formats are to be used, they should be first practiced in a teaching situation and then introduced into the testing repertoire. Objective formats like MCQs and T/F are often used because they are more reliable and easier to mark and analyze. When using these formats, make sure that the N option is dropped from T/F/N and that three response options instead of four are utilized for MCQs. Remember that with listening comprehension, memory plays a role. Since students don’t have repeated access to the text, more options add to the memory load and affect the difficulty of the task and question. Visuals are often used as part of listening comprehension assessment. When using them as input, make certain that you use clear copies that reproduce well.  Coombe/Hubley 29

Timing

The length of a listening test is generally determined by one of two things: the length of the tape or the number of repetitions of the passages. Most published listening tests do not require the proctor to attend to timing. He/she simply inserts the tape or CD into the machine. The test is over when the proctor hears a pre-recorded “this is the end of the listening test” statement. For teacher-produced listening tests, the timing of a test will usually be determined by how many times the test takers are permitted to hear each passage. Proficiency tests like the TOEFL usually allow one repetition whereas achievement tests usually repeat the input twice. Buck (2001) recommends that if you’re assessing main idea, input should be heard once and if you’re assessing detail, input should be heard twice. According to Carroll (1972), listening tests should not exceed 30 minutes.

It is important to remember to give students time to pre-read the questions before the test and answer the questions throughout the test. If students are required to transfer their answers from the test paper to an answer sheet, extra time to do this should be built into the exam.

Scoring

The scoring of listening tests provides numerous challenges to the teacher/tester. Dichotomous scoring (questions that are either right or wrong) is easier and more reliable. However, it doesn’t lend itself to many of the communicative formats such as note-taking. Other issues are whether points are deducted for grammar or spelling mistakes or non-adherence to word counts. When more than one teacher is participating in the marking of a listening test, calibration or standardization training should be completed to ensure fairness to all students.

Finding Suitable Texts

Many teachers feel that the unavailability of suitable texts is listening comprehension’s most pressing issue. The reason for this is that creating scripts which have the characteristics of oral language is not an easy task. Some teachers simply take a reading text and ‘transform’ it into a listening script. The transformation of reading texts into listening scripts results in contrived and inauthentic listening tasks because written texts often lack the redundant features which are so important in helping us understand speech. A better strategy is to look for texts that concentrate on characteristics that are unique to listening. If you start collecting texts that have the right oral features, you can then construct tasks around them. When graphics or visuals are used as test context, teachers often find themselves ‘driven by clip art’. This occurs when teachers build a listening script around readily.  Coombe/Hubley 30

To produce more extemporaneous listening recordings, use available programs on your computer like Sound Recorder or shareware like Audacity and PureVoice to record scripts for use as listening assessments in the classroom.

Vocabulary

Research recommends that students must know between 90-95% of the words to understand a text/script. Indeed the level of the vocabulary that you utilize in your scripts can affect the difficulty and hence the comprehension of students. If your institution employs word lists, it is recommended that you seed vocabulary from your own word lists into listening scripts whenever possible. To determine the vocabulary profile of your text/script, go to http://www.er.uqam.ca/nobel/r21270/cgi-bin/webfreqs/web_vp.cgi for Vocabulary Profiler, a very user-friendly piece of software. By simply pasting your text into the program, you will receive information about the percentage of words that come from Nation’s 1000 Word List and the

Academic Word List.

Another thing to remember about vocabulary is that ‘lexical overlap’ can affect difficulty. Lexical overlap refers to when words used in the passage are used in the questions and response options. When words from the passage are used in the correct answer or key, the question is easier. The question becomes more difficult if lexical overlap occurs from the passage/script to the distractors. A final thought on vocabulary is that unknown vocabulary should never occur as a keyable response (the actual answer) in a listening test.

Final Recommendations for Listening Assessment

No matter what the skill area, as always test developers should be guided by the cornerstones of good testing practice when constructing tests.

• Validity (Does it measure what it says it does?)

• Reliability (Are the results consistent?)

• Practicality (Is the test “teacher-friendly”?)

• Washback (Is feedback channeled to everyone concerned?)

• Authenticity (Do the tasks mirror real life contexts?)

• Transparency (Are expectations clear to students? Do Ss and Ts have access to information about the test/assessment?)

• Security (Are exams and item banks secure? Can they be reused?)

Coombe/

CHAPTER 4: ASSESSING WRITING

Assessing Writing

 

Most teachers find that it is relatively easy to write subjective test item prompts as contrasted to objective ones. The difficulty lies in clearly specifying the task for the student so that grading is fair and equitable to all students. Some teachers find that the best approach is to write a sample answer and then analyze the elements of that answer. Alternatively, it is useful to ask a colleague to write a sample answer and critique the prompt. Writing good subjective items is an interactive, negotiated process.

The F/SL literature generally addresses two types of writing: free writing and guided writing. The former requires students to read a prompt that poses a situation and write a planned response based on a combination of background knowledge and knowledge learned from the course. Guided writing, however, requires students to manipulate content that is provided in the prompt, usually in the form of a chart or diagram.

Guided Writing
 

Guided writing is a bridge between objective and subjective formats. This task requires teachers to be very clear about what they expect students to do. Decide in advance whether mechanical issues like spelling, punctuation and capitalization matter when the task focuses on comprehension. Some important points to keep in mind for guided writing are:

• Be clear about the expected form and length of response (one paragraph, a 250-word essay, a letter etc.).

• If you want particular information included, clearly specify it in the prompt (i.e. three causes and effects, two supporting details etc.)

• Similarly, specify the discourse pattern(s) the students are expected to use (i.e. compare and contrast, cause and effect, description etc.)

• Since guided writing depends on the students manipulation of the information provided, be sure to ask them to provide something beyond the prompt such as an opinion, an inference, or a prediction.

• Be amenable to revising the anticipated answer even as you grade.

• Be clear about the expected form and length of response (one paragraph, a 250-word essay, a letter etc.).

• If you want particular information included, clearly specify it in the prompt (i.e. three causes and effects, two supporting details etc.)

• Similarly, specify the discourse pattern(s) the students are expected to use (i.e. compare and contrast, cause and effect, description etc.)

• Since guided writing depends on the students manipulation of the information provided, be sure to ask them to provide something beyond the prompt such as an opinion, an inference, or a prediction.

• Be amenable to revising the anticipated answer even as you grade.

Coombe/Hubley 20

Fundamentals of Language Assessment

Free Writing

All of the above suggestions are particularly germane to free writing. The goal for teachers is to elicit comparable products from students of different ability levels.

• The use of multiple raters is especially important in evaluating free writing. Agree on grading criteria in advance and calibrate before the actual grading session.

• Decide on whether to use holistic, analytical or a combination of the two as a rating scale for marking.

• If using a band scale, adjust it to the task.

• Acquaint students with the marking scheme in advance by using it for teaching, grading homework and providing feedback.

• Subliminally teach good writing strategies by providing students with enough space for an outline, a draft and the finished product.

• In ES/FL classrooms, be aware of cultural differences and sensitivities among students. Avoid contentious issues that might offend or disadvantage students.

Writing Assessment Scales

The F/SL assessment literature generally recognises two different types of writing scales for assessing student written proficiency: holistic marking and analytical marking.

Holistic Marking Scales

Holistic marking is where the scorer “records a single impression of the impact of the performance as a whole” McNamara (2000:43). In short, holistic marking is based on the marker's total impression of the essay as a whole. Holistic marking is variously termed as impressionistic, global or integrative marking.

Experts in holistic marking scales recommend that this type of marking is quick and reliable if 3 to 4 people mark each script. The general rule of thumb for holistic marking is to mark for two hours and then take a rest grading no more than 20 scripts per hour. Holistic marking is most successful using scales of a limited range (i.e. from 0-6).

FL/SL educators have identified a number of advantages to this type of marking. First, it is reliable if done under no time constraints and if teachers receive adequate training. Also, this type of marking is generally perceived to be quicker than other types of writing assessment and enables a large number of scripts to be scored in a short period of time. Thirdly, since overall writing ability is assessed.

Coombe/Hubley 21

Students are not disadvantaged by one lower component such as poor grammar bringing down a score.

Several disadvantages of holistic marking have also been identified. First of all, this type of marking can be unreliable if marking is done under short time constraints and with inexperienced, untrained teachers (Heaton, 1990). Secondly, Cohen (1994) has cautioned that longer essays often tend to receive higher marks. Testers point out that by reducing a score to one figure tends to reduce the reliability of the overall mark. The most serious problem associated with holistic marking is the inability of this type of marking to provide feedback to those involved. More specifically, when marks are gathered through a holistic marking scale, no information or washback on how those marks were awarded appears. Thus, testers often find it difficult to justify the rationale for the mark. Hamp-Lyons (1990) has stated that holistic marking is severely limited in that it does not provide a profile of the student's writing ability.

Analytical Marking Scales

Analytical marking is where “raters provide separate assessments for each of a number of aspects of performance” (Hamp-Lyons, 1991). In other words, raters mark selected aspects of a piece of writing and assign point values to quantifiable criteria (Coombe & Evans, 2001). In the literature, analytical marking has been termed discrete point marking and focused holistic marking.

Analytical marking scales are generally more effective with inexperienced teachers. These scales are more reliable for scales with a larger point range.

A number of advantages have been identified with analytical marking. Firstly, unlike holistic marking, analytical writing scales provide teachers with a "profile" of their students' strengths and weaknesses in the area of writing. Additionally, this type of marking is very reliable if done with a population of inexperienced teachers who have had little training and grade under short time constraints (Heaton, 1990).

 Finally, training raters is easier because the scales are more explicit and detailed.

Just as there are advantages to analytical marking, educators point out a number of disadvantages associated with using this type of scale. Analytical marking is perceived to be more time consuming because it requires teachers to rate various aspects of a student's essay. It also necessitates a set of specific criteria to be written and for markers to be trained and attend frequent moderation or calibration sessions. These moderation sessions are to insure that inter-marker differences are reduced which thereby increase the validity. Also, because teachers look at specific areas in a given essay, the most common being content, organization, grammar, mechanics and vocabulary, marks are often lower than for their holistically-marked counterparts. Another disadvantage is that that analytical marking scales remove the integrative nature of writing assessment.