|PA Home > Modules Main Page > Part 4: Creating Rubrics > Setting Criteria
8:1 Setting Criteria
Go back to part 7
Go on to Part 9
This manual is based on the assumption that instruction and assessment are a continual cycle. Learners must be assessed on what they have learned. The portfolio model in this manual is based on the premise that assessment is the first thing to consider when planning instruction. This means that you carefully consider the cycle of instruction and assessment and that you continually check your instruction against your objectives and your assessment against the objectives or criteria. You may find that you are unable to reach all your instructional objectives. With classroom portfolio assessment, you have the flexibility of deciding to modify your objectives and your assessment.
To illustrate this cycle, consider the work of a diving coach and diver. Each dive is judged on very specific criteria by the judges in a competition. If a coach wants to prepare a diver for competition, the coach will need to know which dive the diver can and/or must use and the criteria on which each dive will be judged. The coach will tell the diver the criteria and provide instruction to help the diver make progress toward the objectives for each dive. At some point, the diver will compete, performing dives that are judged on exactly those criteria which were learned. It would not make much sense for the diver to go into competition not knowing what was expected or what was needed to get a 10. It also would not make much sense if the coach had taught the diver to do a back flip when a front flip was going to be judged.
The same thing is illustrated in gymnastics, figure skating, and other types of competition. The objectives become the criteria for assessing. In the same way, we need to share the objectives/criteria of our teaching with our language students and let them know that the objectives are also the assessment criteria. We also need to think of our objectives/criteria as we begin planning for instruction and as we design assessment.
Criteria answer the questions:
When you have established your criteria you need a way to measure whether and to what extent students have reached the objectives. In other words, you need to define quality or quantity. To do this we use rating scales. There are different types of scales, for example, checklists, number ratings, descriptive words, analytic, and holistic. You need to choose a scale that reflects your purpose and works with your criteria. For instance, if you want to identify whether students engage in a specific behavior or use a certain grammatical structure you may decide to use a checklist or a yes/no scale. If you want to differentiate performances you may decide to use a number scale of 1-3 or 1-5, being sure to describe what each number represents. Descriptive words are useful for identifying the frequency with which students do something, for example, usually, rarely, or sometimes and to qualify a type of behavior as excellent, average, or poor. In each case, of course, you will need to determine what constitutes each rating. For example, you would need to make it clear what determines a rating of usually or poor.
Numerical and descriptive adjectives are often perceived differently by students, parents, and teachers. Numerical data allow us to deal with the data quantitatively; this means the information can be easily manipulated for statistical purposes. However, we have to be careful of what the data tell us. For instance, it may be clear that a 4 is better than a 2 but it may not be literally twice as good. One can also not assume that a 3 on one scale is of equal value to a 3 on another scale.
How many scale points should there be? It is usually easier for raters to agree when there are fewer points because it is easier to agree that something is excellent, good, fair, or poor than to agree whether it is excellent, very good, good, fair, poor, marginal, or unacceptable. On the other hand, fewer points may fail to capture differences in levels of student work and progress over time. You will find rubrics with different rating scales throughout this .
Click here and you will find sample rubrics for measuring student use of learning strategies. While the questions are all about one general topic, learning strategies, you will see the way of eliciting the information is different, which means that slightly different information is gathered. Having different questions and scales is a way to triangulate information, that is, gather similar data from different perspectives to gain deeper understanding.
Everyone who uses the rubrics needs to understand the criteria and the rating scale. It is important to make sure that scales help the audience/assessor understand the meaning of the performance so that use of results is fair. Benchmark examples of students' work corresponding to each level on the scale help to define the meaning of scores. Such benchmark examples can be very useful for your students as well, before they begin work.
Teacher reflection: Last year I asked my German students to help set criteria for their oral presentations. I wanted to involve students in the entire presentation process to make it more meaningful for them. After I introduced the presentation unit, I had them write lists in their journals for homework: How to Give a Good Speech or How to Give a Bad Speech. In the class they worked in groups to draw up list of the criteria by which to evaluate a speech. We discussed the criteria, deciding which were valid. For example, we took off "interesting" because we decided that couldn't be measured. I also added a couple of things that I thought were important. We used an "acceptable" and "unacceptable" rating. We used the checklists for teacher, peer-, and self-evaluation. I thought that having them write the evaluation criteria gave them much more of a stake in the whole process and improved the entire experience- for them and me. I think the point is that the students needed to know for themselves what to do and how they did, not just have me tell them.
<< Back | Next >>