Kerfuffle over evaluating performance.

All Whites win Supreme Halberg Award

I am sure you are all aware of the commotion in New Zealand over awarding the Supreme Halberg Award for 2010  to the All Whites football team. http://tinyurl.com/4vghe24 The problem appears to be confusion over evaluating performance of individuals and teams. This confusion is present in education just as it is in sports awards.

When it comes to evaluating anyone or team I am always reminded of these lines from F. Scott Fitzgerald‘s Great Gatsby.

In my younger and more vulnerable years my father gave me some advice that I’ve been turning over in my mind ever since.

“Whenever you feel like criticizing any one,” he told me, “just remember that all the people in this world haven’t had the advantages that you’ve had.”

This quote always reminds me that in evaluating performance we need to consider where we have come from to where we are now. We need to ensure we are not comparing apples with oranges. In education terms, we need to know where are students are at before commencing their study, so that we can track their performance over time. We need to ensure our performance criteria are clear, and observable and integrated with our learning goals.

I used to teach evaluating personal performance to students, who were going out into industry to be placed in business. As part of their assessment process the students self-assessed their own portfolios. It was essential the students understood the concepts and principles of evaluation and had the opportunity to practice their knowledge and skills.

Apples versus Oranges

We always used the same practice exercise, evaluating oranges. The students were asked to identify four characteristics of an orange to be exported to Australia. We asked students to ensure their criteria were objective and observable. Naturally, students chose characteristics of size, shape, colour and smell. Then we tested our criteria. I would have a bag of oranges and hand out one to each group. They evaluated and graded the orange according to a pre-agreed points system. I always had one apple to evaluate as well.

It always amused me that the students were so consistent in their grading. As we lined up our oranges from best to worst, it was clear our criteria had served us well. Inevitably, the apple was last, and the poorest performer.

Those same skills can be applied in any situation where we evaluate another person’s work or performance. This is particularly relevant to student assessment, and why we must take great care with designing assessment items, marking schedules and feedback mechanisms. It is important to our students that any evaluation of their work or performance is fair, and valid.  Here are a few questions to help you with evaluation processes.

1. What do you want  your students to be able to do?

2. To what standard or level do you want  your students to be able to perform?

3. What knowledge, skills and attributes must be demonstrated?

4. Who are you assessing? An individual? A group? A hybrid of individual and group work?

5. What resources do your students need in order to perform? Does everyone have access to those resources?

6. How does this assessment fit with the learning outcomes of the course? Does it assess one or more outcome? Is their equal emphasis on the performance of each outcome?

In any situation, where we evaluate we need clear criteria. Whether we are assessing student performance, the performance of a colleague (peer observation of teaching), our team performance or our own performance, we need clear and agreed criteria.

The criteria can help us to establish student progress over time. Our evaluative criteria can inform a short diagnostic assessment at the commence of a course, to establish the knowledge and skill levels of students. Tracking progress over time helps us determine who has made the greatest progress in a year or semester of study.

So before you start off on your next task, assessment or project take time to establish a clear set of evaluative criteria.  These will serve you well, in evaluating the performance or yourself or others.

Cross blogged from http://www.ethosconsultancynz.com/profiles/blogs/kerfuffle-over-evaluating

Advertisements
This entry was posted in performance and tagged , , , , , , . Bookmark the permalink.

One Response to Kerfuffle over evaluating performance.

  1. I was just reading your ideas and guidelines here, and I started to wonder, at a fundamental level why do we evaluate performance? I just did a quick Google search “Why do we evaluate?” – the first three of 31,500,000 results attempt to answer the question ‘why’ but the majority focussed on ‘how’. One of the articles suggested that evaluation is part of our natural instinct as human beings whereby we evaluate our performance and things around us as a survival mechanism. A quick search on ‘Why do we evaluate in education?’ returned a couple of interesting results, one of which explains very clearly the differences between evaluation, assessment and evaluation. In this article it is suggested that “Evaluation is perhaps the most complex and least understood of the terms. Inherent in the idea of evaluation is “value.” When we evaluate, what we are doing is engaging in some process that is designed to provide information that will help us make a judgment about a given situation.”

    The act of evaluation of one’s own and other’s performance, as well as the world around us, therefore is grounded in notions of value (‘fit for purpose’ as well as intrinsic value), appropriacy, benchmarking, safety…and, when the results of evaluation are then applied for improvement and / or change, form part of an invaluable iterative cycle – as long as (as you point out) we have a set of criteria from which to work.

    Thanks for sharing these guidelines and suggestions 🙂 Appreciated.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s