Pink Folder/Define assessment procedures

From YWAMKnowledgeBase

Pink folder.png
Making Evaluation/Assessment Work in Your School

by Brad Tout.

As Tom Bloomer has explained so well in the preceding article entitled "?[correction needed]" evaluation of our students is not simply an academic exercise. Yes, we do produce grades and evaluations that result in credits and satisfactory completion of courses. But the academic result is not the core purpose of evaluation. Evaluation and assessment is the process by which we and the student know just where they are at in their progress toward achieving the goals of their learning. It shows strengths, gifts, and points needing development. It allows us to know whether or not the student is ready to put their learning into practice and/or move on to the next level of learning.

This is a purpose we also see reflected in scripture. Whenever God corrects an individual, it is not so that He can say, "I've caught you!" Instead it is to show us just where we still need to grow and what things need to change in our lives in order to live as a reflection of who He is. It is also an encouragement to us when we understand the test and can feel confident that we can step forward into the future. It is a process designed for growth, not condemnation.

Evaluation and assessment should not be about judgement or comparisons between one student and another. In fact we want to encourage cooperation, learning together, achieving common goals through joint effort. Evaluation is a way of checking out how far a person has progressed towards God's vision for them and signposts to the next stage of learning. The word '?[correction needed]' has a Latin root which means 'beside' This provides a word picture with two helpful interpretations. You can see two people sitting beside one another, not one person taking a superior position but an expression of mutuality. The second thought is laying one thing alongside another to see where it is the same and where it is different. In the context of training it is comparing the student against something which shows just what they need to know and be able to do in order to fulfil the tasks for which they are being trained.

It is not our desire to compare one trainee against another but rather to check them against what they need to know and do in order to move into the ministry or vocation that the course is training them to do. Evaluation is a cooperative effort between trainee and trainer in order to maximise the learning experience and develop the tools the trainee will need in order to move on to the tasks and/or next phase God has for them.

How do I Plan the Assessment for My School?

When it comes to developing an assessment for a school, there are a number of considerations.

What do We Want to Actually Assess?

For the end result we will be aiming at (for most of our courses will usually be a combination of):

  • certain knowledge,
  • certain skills and
  • using these in the way God would use them (character/attitudes).

So you need to define what knowledge, skills and character aspects are important in order to be effective in the type of ministry or field of study your school is preparing someone for. This can involve visualising the person doing this ministry/work after they have been trained. Ask yourself: what are they doing; what skills are they using; what knowledge are they using; how are they showing God's character. Once you've done that, you can start to design appropriate tools for measuring progress in those things.

What is an Appropriate Tool or Method?

First, it should measure what it is meant to measure.

For example, if you are wanting to check someone's knowledge, do you want to test their memory of that knowledge or their ability to apply it to certain types of situation? If it does not need to be a memory test, then you could allow the students to refer to notes and books while they answer the questions.

You can also ask other questions that help, such as: is spelling and grammar important or not?

Because of the variety of nationalities, cultures and educational backgrounds in our schools, an assessment tool also needs to be flexible in how you use it, so that it can be adapted to the needs of the student where that is appropriate.

It should be reliable and objective enough to produce a consistent result with different students and each different time that you use it.

The tool, or combination of tools, should provide enough information, so that you can make a clear decision about the person's knowledge or skill or character.

What is an Inappropriate Tool or Method?

To begin with, refer back to Tom Bloomer's article entitled "?[correction needed]". This article gives guidelines on principles that should shape our evaluation, including some aspects that should not be followed in the U of N. Any tool that incorporates these aspects is inappropriate: grading on a curve; comparing one student against another; not considering cultural variations; not considering learning challenges, such as attention deficit and dyslexia, not considering language, literacy and numeracy difficulties related to nationality, culture and/or educational experience.

Tools which are too subjective or are too easily influenced by the examiner's personal views and pre-conceptions are inappropriate.

Tools which are not understood by the students are inappropriate. The directions need to be clear and not possible to be misinterpreted.

What we are asking them to do or present must be achievable. A tool which tests knowledge or skills we have not yet taught them or is beyond a reasonable expectation that they can be achieved is also inappropriate.

A tool that is too rigid and does not allow for different forms of expression is also inappropriate. We all have different learning styles and different strengths in the way we express our understanding. Our tools need to allow for this variety. Rosalie Pedder has produced a number of resource books which provide more information regarding learning styles and ideas for developing tools which allow for variation of expression. These are available through the International DTS Centre - www.ywamdtscentre.co.za; email: ywamdtscentre@mweb.co.za.

You can also find some helpful suggestions in "Made Practical" by Dawn Gauslin. This helpful guide is highly recommended to all who are pioneering and leading schools and is available in the resources section of her website - www.ywamconnect.com/sites/DawnGauslin/dgResources.

Whatever tool you develop, it is a good idea to '???[correction needed] it out' on someone before you give it to students. That way you can find out where it is unclear or not working in some way, and make the necessary changes.

When Should We do Assessment?

One answer is "???[correction needed] and often" Remember that a significant goal of evaluation is to help a student progress toward a positive end result. That means it is helpful to start giving them feedback as early as possible, so that they have time to improve before the end of the course. Of course, you can't assess something before it is taught! But there are a number of things you can start looking at early and continue to look at throughout the school.

Here it is helpful to understand that there are at least three types of assessment. The one we are most familiar with is the assessment that is at the end of a period of study, doing an assignment, application of skills or the end of a course. This final assessment is called summative assessment. It sums up all of the training and how well the student has done.

A second type of assessment is formative assessment. This is usually smaller '[correction needed]' or points of evaluation that show how the student is progressing toward getting a good result at the final, summative, assessment. It may form a part of the final grade but not necessarily. It helps the student know where they need to improve. Your regular one-on-one meetings (the U of N sets a minimum of at least one every two weeks and encourages weekly meetings) are one such evaluation point. Many schools meet together as staff three or four times during a lecture phase or outreach and evaluate where each student is up to in their learning and development.

The third type of assessment we will mention here is diagnostic assessment. This is done when a student seems to be having difficulty learning something so we want to try to find out why they are having this difficulty. There may be a language problem. There may be a learning difficulty. There may be something in their world view or life experience that makes it difficult to understand a particular concept. If they are performing badly we should not just accept that but try to see if there is some way we can help them perform better.

What Sort of End Result do We Need to Pass on to Others?

At then end of each course you will need to assign a grade (see Grading Explanation[citation needed]</noinclude>). This grade usually relates to a percentage result which is gained by assigning numeric value to that assessment. Assigning this numeric value can be easier in some forms of assessment than in others.

When your assessment consists of a set of questions you can simply assign values to each question depending on their relative importance or depth and then add up the numbers for the questions that the student gets right. This is a simple way but this type of assessment may not always be the most appropriate. For book reviews, research and other written work you can make up a checklist of key things you are looking for and assign values to each of these. This is an objective way to evaluate such work as opposed to just giving a grade based on how much you '[correction needed]' what they had to say.

When your assessment is looking at skills in action, or attitudes reflected by lifestyle and behaviour, one way to assign numeric values to this is to observe the student against a scale related to how well they have integrated this skill or behaviour into their work/ministry or lifestyle. This scale can range from not being able to do it at all (assigned a value of zero) up to performing very well in every circumstance (assigned a value of 10 or 5 or whatever depending on how many levels you include in your scale).

For the student we will want to give very clear, regular and specific feedback in one-on-ones that encourages them, shows them where they still need development or gives them new direction if they appear unable to develop sufficient skills or knowledge. The grade that we get from adding up the numbers can be a part of that, but it doesn't give the student the full picture. It should be our aim to know them well enough to give them specific feedback that can help them in their future.

How Have Others Done This?

It is always helpful to look at what has worked for others, both in similar schools and in different types of schools. Different people can come up with different great ideas and by sharing those ideas we all benefit. So we encourage you to check out other schools near you and perhaps contact a representative of the college under which your school is registered.

It is important to understand that a plan is not an absolute. It is a guideline to help remind you of when and how assessment can happen. However the plan needs to be flexible both in terms of assessment method and timing. Different students will be able to demonstrate competency at different times and through different means. Some may demonstrate competency in all the outcomes halfway through the lecture phase and others only demonstrate it in the last week of outreach. Treat the students on an individual basis and help them to work towards competency.

How Should We Conduct the Assessment

We want to give our students every chance for success while at the same time making a 'judgement'. That means:

  1. Take time early in the school to discover our students' background, educational experience (and possibly wounding), learning styles and anything else which may inhibit or help them to achieve success. Then working with them to overcome obstacles and maximise strengths.
  2. Explain all of the assessment clearly at the start of the school. Then explain each assessment activity again in the few days leading up to it and at the time of the assessment. Give them very clear instructions about how the assessment will happen and what end result is required.
  3. Make sure you organise the assessment well. Who needs to be involved? Do they know what they need to know? What resources and facilities do you need? How are you going to make sure they are available when you need them? When and where will it happen? Is someone planning to dig up the drains outside of your classroom on the day you are doing the assessment? Noise and other disruptions can be a problem, so we need to be thinking about how we can make sure that the environment is suitable for whatever assessment we are doing.
  4. Help the students to have peace about the assessment, encourage them through it and providing any clarification if it is needed.

Before the assessment actually begins, students may be nervous, so you will need to support and reassure them. You should consider their comfort and ways they can be put at ease.

  • Create a positive atmosphere and pay attention to the surroundings.
  • Ensure the trainees are relaxed and not apprehensive.
  • Allow time for them to settle in and feel comfortable.
  • Chat about something unrelated to the assessment for a minute or two.
  • Give a brief overview of the assessment process in terms that are easily understood.
  • Advise on the time allowed for the assessment.
  • Invite them to ask questions.

During the assessment provide encouragement and guidance where needed.

No matter which method and tool you use, you need to record the results in some way. That way you are not making a decision based simply on memory. It also means that you can show the results to others to either get their opinion or back up any decision you make.

If the assessment involves observing the student as they do something then the simplest, and usually most effective, type of record is some sort of check-list with additional space for specific feedback. List the important things you need to see them doing and key aspects of how they should be doing it and then tick each of these off as they are completed. Then write down any specific feedback comments that will help them in the future.

If the assessment is something that they are presenting to you on paper you can make notes on that paper of the grades and any specific feedback. You can also have a separate paper that records these, if that works better for you. Once again, having a clear checklist of the critical aspects helps you to make an objective and unbiased judgement.

Principles for Grading the Assessment

The key purpose in assigning a grade to each piece of assessment is to give an accurate measurement of what a student has achieved. That means the grading needs to be just, objective and lacking in personal bias. The best way to ensure this, as has been implied in previous sections of this document, is to have a list of key objectives related to each assessment activity against which a judgement can be made.

When the assessment tool is designed, part of the design is how we assign numeric values to the end result. We need to work through this very carefully at the design stage so that a clear measuring standard has been agreed by all those who are doing the assessment. It is like having a measuring tape that is used to assess how high someone jumps in a sporting event. All of the '{correction needed}' need to use the same measuring tape. You need to ask yourselves: what does it actually mean, in practical and measurable terms, when we give one person eight out of a possible ten and another person five out of ten.

All of these assessments will then be put together to produce an overall grade. For an explanation of assigned letter grades relevant to percentage grades see the article on 'Explanation' in this guide.[citation needed]</noinclude>

Often a struggle for staff as they grade a student's work is a desire to be merciful which can lead to assigning a grade which is not a true reflection of the work. This does not help the student. This is not true mercy nor truly loving your student. By making an accurate judgement the student knows when they need to work harder or need to do something different. To give an honest accurate grade to a student who is not producing work of appropriate quality is actually the best help we can give them. If we send a student out with a false judgement of their abilities we are setting them up for disaster. They will be under-prepared or unsuited for the work they will be doing. In this way they may cause damage and will also suffer themselves.

Going back to our high jump image, it's not enough that they get close to the standard. If they don't get over the bar then they haven't completed the jump. Imagine the chaos in the Olympic competition if the judges used that approach! Nor should we fall into the opposite trap of being so rigid in our thinking that we cannot see where our assessment or even the assessment tool may be flawed. All of us are fallible and have our own perspectives. We should be humble in our approach and work with others, each looking at the same student's performance or submitted work, so that together we come to as impartial an evaluation as possible. In this way we can recognise any bias or misconception that may be in us.

This will also sometimes allow us to see where a point of assessment may not be giving an accurate measurement. By looking at more than one student's results, sometimes something will become apparent about the test we have designed or the standards we have set which are inappropriate in some way according to the guidelines we outlined earlier.

It can also be helpful to show the overall assessment results at the end of a lecture phase and/or outreach to base leader, training director, or someone else who has appropriate understanding but is not on staff of the school. This gives a final validation of the results before sending the form C to the international registrar.

Justice and mercy work equally together to help a student to know their abilities, including the work for which they are best suited, and maximise them.

"He has showed you, O man, what is good. And what does the Lord require of you? To act justly and to love mercy and to walk humbly with your God." (Micah 6:8, NIV)

Another justice issue we will mention briefly here is that of plagiarism and cheating. Plagiarism is the copying of someone else's work and calling it your own. It is cheating and ground for academic dismissal from a course. For a more complete discussion of this topic, including how we should help students avoid this, refer to the article titled "Plagiarism and Copyright"[citation needed]</noinclude>.

Giving Feedback to Students

In our feedback to students our purpose is to be encouraging while at the same time giving them a clear picture of where they are up to in their development. The word "[correction needed]" can be loaded with negative meaning and fear associated with past experiences. As we assess our students and give them feedback we need to temper the use of this word, or avoid it completely, by using a better explanation. The feedback that we give to a person should fit into one of three categories.

  • Doing really well and/or ready to be released into ministry.
  • Improving but not quite there yet but with more time and effort they will be (this may be 'failing' grade at this time but also an encouragement to keep at it).
  • Not really gifted in this ministry.

In the second category we should be giving them advice as to what they should do next in order to take the next step in development.

In the third we must assure them that '[correction needed]' the course because they just aren't suited to this kind of ministry or field is NOT failure as a person. It is only a recognition that their abilities and gifts lie elsewhere. Everybody is able to serve God in one way or another. We can encourage students who are in this category by helping them to identify their true gifts and where they may get training or find a place where they can use them.

Assessment needs to be constant and continuous. It is not just about waiting until the end of the course and telling the student that they have either passed or failed. If we do get to the end of the course and they receive a failing grade it should not be a surprise to them. We should have explained what is required to meet the mark, been telling them the areas that they need to improve throughout the course and they will already know that they are not hitting the mark. We need to be helping the student to constantly evaluate their own progress and be giving them regular feedback based on our own assessments and observations. In this way they are affirmed in what they have learned so far and given guidance regarding the areas they still need to work on. If we do not do this, it is we who have 'failed'.

Evaluating Our Own Evaluations

As in everything we do, we ought to have an attitude where we are continuing to learn and continuing to improve. After we have given the assessment, provided feedback, completed Form C and sent the students on their way, it is helpful to sit down as a staff and look back over the assessment to evaluate it. The essential questions we can ask in any review are: what worked well; what didn't work well; and what should we do differently next time. We can review:

  1. The tools and methods we used.
  2. Could the students understand the tools and our instructions.
  3. Could the assessor understand the tools and how they should be used.
  4. The resources used.
  5. The environment.
  6. The assessor's skills in conducting the assessment and giving feedback.
  7. The decisions made.
  8. Were proper records kept.

Evaluation; App'd: ILT Kona 2005; 2005 U of N Reference Guide. Copyright © 1995 by YWAM/U of N; revised 1997, 2000, 2005. All Rights Reserved.