Raspberry Training

The good

So much of what we do in the workforce, we do as part of a team. Some jobs just can’t be done without a team working together. Whether it’s construction workers performing team lifts to move large items, or working collaboratively with a team to make decisions or brainstorm ideas. Working as part of a team is just something that we all have to be able to do effectively to be competent in our job roles.

Naturally, a lot of the units of competency we deliver in VET require the assessment of team tasks. Loads of units require assessment in a group setting, so it makes sense that group assessment is a common practice. We might assess a group of learners while they participate in a WHS meeting to discuss workplace hazards and risks, or we may assess two or more learners while they carry out a team lift or complete tasks together in a confined space. The list of ways we might perform a team assessment is probably endless.

So, group assessment is valuable. It is a great way to assess those skills that need to be performed in a group setting. But that is the only time we should use group assessment! If a person wouldn’t perform the task in a group setting in the real world, then group assessment just shouldn’t occur.

The bad

While I’m a fan of group assessment (done properly), there are just so many ways that group assessment can be problematic.

If I could nut the problems down into two main categories, its these two:

• Authenticity issues
• Reliability issues 

Authenticity

Let’s start with authenticity. Authenticity requires that:

The assessor is assured that the evidence presented for assessment is the learner’s own work.

CLAUSE 1.8 STANDARDS FOR REGISTERED TRAINING ORGANISATIONS 2015

ASQA clarifies these requirements within the Users Guide to the Standards for Registered Training Organisations 2015. On implementing the rules of evidence, the guide states the following regarding authenticity:

Ensure that evidence gathered ‘belongs’ to the student being assessed and provides evidence of that person’s skills and knowledge. For example, group assessments may not provide authentic evidence for each student involved in the group assessment. *

PAGE 54 USER’S GUIDE TO THE STANDARDS FOR REGISTERED TRAINING ORGANISATIONS (RTOS) 2015

So, let’s break that down a bit. Why might group assessment not provide authentic evidence?

Firstly, it’s a tough job for an assessor to be assessing multiple learners, against every single criterion on a checklist, at the same time. Not impossible, but it takes a keen eye to be watching everyone’s performance to the same degree at the one time. If the assessor isn’t doing that, how can they ‘be assured’ that the learner has met every criterion? That's a validity issue right there. If the assessor is assuming that the learner met the criteria because the group did well, then they are not assured that the evidence they have is from the individual learner. They don’t have authentic evidence.

Let’s say we’ve got a group assessment task where a team meeting is simulated to discuss and address WHS issues. We have 6 learners participating in this simulated meeting.

• 2 extroverts
• 2 introverts
• 1 person who hasn’t turned up to class or done any of their course work
• another who has been part of his company’s WHS safety committee for the past 5 years.

It’s probably natural that 3 of the people in this group are going to have a lot to contribute to the discussion. One can probably lead the discussion because they’ve been doing this type of thing for years. The introverts might know their stuff really well, but maybe they’ll sit back and let the extroverts do the talking. And our learner who hasn’t bothered with anything else in the course so far, he can just cruise along and agree with the discussions and agreements the other team members contribute to.

So at the end of the meeting, the team achieves its objective to discuss and document hazards and risks, and make recommendations for risk controls.

• Is everyone competent?
• What about those introverts? They didn’t contribute as much as the extroverts, does that mean they’re not competent?
• Our lazy student, he threw in some comments here and there – mostly based on what others were saying, but it looked good. Does he cut the mustard?

Or we could flip the scenario around. A diligent student does her very best in the meeting, has prepared notes, contributes to the discussion, but the rest of the group had no idea, and made no real effort. The team did not achieve its goals. Are they all not competent?

Never base an assessment decision on how the group performed. The outcome of the group is irrelevant in terms of assessment. It doesn’t matter! The only purpose of a group assessment is to set up a situation where individual learners can demonstrate their skills and knowledge.

We must always assess individual performance in a group assessment – regardless of the group outcome.

Tips:

• If an assessor needs to observe a group of 3 or more learners, they should not participate in a simulation or role play. They need every bit of their attention focussed on assessing the performance of every individual.
• A second assessor should be bought in when larger groups are being assessed. Split the learners up so each assessor can focus on a smaller number of students.
• In simulations, you may need to bring in an actor or third party to help facilitate the scenario. For example, in a group meeting, this person might facilitate discussion to ensure that every person in the meeting is given equal opportunity to contribute. This will allow every learner to have the same opportunities for assessment.
• Make sure the learners understand that they will be assessed on their individual performance.
• Ensure everyone is clear that the group outcome is not being assessed, and that assessment decisions may vary based on individual performance.

Reliability

Reliability requires that the evidence presented for assessment is consistently interpreted and assessment results are comparable irrespective of the assessor conducting the assessment.

CLAUSE 1.8 STANDARDS FOR REGISTERED TRAINING ORGANISATIONS 2015

Before decisions can be consistent, we must first ensure that the assessment is conducted consistently.

From the guide again:

Provide an assessment system that details the context and conditions of assessment for assessors (in order to ensure consistency in the application of assessment across different assessors and different student cohorts).

PAGE 54 USER’S GUIDE TO THE STANDARDS FOR REGISTERED TRAINING ORGANISATIONS (RTOS) 2015

Let’s go back to our WHS group assessment scenario. The assessment tool needs to have enough instruction to assessors so that we can be sure that the meeting simulation is set up and implemented the same way, regardless of who the assessor is.

Let’s say the assessment tool for our WHS scenario is not very specific in its instructions. One assessor, Jane, just sets up a simulation in the lunchroom, and instructs the learners to have a meeting to discuss workplace hazards and risks while she observes. That might work, assuming its work-based and the organisation has processes and actual hazards and risks.

Joe on the other hand might prepare for the task by speaking with the workplace Health and Safety Representative (HSR) and gathering information about recent hazards and incidents. Joe might facilitate the meeting himself to ensure all team members have an opportunity to contribute, and video the meeting so that he can review it carefully when making assessment decisions about individual contributions.

How likely are this pair to have consistent decisions? Not very likely in my view.

The lesson here – Assessment tasks, especially group assessment tasks, must have clear directions so that all assessors set up and implement the same version of the task. Without consistent implementation we’ll never achieve consistent assessment decisions!

Role Plays

Often role plays are set up for simulation purposes, for example a role play may be set up to assess learners’ ability to deal with multiple customers at one time. Sometimes, assessors might perform this type of role play in a group setting, with other learners playing the roles of customers or witnessing the role play interactions.

This is a big problem. By doing this, consistency in the application of assessment has gone out the window. Learner 1 is the first to be assessed, but by the time we get to learner 10, she’s just witnessed 9 other people do the task, and so her experience is not the same as learner 1’s. Reliability…gone.

I’ve written an entire blog on this, so I’ll point you there instead of repeating it all.

Do not use learners to observe or play roles in role play assessments that they themselves will undertake. ASQA is on the lookout for this at audit, so just don’t do it.

The ugly

Never, ever, put a group of learners together to complete tasks that were designed for them to do individually. For example, putting people into teams to complete a knowledge quiz. This is nothing more than setting up a plagiarism group!

One person knows the answer to question 1, everyone writes it down. A group of 6 students just needs 1 team member who is competent to answer all those questions.

This is literally the opposite of being assured that the evidence presented for assessment is the learner’s own work.

I gotta say this one really gets my goat. It’s really poor assessment practice, and needs to be stamped out across the sector. Luckily, this behaviour is easy for an ASQA auditor to spot. I don’t even get how anyone thinks they’d get away with this when everyone will have the same answers.

Back to the fundamentals

At the end of the day, the principles of assessment and the rules of evidence must underpin every assessment we implement.

When we fully understand and implement these fundamentals, conducting compliant assessment is not a hard task. Even if you’ve been assessing for 20 years, spend some time unpacking these basics to make sure you’re applying the rules and principles for every assessment you implement and determine.

Raspberry Training has just released a series of micro-courses that unpack each of the rules of evidence and principles of assessment. If you or your team members could do with a refresher, check out the link below.

https://raspberrytraining.thinkific.com/

Have a great week!

Coleen

* The quote from the standards “Ensure that evidence gathered ‘belongs’ to the student being assessed and provides evidence of that person’s skills and knowledge. For example, group assessments may not provide authentic evidence for each student involved in the group assessment“ is a contentious one sometimes.

I’ve often heard this statement as a reasoning for deeming all group assessment as non compliant. I’ve highlighted the ‘may’ above, because ‘may not’ is not the same as ‘does not’. Yes, it ‘may’ not provide authentic evidence if it is not designed and implemented properly, but group assessment is NOT non compliant in some broad brush way. It is high risk and if not done well, can easily lead to evidence that is not authentic. That is why ASQA made the statement. But if we follow the advice of the first part of this statement – we’re good.

Do it right, or don’t do it at all.

Originally posted 5th March 2022