Links from Paul Kearney - The Myths Of Assessment

• The Miro notes from this session 
• Connect with Paul on LinkedIn

Automatic Transcript of Paul Kearney - The Myths Of Assessment

[Paul Kearney] 15:32:07
Okay, my background, probably three fold is a teacher, a, an owner of an RTI registered training organisation, and a consultant. And of course, most importantly, which will always forget as a student, particularly when we think about assessment, and then

[Paul Kearney] 15:32:27
we need to think about that.

[Paul Kearney] 15:32:28
So, I'm certainly not an expert in what we call normative based education assessment systems.

[Paul Kearney] 15:32:37
By the way, can we just have an aside, there have been listening to radio, and like doing a bit of a shout out to those people leave New South Wales Queensland who are very heavy, my flights.

[Paul Kearney] 15:32:52
We don't get too bad news from there.

[Paul Kearney] 15:32:54
And to the rest of you also have been in isolation for now for years I'm going to be cabinetry this from a bit strange. That's why. The other thing is a bit of a problem.

[Paul Kearney] 15:33:08
My ability lies manually so might be shuffling some papers and muck around with the mail so you might just have to be with me.

[Paul Kearney] 15:33:15
So, we are not an expert in measurement systems normative approach.

[Paul Kearney] 15:33:37
Sometimes I call it the home approach for American friends it's probably personal, the Washington is where system consists of putting people through the wringer to test them take them out hang on the one one which is the style.

[Paul Kearney] 15:33:40
They may have a relative brightness. A few people have spoken about measurement systems, sampling approaches, and so on, I think, Nick over made an allusion to the idea of you can all you can do.

[Paul Kearney] 15:33:54
Excellent and some of the best.

[Paul Kearney] 15:33:57
I'll just make 1.2 colleagues who do deal in the area of normal different systems measurement based assessment.

[Paul Kearney] 15:34:06
It's this, but if you're in a plane.

[Paul Kearney] 15:34:13
And the pilot got 90%, getting the examination or even 99% in the assessment. I still wouldn't feel safe. If I didn't know that 1% was landing in difficult conditions.

[Paul Kearney] 15:34:25
Now maybe in a workplace situation. It's not a matter of how good you are necessarily. The bottom line is, can you or Can she do it.

[Paul Kearney] 15:34:35
So that, and I'm certainly not an expert in online assessment.

[Paul Kearney] 15:34:43
Yet, my experience, or my focuses heavily in on the workplace. It's competency based can do do those versions of that. and the second entry.

[Paul Kearney] 15:34:53
And that really means collecting arrange a mixture of direct and indirect evidences.

[Paul Kearney] 15:35:00
So that's basically where I come from. and that's what I talked about with Robin.

[Robin Petterd - Host] 15:35:07
Paul was one of the people who helped with some of the technical reforms of the vocational education system in the 90s, what was really interesting I think about that is an object he saw some of the confusion around what competency based assessment was

[Robin Petterd - Host] 15:35:27
Couple things I would like to like to add.

[Robin Petterd - Host] 15:35:29
actually developed a number of books, one on collaborative assessment is peer to peer and self assessment we've been talking about working in a collaborative ways as well as one on one on workplace assessment as well.

[Robin Petterd - Host] 15:35:42
But to get to the core of the compensation Paul, what you think of the major piece of assessment.

[Paul Kearney] 15:35:55
So let's just back up a bit.

[Paul Kearney] 15:35:56
Go back to what I was talking about experience and my folks and you know obviously we're all coming from different realities, different contexts.

[Paul Kearney] 15:36:07
And I was listening to care or someone use the word authenticity. Right. She was using in the way all of us the word validity. So, you have to forgive me and we will have to translate some of that language between two different systems, online actually

[Paul Kearney] 15:36:21
talking about a criterion reference system in this case of work based one which is competency but.

[Paul Kearney] 15:36:29
So, library and very much about getting people to think in a reflective way. I'm not so much involved here, you Robin about practical approaches that will be a little bit more critical about what we're doing when it comes to the assessment.

[Paul Kearney] 15:36:48
And I don't mean theory will know Frank, who say there's nothing more practical than a good theory.

[Paul Kearney] 15:36:57
So, I just like to elevate our thinking a little bit above and sort of how to to reflect on the whole notion of assessment what it is, what it's meant to achieve what we think it is what it could be.

[Paul Kearney] 15:37:16
So, from an area

[Paul Kearney] 15:37:18
that most of their experience and initial experience the most indelible experience with assessment comes from our school and maybe later on which University, pay for technical institution, or whatever it is.

[Paul Kearney] 15:37:34
But we've all got that in common, regardless their existing context at the moment. So what does she do he just made a drop down menu.

[Paul Kearney] 15:37:42
So, just get familiar with the three things which come to mind feelings.

[Paul Kearney] 15:38:05
For your own good.

[Paul Kearney] 15:38:07
While in the dream.

[Paul Kearney] 15:38:09
Things which assume with you when you think about him of your experience as a student, when you have asked this question many times and I'm not going to guess what's in your mind, so I won't ask you to just do that for a moment.

[Robin Petterd - Host] 15:38:13
One word that comes to my mind.

[Robin Petterd - Host] 15:38:18
Interesting.

[Paul Kearney] 15:38:20
The ones which come to our classes, Many times, I think many people out there.

[Paul Kearney] 15:38:27
These will bring bills but I think the number one, feeling that people have when it comes to assessment is anxiety.

[Paul Kearney] 15:38:35
It's not a pleasant experience the most.

[Paul Kearney] 15:38:45
And I don't think it necessarily has to be an unpleasant experience but beyond that, I think uncertainties the other one because most of us in school system came up with that idea of a measurement basis from grading, where you sample bits and pieces of

[Paul Kearney] 15:38:53
the work that people in down the loan students.

[Paul Kearney] 15:38:57
So that creates sort of the guesswork what's on the paper, all that sort of stuff so uncertainty, the anxiety, generally to negative but the other thing is, it takes up.

[Paul Kearney] 15:39:09
And this is in a way around mix area, too.

[Paul Kearney] 15:39:13
It's about the idea of them and us mentality in assessment.

[Paul Kearney] 15:39:18
And as Robin said a moment ago my parents you said earlier on.

[Paul Kearney] 15:39:22
That word assess comes from two to sit with.

[Paul Kearney] 15:39:27
It does not mean to see an exam, can't make that but it basically means to sit with, in my mind, efficient assessment.

[Paul Kearney] 15:39:37
Effective assessment is a relationship between the assessor, and the cyst and other people who may be involved in it, including the peers. So, my premise on put to you is that where the learner is involved in assessment long just a demo, you know overall

[Paul Kearney] 15:39:58
basis you and move type thing where the learner plays an active, a prominent role in the assessment process, I would argue and I know from experience and that of others, that it can be not only highly effective, but really importantly we're we're concerned,

[Paul Kearney] 15:40:16
where we're nibbled to death by ducks robot pressure on us. Someone wants to be administration someone wants to do a bit of training you know it's always something wants your attention.

[Paul Kearney] 15:40:27
I would say that. The other important thing about involving people in the assessment process, services, people were season. Is it, it is efficient. It's not inefficient.

[Paul Kearney] 15:40:40
It's not inefficient. If you do it correctly, or if you do it with a degree of experience and understanding the and putting ideas of the collaborative approach.

[Paul Kearney] 15:40:51
And so I'm not using collaborative approach just a new idea of your system usually no idea of inclusion.

[Paul Kearney] 15:40:56
You're not doing it to the candidate to the learn that you're doing with them. That's not only mentality, but it's a very serious idea about the mechanics of how to do that were rather not think so I'm not criticizing, but rather not really, I think when

[Paul Kearney] 15:41:14
So I'm not criticizing, but we're rather not really, I think when it comes to the role of the learner, as just the subject of the, the, the undertaking.

[Paul Kearney] 15:41:25
So what do you think smiled to me to ask Robert.

[Robin Petterd - Host] 15:41:29
Yes, yeah that was all over the place.

[Paul Kearney] 15:41:34
Okay. The young Rafi I must be watching you a fair bit today Ralph, if you don't mind you, the spitting image of a friend of mine, you're a little bit more handsome that be the spitting image you know also congratulate you on your perseverance old guy,

[Paul Kearney] 15:41:51
not that hasn't been good that you've been very tentative I've noticed.

[Paul Kearney] 15:41:56
I think the my get make this just what I was going on about is, it's a one sided affair. It's finally the learner, who's actually being assessed know why.

[Paul Kearney] 15:42:07
Even the assessment process, you are being assessed as well, that your quality of your teaching is reflected in the result, the quality of the way that you organize and arrange and facilitate the system is being assessed as well so please understand that

[Paul Kearney] 15:42:26
I have a bit of a passionate about this idea that it's more than, it's just not a one sided approach at all.

[Paul Kearney] 15:42:35
So we're all in it together.

[Paul Kearney] 15:42:37
It's a collaborative approach, the more we're in it together, the more we share the task. I think the more effective, particularly the more efficient, it is and I know Do not put up with this stuff about people saying a collaborative approach is somehow

[Paul Kearney] 15:42:46
clumsy. Right, or it's more time consuming and so I would you say you're doing a wrong.

[Paul Kearney] 15:43:00
Wrong. You're not quite doing as well as it could be done.

[Robin Petterd - Host] 15:43:03
It's not too harsh is it not. If you don't, Yeah.

[Robin Petterd - Host] 15:43:10
So, I run it. Cool.

[Robin Petterd - Host] 15:43:14
Just got a question I just wanted to explore a little bit with this. So, I think I've actually heard that quite often learners will be on themselves with self assessment.

[Robin Petterd - Host] 15:43:28
So don't want an actual part.

[Robin Petterd - Host] 15:43:33
Additional assessor will be, which is sort of interesting in terms of inputs that sometimes helps people to raise the standard by them sitting there going, Oh, actually, you have to do your self assessment, and then the formal assessors just valid validating

[Robin Petterd - Host] 15:43:47
that, um, it's partly leads into into our next question. Sometimes people will worry about with collaborative approaches and giving more power to the learners, about the validity of it.

[Paul Kearney] 15:44:01
So, what's your thought on that on the, on the delivery of these types of shows.

[Paul Kearney] 15:44:16
I was actually going to think about that further down the line to. Jo, just to go back about what I was talking about, keep that thought Robin Don't let me lose it.

[Paul Kearney] 15:44:23
To go back, but it's not only an efficient way, the collaborative inclusive in the assessment, but it's really important in the, in the sense of it makes the learner reflective make some.

[Paul Kearney] 15:44:41
Take responsibility know where they are where they're going. Reflect on the journey through treatment.

[Robin Petterd - Host] 15:44:44
But as I also said the assessor, the teacher the instructor is also being assessed at the same time, they become a reflective thinking critically thinking, teacher, and they are as such a reflective worker, someone who's focusing on quality improvement

[Paul Kearney] 15:45:08
knowing with weaknesses, I had to deal with those issues, and hopefully the learner will become a reflective worker as well. So, once again I'm seeing you on the institution like many of many of you are but Robin, you answer that question again.

[Robin Petterd - Host] 15:45:26
You asked me that question again. Sorry. The the question about all over the place like the rain in New South Wales.

[Robin Petterd - Host] 15:45:33
I wish I'd known that about the fans by the way as well that's good to someone else's what someone's keeping an eye on what's happening outside.

[Robin Petterd - Host] 15:45:43
So, in terms of this collaborative assessment approach, how does it affect the Lucy.

[Paul Kearney] 15:45:48
Okay.

[Paul Kearney] 15:45:51
It's a sort of the old third rule, the one that people worry about probably most of all, is with the inclusive approach.

[Paul Kearney] 15:46:02
First we need to appreciate the value of a collaborative approach for enriching the learning, and it's a powerful form of learning I would know many people are out there, who've already in touch with the idea of assessment for learning, learning through

[Paul Kearney] 15:46:20
assessment if you're not google it assessment for learning.

[Paul Kearney] 15:46:24
It's a very powerful approach to learn, makes a learning visible and making the learning visible means that we can talk about it.

[Robin Petterd - Host] 15:46:32
You know, if you don't have criteria you know where you can enrich the actual learning experience itself.

[Paul Kearney] 15:46:52
The final, the final outcome.

[Paul Kearney] 15:46:56
Having multiple perspectives, including, not only the learn at the PS, and also other third parties, particularly in the workplace. Having those extra link says must help us make a better decision to just throwing it on.

[Paul Kearney] 15:47:14
But as for the our question Will you know the validity.

[Paul Kearney] 15:47:18
There is no reason why that formative assessment, what do you want to call it, where you forming the learning using assessment to do that contribute to the summative assessment which is an official summary of what the outcome is.

[Paul Kearney] 15:47:33
So that's where that question comes in. I only have one simple response to their idea of the validity of it. No one's telling us, you actually has to officially include the learners judgments in that final report or in that final decision.

[Paul Kearney] 15:47:56
Ultimately the responsibility of the last call is with the teacher with the assessor.

[Paul Kearney] 15:48:01
They're just type in the boot camp this other this other information.

[Paul Kearney] 15:48:05
So, really, and the responsibility is with the trainer, the assessor. So I think, you know, it's a misunderstanding I think of the notion that we somehow including those learner perspectives in the official process, it can contribute to it as I said,

[Paul Kearney] 15:48:24
it gives you those extra inside information that we may not otherwise have it can do that, and it's really important so contribute to it, but it doesn't, it's not an official part of your the unofficial can take off the qualification.

[Robin Petterd - Host] 15:48:42
So don't go anywhere new internet Roman. It really beautiful beautifully in Europe all

[Robin Petterd - Host] 15:48:51
the

[Paul Kearney] 15:48:55
economics into the depths that sometimes people perceive these types of processes, it's more complicated.

[Robin Petterd - Host] 15:49:03
And they do need more involvement, not learn what some other sort of misconceptions about financing assessment, sort of some sort.

[Paul Kearney] 15:49:24
when it comes to how you pronounce it let's get. So that's, yeah, we're talking there.

[Paul Kearney] 15:49:28
So in Australian context in the competency most context, we're talking about what's officially called reasonable adjustment. That means, you know, in basically just the learning process to allow people with different names to actually prove the competence,

[Paul Kearney] 15:49:44
where there are some regulations and guidelines about that within the, the Australian system about that.

[Paul Kearney] 15:49:52
But it comes to a point, I think where we've got this notion and it goes back that's why I asked you about my own experiences, it comes back to this idea that somehow assessment has to be tough.

[Paul Kearney] 15:50:06
It's testing, it's hot. There is no law which says it has to be hard to prove that you're competent.

[Paul Kearney] 15:50:14
Actually, can be done very easily, depending upon the method. Now, let me sort of a, think of some examples that one like the

[Paul Kearney] 15:50:25
one of my colleagues got a son who's got a bit of writing, it's quite difficult expressing himself with writing. So, but he's very bright forward.

[Paul Kearney] 15:50:38
Now, in that case, he should be able to be do that assessment verbally, many different ways it can be done virtually now.

[Paul Kearney] 15:50:49
Can, that's just not a reasonable adjustment, but it's also a preference. I should be rather than write down an answer to something I should be able to tell you, if that's got no effect on the actual competence that I end up.

[Paul Kearney] 15:51:13
problem with, allowing me to prove it in a, in a different way. So, it's a sort of thing.

[Paul Kearney] 15:51:16
And you can do that in multiple choices I know people who design should be able to turn some of your quite easily some of your model choices approach into audio one's quite easy.

[Paul Kearney] 15:51:29
It's just giving people the choice. It doesn't have to be hard, people would prefer to do it a pitch it for the right for me.

[Paul Kearney] 15:51:36
It's not about going through the winter, when it's about proving or disproving.

[Paul Kearney] 15:51:54
I think we can get into this mentality that we're there as a systems to disprove that there is another side of that coin but too much. But there are other areas, not only the, the method you use can cause make the assessments or had were texting unnecessarily.

[Paul Kearney] 15:52:04
We're texting unnecessarily. But some of the general sources couldn't be things like putting time limits on things.

[Paul Kearney] 15:52:13
Why, but usually because we've got a group situation. We're going through a sort of a treadmill of units or whatever so time is important to you and it's convenient to you.

[Paul Kearney] 15:52:25
It's not necessarily convenient to the learner. If the learner takes more time it may not affect the validity of the outcome at all.

[Paul Kearney] 15:52:33
classic case with my daughter, Ella,

[Paul Kearney] 15:52:39
Ella was a perfectionist.

[Paul Kearney] 15:52:41
In mathematics. When she was tested because she wasn't doing too well in mathematics.

[Paul Kearney] 15:52:48
It was founded. She was the time was for the issue not her capacity to deal with mathematics itself, because she was a perfectionist she took more time, because she took more time, she got list, done.

[Paul Kearney] 15:53:01
So it was very unfriendly approach. And this lead is actually a part of the competence, that you're assessing, it's invalid. It's an invalid thing time.

[Paul Kearney] 15:53:11
It's a matter of preference

[Paul Kearney] 15:53:14
practicality exams. What I'm asking is, in question reasons, building redemption processes and I know we have that quite a bit, you know to get it wrong cycle to get it wrong, you should be able to have more than one.

[Paul Kearney] 15:53:31
Yeah. And there's also remove the guessing game, as much as possible. This idea that we're not going to what's on the test. We're going to use your traction to multiple colors, multiple choice questions all sorts of things to make it happen.

[Paul Kearney] 15:53:46
What is the validity of the method. When you were well you know where I'm going.

[Paul Kearney] 15:53:53
Okay.

[Robin Petterd - Host] 15:54:02
Robin I sit on top of validity of your question about that. There's a different view, like, let's call it I'm actually going to ask you, the wrap up question Paul because we only got five more five more minutes left, so.

[Robin Petterd - Host] 15:54:21
Yeah, um, what's your parting comment on this, on the idea of collaborative assessment.

[Paul Kearney] 15:54:21
Also, did that.

[Robin Petterd - Host] 15:54:23
Oh, okay.

[Paul Kearney] 15:54:27
No one is that it makes sense, right from an effective and an efficiency point of view, and that is a big one. People don't understand that it's not an encumbrance, it's really, it's a tool of efficiency in the whole process, but I just like to say something

[Paul Kearney] 15:54:42
Robin when I'm taking over this thing here but I just like to say something about validity and assessment, because validity is God in assessment. You have to stick to what you say you are assessing what the state and sales.

[Paul Kearney] 15:54:57
So we, so for instance sustaining.

[Paul Kearney] 15:55:00
So in the workplace says, Identify workplace hazards within satisfaction.

[Paul Kearney] 15:55:07
It says identify it does not say, describe, so I can point to the workplace.

[Paul Kearney] 15:55:15
hasn't brought that that's legitimate that's valid assessment if that's what the criteria is as a data for workplace assessments. That's it. Right.

[Paul Kearney] 15:55:24
Describe hazards, like these are different competence, like can be assessed but it's a different criteria.

[Paul Kearney] 15:55:32
So you know we need to look very closely at the criteria, where we sit in any sort of assessment process will occluded that is look at the verb. The verb will always give you the clue, like, a classic one would be safety, security protocols on Databases

[Paul Kearney] 15:55:51
with IT organization sense optimist.

[Paul Kearney] 15:55:56
That. In those cases, if the criteria might say, us, or, or abide by the criteria.

[Paul Kearney] 15:56:04
Above the protocols.

[Paul Kearney] 15:56:07
But it doesn't say describe them so asking someone to describe them.

[Paul Kearney] 15:56:16
Now, I think the same thing is true for role play. Sorry, you must have your Bible to assessment.

[Paul Kearney] 15:56:25
In my book is validity, you assess what you're supposed to assess and know that the way you do it way interpret interpret the standards, right, affects that.

[Paul Kearney] 15:56:35
Now, the other thing I was gonna say a classic example not one of us role play for system. Fine, popular can be very effective in some. But once again, you're not necessarily only assessing what you think you are in that role play but you're also assessing

[Paul Kearney] 15:56:57
your ability to pretend. Some of the do that better to act.

[Paul Kearney] 15:57:00
So there are cultural things which are going down some cultures just do not like role play.

[Paul Kearney] 15:57:07
I know some cases particularly African ethnic groups have got a problem that age can be affected gender can be affected with.

[Paul Kearney] 15:57:18
Let's see where we go and,

[Paul Kearney] 15:57:21
you know, it's also the ability of the other planets, you know, you might get some person in there who is not up playing the other role if there are other roles involved, etc.

[Paul Kearney] 15:57:33
And so in mostly role play as with scenarios. The outcome is very much dependent, not only on the learner, the candidate is very much dependent on your capacity to design a damn good experimental method, which is not only good in a whole range of ways

[Paul Kearney] 15:57:55
but which hits validity on the head.

[Robin Petterd - Host] 15:57:57
So, where else you'd like me minutes we got about three more minute myth.

[Paul Kearney] 15:58:07
The, say something. Yeah, what would you like to have Would you like to wrap up.

[Paul Kearney] 15:58:11
I want to say something about knowledge because I know it's a big issue in the instructional design area. The digital based approach, and some. In, when I was back in the working in the early days in the national reforms, we came across a world where

[Paul Kearney] 15:58:31
most Chinese mass education related to testing knowledge.

[Paul Kearney] 15:58:37
And then, that is still so you've got an over reliance on on knowledge, it still hangs around in the institutions, and it's fair enough that's a reality of some of those things.

[Paul Kearney] 15:58:48
But, so, you had to swing away from knowledge, knowing what to knowing how, in the workplace to Performance Base.

[Paul Kearney] 15:58:58
That has got similar problems that sort of got the problem of sort of monkey see monkey do.

[Paul Kearney] 15:59:04
And I would just say this about that,

[Paul Kearney] 15:59:09
having knowledge, does not guarantee that you can do something, but I can guarantee you that not having knowledge is a probably guarantee that they can't do it if you if you know how to make.

[Paul Kearney] 15:59:21
So they do go together, and underpinning knowledge, and performance go together and it on occasion with discusses before, Robin about the importance of getting mixes of evidence overlap, all those sorts of things, just the last thing on the knowledge

[Paul Kearney] 15:59:35
is sometimes you don't have to ask, you can see knowledge, inherent in a performance, or in a product. In that case, stop Macarena kick it off when you see it.

[Paul Kearney] 15:59:47
You don't have to ask them, they don't have to talk all the time you can see the dolly inheriting things. So, you know, I'm sorry for preaching that they are saying is Robinson can teach you pretty

[Robin Petterd - Host] 16:00:00
cool. That's a lovely wrap up Paul and I think that's actually that last line around that the power of observation and seeing what people are actually doing is a really interesting way way way to wrap wrap up.

[Robin Petterd - Host] 16:00:15
I'm sorry we actually haven't had time to address some of the questions or go off on tangents that have been been in chat.

[Robin Petterd - Host] 16:00:24
If you think that people have sat down with actually don't miss such and such as this type of session as well. So, thank you so much for for for a great conversation on please my questions spot.

[Robin Petterd - Host] 16:00:39
Lots of distance discussion in you in things as well.

[Robin Petterd - Host] 16:00:41
I'm pulled into on LinkedIn, so it's probably the best way of contacting him.

And also if you just do a Google search on him. Paul does come up in enterprise learning website as well, with a new book coming

[Robin Petterd - Host] 16:00:57
me write a book on assessment digital learning so I think I think I said yes this time.

[Paul Kearney] 16:01:05
Really appreciate it. Good luck everyone, sincerely.