Mick Gwyther - Possibilities and Challenges of Peer Assessment
Links from Mick Gwyther - Possibilities and Challenges of Peer Assessment
- The Miro notes from this session
- Connect with Mick on LinkedIn
- Learning While Working Podcasts with Mick Gwyther
Automatic Transcript of Mick Gwyther - Possibilities and Challenges of Peer Assessment
[Mick Gwyther] 14:47:33
Thanks Robin and Sprout crew for extending an invitation to us all to participate in this as a consumer or the presenter.
[Mick Gwyther] 14:47:45
I figured out peer review and I think it's like with the most challenging POV opportunity I've had in the last couple of years was jamming back from a training session I did in July, which is that how the case might have in Ballarat Victoria, Australia,
[Mick Gwyther] 14:48:00
and I'm coming to this really pretty town I'm thinking of my ancient little bit hard.
[Mick Gwyther] 14:48:05
And then, it blew up right in the center of town which is really unfortunate because it was a cafe I could wait.
[Mick Gwyther] 14:48:11
Anyway, what that made was that to go to the garage, that would the car was booked in for to say, we won't come in with a car because they're blowing up.
[Mick Gwyther] 14:48:20
While I was there I started to think God now we're going to buy a new car. I've got particular benchmarks about the kind of car I want my partner has got other benchmarks and the two are almost irreconcilable so almost like within our organization.
[Mick Gwyther] 14:48:35
We've got a capability crisis. So, I stumbling around thinking oh my god this is going to be a nightmare getting through this process or getting a new car.
[Mick Gwyther] 14:48:56
And we got the car. So benchmarks are kind of really important in terms of like compromise and things but how I got interested in peer assessment which essentially is the participation of students participants, whatever.
[Mick Gwyther] 14:48:57
I spot a car that never seen before. That meant, all the boxes as, and my. And some of us we hadn't even considered.
[Mick Gwyther] 14:49:11
In the process of defining the benchmarks that that will use to assess each other, and also to create their assessment response.
[Mick Gwyther] 14:49:23
So, I used to teach vid in schools which is occasional training.
[Mick Gwyther] 14:49:30
Low certificate level credited training in multimedia schools will all come into this room kids run with the tools get changing out of the uniforms into, into funky t shirts and whatever.
[Mick Gwyther] 14:49:41
And then we we basically teach multimedia skills Photoshop video editing sounding whatever.
[Mick Gwyther] 14:49:51
Now was great because the competencies that are used that national were very discreet.
[Mick Gwyther] 14:49:54
How to do a sound file, have a video file, but then somebody come up with the idea that these competencies should probably reflect the world of work in the soft skill area.
[Mick Gwyther] 14:50:25
And so these beautiful discrete units became menagerie of team building and teamwork, I'm coming back on the train from the briefing thinking, How the hell am I going to get this get this going. And from there, I kind of come up with the idea of, they
[Mick Gwyther] 14:50:38
a step back, getting people to work together in groups kids, work together in groups, they got school I've shown, you've got motivated skills which I love. You got teamwork, stuff that that unlike so much. So my main role in teaching was around processes
[Mick Gwyther] 14:50:45
the skills almost took care of themselves, but the teamwork places where were really quite difficult to church. And so what that told me was that with these kids are acquiring these things now, and I have been debating, how does that what does that say
[Mick Gwyther] 14:51:02
about people already in the world of work, and we take for granted, the fact that people have a skills, and of course, the more you navigate the world of work, the more you realize that that is really quite, not the case.
[Mick Gwyther] 14:51:14
So, I've kind of my talk today a little bit about defining peer review and opportunities. Some of the examples that I've had challenges that have had along the way and I've mentioned a couple of days, and then we'll talk a bit about some of your opportunity
[Mick Gwyther] 14:51:35
And then we'll talk a bit about some of your opportunity now I'm using mirror here so I can see the full screen, quite sure the full screen, but the top there you should say I link, they can follow the presentation by going to that URL, and later on will
[Mick Gwyther] 14:51:43
participate in a couple of activities we can contribute your, your own thoughts. This is a kind of broad overview of what it is, what it all is all about working together to create build and apply knowledge.
[Mick Gwyther] 14:51:58
That might mean that it's a vocational training of China might need to intervene quite seriously in a world of work, maybe not so much a teaching or learning or project process.
[Mick Gwyther] 14:52:10
So rather than assessment being in point of teaching and learning.
[Mick Gwyther] 14:52:15
It's subsumed within the process of teaching.
[Mick Gwyther] 14:52:31
It's collaborative and potentially competitive. So with my students I put them into four or five different groups that are doing different things but there was a sense of competition and competitive to the extent that it did create a lot of fiction.
[Mick Gwyther] 14:52:34
And so you have to be quite mindful when you apply to group work projects of how you manage people. And, you know, their responses to their ideas not getting up not being listened to the kind of stuff we probably exposed to every day.
[Mick Gwyther] 14:52:50
of assessment to learners in a way that they can contribute to benchmarking that that will assess each other and provide their response to, you still have the capability statement or your competency statement that as an overarching thing, but the role
[Mick Gwyther] 14:53:09
of facilitator supervisor.
[Mick Gwyther] 14:53:14
Teacher, is to guide the level of those benchmarks as students create, to the appropriate level of statements, they are not aiming too high, or to like kind of getting it just just right.
[Mick Gwyther] 14:53:28
and that can be a bit of a challenge.
[Mick Gwyther] 14:53:31
That's a peer review work in all these applications of assessment in vocational learning clearly work products work products could be something as simple as building a cabinet.
[Mick Gwyther] 14:53:44
It could be creating a process. It could be creating a letter or could be providing creating a series of best practice principles for a set of professional behaviors, soft skills, as I mentioned earlier group work problem solving knowledge application,
[Mick Gwyther] 14:54:02
even work placement, which is fairly big in the health industry is where we play students into places like HK facilities health facilities to get some additional nursing additional experience that they can then relate back to their theoretical knowledge
[Mick Gwyther] 14:54:19
and also take off competencies there in the industry, so that when they graduate, they've met met the two of them.
[Mick Gwyther] 14:54:37
Communication obviously and things like terminology my experiences mainly when a word product group work stuff and knowledge building an application which is kind of central to those thing I'll give you a couple of examples of why.
[Mick Gwyther] 14:54:42
So I thought it's really important to convert a couple of terms here for you because the stuff that I've been doing really comes out of the vocational education training sector, and also higher education with a peer review is often a bit of an add on
[Mick Gwyther] 14:54:54
to particular things.
[Mick Gwyther] 14:54:57
And my focus around, sharing ideas around discussion, rather than contributing to a project that's probably less likely and things like steam which is, it's really quite large and learning and development, we might look at say trainers and assessors and
[Mick Gwyther] 14:55:12
I did notice somebody put up a comment and one of them your bullets that you can't relate to to the you know the training sector and the only but I think you can because of the way that we've embedded employability skills into what we do.
[Mick Gwyther] 14:55:26
So we've got capability statements now that exists for trainers, So they're exposed to that world and their knee deep in training and assessing for the kind of workplace employability skills that we think people have when they come into the world of work,
[Mick Gwyther] 14:55:42
not always the case.
[Mick Gwyther] 14:55:43
So, I'm trying to say we can make it a bit of a conversion and we think about peer review because my experience comes from that.
[Mick Gwyther] 14:55:54
There's two sectors.
[Mick Gwyther] 14:55:56
So in terms of what a good process might look like.
[Mick Gwyther] 14:56:00
It's often about an opportunity so opportunity could be a problem. It could be a project could be an innovation and challenge. So for example, my students had to create some for multimedia product.
[Mick Gwyther] 14:56:12
And I could choose a webpage I could show their video, they could choose what we call an audio file back then, which you might call podcast now.
[Mick Gwyther] 14:56:27
Or they might create a photographic multimedia presentation of some kind of that, the ultimate thing and skills to do that a quiet to scrape from the skills to come in together as a group, and just and determining what that projects can look like.
[Mick Gwyther] 14:56:38
So the second part is around developing performance benchmarks and expectations, and not necessarily in the back of your mind is to facilitate a you will have the capability of competency statements that you bring from your national accreditation, or
[Mick Gwyther] 14:56:54
the got embedded into university course or if you've got them as capability statements for positions within a particular industry on the task performance is based on these benchmarks and they give everybody who's participated in the opportunity to understand
[Mick Gwyther] 14:57:14
what what good practice is, once a best practice, but good practices competency education. You've really got it figured out. So, it's good base might be something else you know it's something completely different.
[Mick Gwyther] 14:57:29
And then validating that piece performance using those benchmark so a p contribution to recommendations for improving how they might work will flow from that and I'll give an example.
[Mick Gwyther] 14:57:40
A few years ago I was teaching these poor trainees, who'd been lured into a course at the university and road to get give them a better education at that time, and a certificate for an education and training.
[Mick Gwyther] 14:57:52
We should enable them to teach in secondary schools, but also assess Industry Skills in industry competencies within the schools and a lot of schools are registered training organizations are just bought in as a kind of a, a lackey really to help them
[Mick Gwyther] 14:58:10
with their present digital presentation skills so I teach them PowerPoint, basically for them to be able to present their project.
[Mick Gwyther] 14:58:17
So the first thing that I asked them, you know, they're only seeing one one PowerPoint.
[Mick Gwyther] 14:58:24
Apart from the electrician Fastpass.
[Mick Gwyther] 14:58:26
And so that was my, they hadn't really seen PowerPoint before, so I was the model, my car pounds when actually that right.
[Mick Gwyther] 14:58:34
So I asked him, you know, from this or your own experience with the world and design and signage and media instructions. What do you think makes a good PowerPoint and didn't take long for us to have 20 benchmarks.
[Mick Gwyther] 14:58:48
Just brainstormed. And then my process was to really facilitate that down into some things are achievable. So, guidelines for images tick size alignment and placement remember these people are tradespeople so they're kind of sense of design and balance
[Mick Gwyther] 14:59:07
and integration is quite hectic and quite quite good.
[Mick Gwyther] 14:59:11
And so, that may before it even touch PowerPoint. We had some ideas around what the output should look like and what we should be aspiring to.
[Mick Gwyther] 14:59:25
Then, we could change to that. Once we kind of got the mean and acknowledge their position we could choose to that. So give them a kind of an ownership in what it is that they're not only created, but also had their building, and also one of the things
[Mick Gwyther] 14:59:39
that a couple of them talked about was the importance of being able to look at the rule, and they had a break.
[Mick Gwyther] 14:59:47
And so that was kind of like a name Beavis. I'd benchmark that was difficult to evaluate and had to come down on justification in putting me in the center, no worries.
[Mick Gwyther] 15:00:00
If you were put into the center and have it a certain size and relationship with something else.
[Mick Gwyther] 15:00:05
It's, it's ambiguous So, why did you do that why, why did why did you come up with that. The other day, they went off and created their PowerPoints.
[Mick Gwyther] 15:00:13
They were much better than if I trained them I believe from scratch, and then they broke into teams and evaluated one not this work and gave me the visual feedback based on those 15 points that they kind of created.
[Mick Gwyther] 15:00:25
It was good, I mean my teaching role went from that big to that big. And then I became more of a coach and mentor as I gave them ideas, and maybe insight into how they might achieve what it is they want to turn that I couldn't quite get there.
[Mick Gwyther] 15:00:44
So it's kind of almost like a learning by doing fires, so you're going to understand the task that whatever that task is a present a PowerPoint.
[Mick Gwyther] 15:00:55
But it could also be something like facilitate an online session.
[Mick Gwyther] 15:01:00
And that could be running a meeting. It could be living a training program. It could be presenting a South a marketing product, or it could be providing individual support staff in my case it was training type China's.
[Mick Gwyther] 15:01:29
So I play help them in facilitating a learning online, which is part of a broader deployment that that we're doing. So, one of the issues, there was that competencies are pretty straightforward you know how to use zoom how to solve problems.
[Mick Gwyther] 15:01:36
How do we debate, how to provide feedback how to kind of nod your head and be engaging and make sure you're listening.
[Mick Gwyther] 15:01:41
But there was also a house like 12, different things that these people had to had to show that that we're doing, window facilitating and I don't know quite complex church.
[Mick Gwyther] 15:01:54
So by breaking them up into groups and giving them each.
[Mick Gwyther] 15:01:59
Three of those 12, wherever they get much more meaningful purchase on what those things would look like, and what sort of behaviors will show that they were, they were happening, and then they created those benchmarks for those things put them together.
[Mick Gwyther] 15:02:15
Suddenly, we've got a process where we can. Once they practice, we can give feedback to each other about with your meaning that benchmark you're quite the need to work on on something else.
[Mick Gwyther] 15:02:28
So working through a whole range of of understanding for them and ownership in one is either going to be assessed by other than giving them a marketing guide and say this is how we're going to return to assist you.
[Mick Gwyther] 15:02:44
They contributing to their own marketing guy but you're in the background still using that marketing guy that, that you that you would have had those kind of a bit of a slight of hand, a bit of a con.
[Mick Gwyther] 15:02:53
But it's kind of where you have to understand the competency, so you can pitch it at the right level not too high, not too long.
[Mick Gwyther] 15:03:03
So I guess it's too broad approaches that that I've kind of used so one is where you've got a competency statement, and you kind of brainstorm around those about what, you know, best practices they provide them.
[Mick Gwyther] 15:03:22
And you keep buying pack you brainstorm what they what they understand that that's really just process of interpretation so they understand how they're going to be graded based on the product that you've kind of been given, but could also be a project
[Mick Gwyther] 15:03:37
guideline as well from Bob. Second process which kind of works well to and is probably the first example I gave you around the PowerPoint was, we've got no guidelines or two with it so you either internal, and you come up with him as a good, so there's
[Mick Gwyther] 15:03:57
So there's no competencies I was teaching to in that unit cost about what a PowerPoint might look like it was whatever we come up with and it was up to them to come up with that particular thing that that's probably a much for your process but probably
[Mick Gwyther] 15:04:11
requires a bit more discipline to make sure that you're not. You're not bending the project beyond the parameters of what it is going to play so for example what you know we're intentionally about embedding video because none of them want to do that so
[Mick Gwyther] 15:04:25
we didn't worry about exporting the PowerPoint as a project to fall because it was outside the scope of how of what they had had to submit. So it is kind of important is going to choose what that might be.
[Mick Gwyther] 15:04:37
So in, Ellen day.
[Mick Gwyther] 15:04:39
It might mean that you know your benchmarks are internal or there might be external like could be you know industry benchmarks that you have whereas you know in the IT sector, you've got you've got competencies.
[Mick Gwyther] 15:04:52
Higher Ed got rubrics.
[Mick Gwyther] 15:04:56
It all kind of depends what what it is that's that's kind of guiding you and climbers seeing that Nick Nick's approach earlier. In, that's very similar kind of approach to kind of peer review.
[Mick Gwyther] 15:05:10
So, if you're able to log on to maintain media you're able to kind of participate for your phone here.
[Mick Gwyther] 15:05:16
You can ask a question anytime as well, or news that the border.
[Mick Gwyther] 15:05:23
You might have the board going it's awesome thinking for yourself like this this thing from my little spiel a what what what's your antenna saying about the advantages and disadvantages that you say, with a peer review approach to your own work.
[Mick Gwyther] 15:05:37
So when you log on to meet you made a comment to that code that's there.
[Mick Gwyther] 15:05:46
In, you will be able to, you'll see a form, and you'll be able to pop that in popular respond to and I'll drop into the chat as well.
[Mick Gwyther] 15:05:58
There we go.
[Mick Gwyther] 15:06:00
Great reading some.
[Mick Gwyther] 15:06:03
Oh, yeah. So, well that's a really good question can we use p benchmarks for formal assessment.
[Mick Gwyther] 15:06:09
It's almost like it's a two phase process with Michael at the time consuming one.
[Mick Gwyther] 15:06:14
So you've got you've got your benchmarks or your marketing guide in your formal assessment so what are you using is does inform how you brainstorm and consolidate the benchmarks for the, for the learners.
[Mick Gwyther] 15:06:30
It's a you releasing the same thing, but you're getting the students to define them, and then you're giving up a more formal grading at the top so they feedback and provide feedback to one another.
[Mick Gwyther] 15:06:43
And when you come over the top. With Your feedback is in addition to checking boxes that's a really interesting question.
[Mick Gwyther] 15:06:53
And because it gives opportunity to kind of fight against those terms of having worked in digital assist digital assessments.
[Mick Gwyther] 15:06:59
So the example I gave with the, the trainers who were learning to facilitate online they did their virtual presentation did half an hour presentation to show that they met the benchmarks, students, and the competencies for that unit in took that I could
[Mick Gwyther] 15:07:17
run a session on line that could provide feedback though.
[Mick Gwyther] 15:07:21
They could manage technical issues and they could be engaging. So all those we fed back after the end of every session in a form, and students gave instant feedback about whether they admit that it's benchmark or not.
[Mick Gwyther] 15:07:35
So it was a slider.
[Mick Gwyther] 15:07:38
You know, so, So some that didn't say it at all.
[Mick Gwyther] 15:07:42
And then a text to a text comment to, you know, how could they improve the performance.
[Mick Gwyther] 15:07:48
Because they in the competency have to give her feedback around how they might improve their performance.
[Mick Gwyther] 15:07:54
And that was data for them to help them do certainly does help build informal knowledge because it really does expose people to different points of view.
[Mick Gwyther] 15:08:05
It is time consuming and.
[Mick Gwyther] 15:08:09
And that, that's a really, really interesting point hadn't considered about consensus.
[Mick Gwyther] 15:08:14
And that's that. That proved to be quite true in some of the activities that I did.
[Mick Gwyther] 15:08:21
And it can really get lost in this way of to come back to, to you.
[Mick Gwyther] 15:08:26
You don't to be a bit of a binder not, it's not like you're kind of forcing your opinion on but you really have to keep getting paper to kind of clarify what it is.
[Mick Gwyther] 15:08:35
It's very learner centric, absolutely. It's a chance that moderation establishes shared understanding assessments and the rubrics, it, it takes it takes time it's quite once it's working yeah that's probably a good statement and it might mean that you
[Mick Gwyther] 15:08:48
That's a good statement and it might mean that you know, it could be a really good place to begin with, where you're, you know you're getting students to understand existing marketing garden rubrics difficulty arrive and, you know, hang on contest and
[Mick Gwyther] 15:09:03
create your own your own around a kind of a, you know, set of drums.
[Mick Gwyther] 15:09:07
For example, quality control.
[Mick Gwyther] 15:09:11
Collecting insights that often reflect this man yeah yeah and making sure that correct and reliable. So it is it is it is that bounce coin, you know the competencies you've got, and the ones that you that you want them to kind of come up with that can
[Mick Gwyther] 15:09:38
design assessment neither chain must have more trust in a piece time I tend to under assists. It may be too forgiving in their system, that is, that's a really great distillation of challenge, because certainly willing to students.
[Mick Gwyther] 15:09:45
One of the great issues was and one of the things they suggested that I did it again which I did was to regularly checking in with them in some way about the relative performance of one another.
[Mick Gwyther] 15:10:12
And we can also use some kind of tools to determine people's contribution to any final product particular to written bit more difficult in the digital space, like the creative digital space PV should be welcome to absolutely should be sufficient really,
[Mick Gwyther] 15:10:17
really good points he got that my that's fantastic I'm kind of considered anyways and I'll provide base to to Robin to distribute as well.
[Mick Gwyther] 15:10:27
Potentially subjective is, you know, a wonderful challenge.
[Mick Gwyther] 15:10:31
And I think that's kind of the tendency of most of us, isn't it in a sense that it is subjective and even as trainers, or as facilitators or supervisor interpretation of the competencies or benchmarks or capabilities time with us this is a subjective
[Mick Gwyther] 15:10:54
thing so I went to St. It's kind of that wonderful melting pot of ideas. And even if you're going to use the market guided become really kind of objective and helping students understand that is, is a challenge as well.
[Mick Gwyther] 15:11:14
And some of them just that was a really good point about being a lot tougher. It is tough.
[Mick Gwyther] 15:11:19
It's a tough thing to do.
[Mick Gwyther] 15:11:22
But what it means is instead of assessing your, your, you bring yourself assessing interpretation, accountable advisors and you're giving new opportunities to how you deliver that information.
[Mick Gwyther] 15:11:35
And if you've got a Holic a content.
[Mick Gwyther] 15:11:38
You can then adapt it to what it is the students that come up with.
[Mick Gwyther] 15:11:42
And then because they've given them at ownership in you. I found with both old and young students that their scientists to the benchmarks that contributed to actually research so little beta in their own kind of time.
[Mick Gwyther] 15:12:00
Well that is brilliant.
[Mick Gwyther] 15:12:02
Thanks for that. I've got another one for you.
[Mick Gwyther] 15:12:06
I'm in a bit shorter time and have I've kind of have a PP but I'm going to give you some examples but I'm actually giving you some of those examples as we go.
[Mick Gwyther] 15:12:16
So let's try this one.
[Mick Gwyther] 15:12:19
And so, this is an opportunity to for you to be kind of specific and think of a teaching learning assessment project in your work context.
[Mick Gwyther] 15:12:32
And think about what's, what's a pure opportunity but you can see in your own in your own work. They think okay.
[Mick Gwyther] 15:12:39
This guy's on the something maybe I could do this.
[Mick Gwyther] 15:12:44
Or you might say, this guy, not my kind of guy.
[Mick Gwyther] 15:12:50
But you might think, in your organization there's an opportunity for this so think projects. think new areas.
[Mick Gwyther] 15:13:02
Think challenges. And what products data, the other besides those that deal got an intimate a.com, there's a code. 3295 978 rubbish shirts there, and you can drop in your peer review opportunity idea.
[Mick Gwyther] 15:13:30
Yeah, I'd be nice but not having a chat what you kind of did their what you get it because it some.
[Mick Gwyther] 15:13:39
You're not even if you're not assessing it the process is still with your supervisor training yet.
[Mick Gwyther] 15:13:46
Yes, Yes that is some good examples of that.
[Mick Gwyther] 15:13:53
I've been working with hospital and once and they are teaching hospital and they they promoters clinicians to, to manage your roles, and suddenly they are isolated from their peers and appears Miss trusted them, and they have these little people spun
[Mick Gwyther] 15:14:25
to the epidemic online education practice and collaboration with educated community of practice. So, other educators know what that piece determine yeah that's that's that's perfect piece of spoken easy acceptable behind Yeah, you may know you're not
[Mick Gwyther] 15:14:27
all at a hospital if suddenly wife's on a barge, so bring those people together look at the challenges and brainstorm. How and provide some benchmarks for how they could build rapport and maintain trust, and also do things like have difficult conversations
[Mick Gwyther] 15:14:41
and if you're not made that then, you know, maybe we can look at how we might try and get her to help you, you know, get on board.
[Mick Gwyther] 15:14:50
Don't code for process of uh solution terms of output. Sure, yeah.
[Mick Gwyther] 15:14:55
So it's almost like project management focus and contributing to that.
[Mick Gwyther] 15:15:01
Not in terms of education but I'm a team that I work with, they kind of do that so we we get together and determine together what those kind of codes might be doing a scoring guide to key assessment.
[Mick Gwyther] 15:15:13
Yeah, so you could do that within a within a group of boys standard is one piece of. Yes, yes.
[Mick Gwyther] 15:15:23
Yep, yep.
[Mick Gwyther] 15:15:25
Standards regulations. OHNS are all kind of like you know, have you reaching for the coffee.
[Mick Gwyther] 15:15:38
And really, you have to kind of think about in that area you might store or five hours, so that you've got those, you know, awful clauses and sub sections.
[Mick Gwyther] 15:15:42
You gotta context to apply them true that's it's almost like I'm getting him to reach into scenarios.
[Mick Gwyther] 15:15:50
And then maybe the benchmarks and more around how you evaluated situation in relationship to their to their regulation.
[Mick Gwyther] 15:16:09
Based on what good practice interviews, and then doing some mock interviews featuring them and then talking with you need to make that just to say that.
[Mick Gwyther] 15:16:15
Tell them occasions yeah yeah I'll say this. We're in terms of interview, practice simulation.
[Mick Gwyther] 15:16:19
That is brilliant.
[Mick Gwyther] 15:16:20
Thank you very much. I'll probably gone a little bit over time let's pull it up.
[Mick Gwyther] 15:16:27
So I'll just pause, by saying thank you very much.
[Mick Gwyther] 15:16:35
I got chatting you got storing so straight from the script a little bit but I hope that somebody experiences mean useful for you and someone to consider when you, when you're looking at formal or informal assessments and thank you all very much for your contribution and thanks Robin, sprouts and the team for putting this wonderful event on.
