Links from Robin Petterd - Assessment in Digital Learning

• The Miro notes from this session 
• Connect with Robin on LinkedIn

Automatic Transcript of Robin Petterd - Assessment in Digital Learning

[Robin Petterd - Host] 09:44:53
So, I'm going to start with a activity.

[Robin Petterd - Host] 09:44:57
And I would love to hear in chat, potentially what I'm exploring so what I'm exploring today my particular session is assessment in digital nine.

[Robin Petterd - Host] 09:45:08
I think it started with a activity in chat, please just share what you think is the most common tactic that you're using in your digital learning

[Robin Petterd - Host] 09:45:23
thought about doing this as a poll by the way everyone on the call. Quitting knowledge. All we do have someone doing some pre and post assessments

[Robin Petterd - Host] 09:45:37
reflections.

[Robin Petterd - Host] 09:45:39
Notice I'm picking up. What is maybe the things a bit different.

[Robin Petterd - Host] 09:45:47
Pay studies, quizzes, mostly.

[Robin Petterd - Host] 09:45:50
One person situation would probably the spot where we're going about 40% of people

[Robin Petterd - Host] 09:45:58
actually on the spot where they're actually in the chat area

[Robin Petterd - Host] 09:46:08
branches in our heads. Cool.

[Robin Petterd - Host] 09:46:14
Digital scope stock scorecards, we do have a little bit of diversity here in terms of things on Thank you everyone. It's actually interesting just what I'm really interested in exploring today is the notion that quizzes are one way of doing digital learning

[Robin Petterd - Host] 09:46:30
assessment, digital technology the offer hold on other different types of possibilities as well. And it's really those that the technology enables different possibilities, and we will explore some of those really I what what would what can be done with

[Robin Petterd - Host] 09:46:47
us. And I talk a little bit about my view on assessment for for for little moment.

[Robin Petterd - Host] 09:46:54
And, quite particular set of viewpoints as well.

[Robin Petterd - Host] 09:46:59
Humans a goal focused. One of the great things about assessment is it can give us a goal, so it can be in a spot where we can actually be really focused on something people complained a visit their indoor.

[Robin Petterd - Host] 09:47:20
is I want to go straight to the assessment. Yes, they do, they really know what they need to do to get through something especially if it's an education stock.

[Robin Petterd - Host] 09:47:27
So, actually making the goals really clear, actually makes it really easy.

[Robin Petterd - Host] 09:47:33
We are in a spot where it's about labs we work with the idea of learning ecosystems and systems for us are a mixture of houses money experience foundations knowledge supports garden as a people learning from each other, and then also the idea of streams

[Robin Petterd - Host] 09:47:56
of learning. but this is all held together by pathway.

[Robin Petterd - Host] 09:48:03
And I think the most powerful pathway you can give someone is an outcome that they're working towards to sit there and say, We need you at the end point to be doing dot dot, and that is that where that spot where you can sit there and say this is the

[Robin Petterd - Host] 09:48:17
outcome. This is what you need to be doing at the end. This is the assessment criteria.

[Robin Petterd - Host] 09:48:24
Then quite often, and probably one of the things I think to kind of repeat for the day and what Ray just actually talked to quite a lot about was the quite often when we think about assessment we think about a trainer or assessor measuring someone else

[Robin Petterd - Host] 09:48:39
does it actually powerful ways and what Ray was talking about of people assessing themselves, others and peers assessing and those sorts of sort of different and poker any heart Talk, talk to later in the midst of learning within the live interview.

[Robin Petterd - Host] 09:48:56
In actual fact, talks about developing collaborative learning.

[Robin Petterd - Host] 09:49:02
But hinted before. That's a lovely thing in a spot where we actually think about this word, assess rather than it being a mindset of measuring someone, but to sit beside to give feedback to reflect, to think about how you met how you measure up what you

[Robin Petterd - Host] 09:49:25
might do differently. It's interesting that that the origin of the word is this bizarre, so it's quite a different way of thinking about the philosophy of assessment NEP people need to do this next question.

[Robin Petterd - Host] 09:49:43
Share what you think is the key challenge with digital with, with digital assessment.

[Robin Petterd - Host] 09:49:50
In chat.

[Hub PS Boardroom] 09:49:52
I'm just going to check my phone from a feeling that my team might be texting me my background so.

[Robin Petterd - Host] 09:50:05
No, I might.

[Robin Petterd - Host] 09:50:11
Cool. So got people's into the format of the assessment.

[Robin Petterd - Host] 09:50:16
People focusing on completion more than anything else.

[Robin Petterd - Host] 09:50:19
Yeah, given feedbacks an interesting one is not

[Robin Petterd - Host] 09:50:25
technology can be limiting.

[Robin Petterd - Host] 09:50:38
interesting place so it's not it's not very often optimized for particular lovers

[Robin Petterd - Host] 09:50:45
I'll let you pick this up at the end by the way as well. Luckily, cutting the static

[Robin Petterd - Host] 09:50:49
who, by the way, natural patent lack of lack of equity Kim.

[Robin Petterd - Host] 09:50:53
Someone's just asked, asked you what you mean by that. So if you could pick that up that would be lovely divine. Yeah, it's interesting because essentially competing with more sophisticated forms of assessment they actually quite often involve more complicated

[Robin Petterd - Host] 09:51:09
sets of skill.

[Robin Petterd - Host] 09:51:13
Getting out to say something about coke. Okay.

[Robin Petterd - Host] 09:51:19
There's some, some of the tech the microphone for a moment.

[Robin Petterd - Host] 09:51:32
Cool. Yeah, people with learning disability or dyslexia. Yeah, having that sense of alternatives to certain to certain things and catering for that so you're in diversity that we have in our workplaces.

[Robin Petterd - Host] 09:51:46
The. It's interesting, I think some of the problems we actually have right across assessment is and run across learning actually if we actually copy on what models from education.

[Robin Petterd - Host] 09:51:57
This is one reason why she wanted to open with Ray for this particular event because raised mindset is really focused on what happens in workplace in the impact on business, rather than thinking about models that happen from education, and the, especially

[Robin Petterd - Host] 09:52:14
specially when you're working towards this notion of making an impact in an organization. The tools, they're using education around assessment, a very different.

[Robin Petterd - Host] 09:52:25
Education is quite often sitting there going, you're getting a qualification, you're being certified in something we're giving you UI spot where we're seeing the same you now have the knowledge to be part of this.

[Robin Petterd - Host] 09:52:43
Now knowledge is an important part of

[Robin Petterd - Host] 09:52:44
learning. It's actually a predictor for the first thing you need to notice something quite often before you can change your behavior.

[Robin Petterd - Host] 09:52:53
And that's a really interesting spot.

[Robin Petterd - Host] 09:52:55
But as a few people say they do do the assessment and nothing but knowledge assessments. And that's actually one of the real problems that it's essentially what in terms of performance.

[Robin Petterd - Host] 09:53:10
We're only taking a small slot of things. And we, it's not really a sufficient way of being, our major whether or not someone's making that impact.

[Robin Petterd - Host] 09:53:26
What's being spit expect to digital learning is changing our workplace and we're now expecting it to do different thing.

[Robin Petterd - Host] 09:53:43
The E learning with it was really built on compliance training, being cheaper, faster and easier for everyone.

[Robin Petterd - Host] 09:53:47
I didn't put it on the slide but quite often the learning in Australia was actually built in workplace The line was built around compliance and mandatory training in the finance sector.

[Robin Petterd - Host] 09:53:59
Regulators right across different compliance based industries are really looking for different evidence of compensating.

[Robin Petterd - Host] 09:54:09
I was thinking about this a little bit has to do with the ownership of this as well. In some in many organizations compliance and compliance is seen as something that the organization works with and say something like, a registered nurse in health care

[Robin Petterd - Host] 09:54:27
they need to approve the day keeping on learning and they keep an eye on competency, keeping their competency, so that actual does need to keep their competency together and high is actually a responsibility of the employee a shared quite often with the,

[Robin Petterd - Host] 09:54:46
with the organization as well as the shift as well.

[Robin Petterd - Host] 09:54:51
The thing and somebody needs to develop that it's actually in a spot where it's quite often not easy to get feedback and feedback is an important part of assessment report about that sitting sitting together and getting feedback is good assessment is

[Robin Petterd - Host] 09:55:06
not actually always cheap and easy. Equality does involve a different level, level of engagement and employee.

[Robin Petterd - Host] 09:55:23
Well I've been told what to do more, stakeholders don't want us to do more. The recent project and start to talk to potential potential client, about, and that's when I've been working with our stakeholders, stakeholders have actually specifically said

[Robin Petterd - Host] 09:55:33
oh we have a quiz at the end.

[Robin Petterd - Host] 09:55:36
She was invented and we and we don't want assessment that.

[Robin Petterd - Host] 09:55:51
Actually, I think that probably going to actually happen is the organizations don't want the impact because it's a major shift in the way they work, why they work with their stakeholders.

[Robin Petterd - Host] 09:55:51
But what they're wanting is something that's not a standard way of working, and now they are in a spot where they're not wanting to make too many hoops for other people, what people say this to me the boys don't want to

[Robin Petterd - Host] 09:56:08
see should have to do more. It's an interesting moment because behavior change is not easy.

[Robin Petterd - Host] 09:56:20
really strong powerful neuron links, the poor habits. So essentially a lot of behavioral change the actually changing our habits, and this is not easy.

[Robin Petterd - Host] 09:56:26
As humans were wired to

[Robin Petterd - Host] 09:56:31
I actually see that when people need to do more work. It's actually a positive. And it means that you're learning experiences will actually have more impact on things.

[Robin Petterd - Host] 09:56:44
I think the key to this is a term that I'm actually going to use a couple of times is.

[Robin Petterd - Host] 09:57:08
Blending your assessment.

[Robin Petterd - Host] 09:57:11
So, essentially, in learning we now think about all the different ways we can wake up with a lot, especially in the hose pandemic and in a spot where now we've become really good at seeing that going.

[Robin Petterd - Host] 09:57:30
This is an environment module. This is a live session, this is coaching, this is face to face and becomes it's really nice spot where you you in a, in a really different different world when you're thinking about what's really right.

[Robin Petterd - Host] 09:57:47
to make it branding assessment. Thinking about different ways of taking different types of evidence so work in the flow of learning, like what Ray was talking about before.

[Robin Petterd - Host] 09:57:58
And, and working in a different light.

[Robin Petterd - Host] 09:58:00
And I think this this had some really different subsets of possibilities.

[Robin Petterd - Host] 09:58:05
By focusing on you know knowledge quite often in assessment and digital learning around quizzes. We're actually taking what we've seen in the education and sectors and applying it back into, into workplace on.

[Robin Petterd - Host] 09:58:20
And I think this is interesting because essentially when someone say join a university course, there might be some placements now, but they don't actually have a workplace to work in a loving chat to hear what your thoughts on how workplace assessment

[Robin Petterd - Host] 09:58:36
compares to education setting.

[Robin Petterd - Host] 09:58:41
Now, I'm going to confess, I haven't been really following chat, I'm having trouble people are going to open so is this, things have been happening in the chat, other than lovely to see it happening, by the way, in the should be addressing I haven't had

[Robin Petterd - Host] 09:58:54
a chance it's because my eyes haven't been able to go across.

[Robin Petterd - Host] 09:59:01
Lisa that's a nice, nice one actually education is voluntary.

[Gaea McTíre] 09:59:09
Yeah. All right, I do have edit this guy McTeer I'm in Vancouver, Washington, and I'm speaking up because I'm in a heritage education doctoral program but also work in a corporate environment.

[Kayla Palombo] 09:59:21
So, I have the ability to look at both sides so nice.

[Robin Petterd - Host] 09:59:26
Cool. So what's your thoughts on this I'd love to hear.

[Gaea McTíre] 09:59:30
Basically when it comes to hire additional assessments.

[Gaea McTíre] 09:59:34
They really focus in on the knowledge that you're they're trying to impart, whereas in corporate environment as the former as Ray was saying earlier, impact business impact is so important.

[Gaea McTíre] 09:59:48
We're as, you know, so if you're talking about corporate, it's all about numbers it's all about business impact what kind of results can I get by completing the assessments and going through the training program, whereas in higher education environments.

[Gaea McTíre] 10:00:02
It's all about Okay, did you learn what I wanted you to learn did you learn the topic, did you learn the subject, you know, so it's a lot more knowledge based computer compared to results.

[Robin Petterd - Host] 10:00:15
Thank you. That's a lovely summary of what I think some of the some of the things he is some of the things is that that essentially that difference between yet Have you got Have you got the knowledge we're trying to impart compared to have we got the

[Robin Petterd - Host] 10:00:29
business, business impact as well.

[Robin Petterd - Host] 10:00:32
Thank you for taking the microphone it's great to hear someone else's advice.

[Robin Petterd - Host] 10:00:37
Cool.

[Robin Petterd - Host] 10:00:40
Business pressures.

[Robin Petterd - Host] 10:00:48
Okay, let's see, that's

[Robin Petterd - Host] 10:00:53
mandatory. Okay, and then move on lovely set of things, things things as well I'm just to reflect reflect one of my points on.

[Robin Petterd - Host] 10:01:14
the spot quite often, we can always apply it in workplace, what places we've got this spot where we can apply the learning to them so for it to be absorbed absorbed.

[Robin Petterd - Host] 10:01:21
Now, I'm clutch eyes to the people who are joining us in this time zone, from the US because I'm going to talk a little bit about vocational education system in Australia.

[Robin Petterd - Host] 10:01:32
It's some people in the room, may throw things at me.

[Robin Petterd - Host] 10:01:37
I'm the principal.

[Robin Petterd - Host] 10:01:39
I come from a vocational education background, I never really engaged with the assessment side of things completely. it got into and started working with learning management system because that there were only gone over this paperwork I have to do to

[Robin Petterd - Host] 10:01:56
cope with all this paperwork paperwork. Look, I'll just set up an online Dropbox, and that means the students can get their project work to me in a digital format.

[Robin Petterd - Host] 10:02:05
And because our teachings design it was all quite straightforward but. So that was it got started in digital learning by automating and digital I think my system processes.

[Robin Petterd - Host] 10:02:15
But I found this sort of complexity around the vocational educational really challenging and his own really strong system.

[Robin Petterd - Host] 10:02:26
And I think one of the things that happens is many people in Australia do this for certificate point and training and assessment, and then they see this gold standard, as actually what needs to be done with assessment, actually had a couple of clients

[Robin Petterd - Host] 10:02:40
who essentially operate pseudo register training organizations inside their organization, and complete with quality systems and measuring against the quality system system, just because they sit there and go, that's the standard, that's what we should

[Robin Petterd - Host] 10:02:54
be doing.

[Robin Petterd - Host] 10:02:57
But that's possibly not what we need to be doing in workplaces, to beginning the business outcomes, to make sure that people are actually moving forward with those results and those those numbers we're talking about bit about before is not not is a different

[Robin Petterd - Host] 10:03:15
thing to sit there and go, we're changing and shifting a process in an organization where the issuing a qualification.

[Robin Petterd - Host] 10:03:24
That's not to say that some of these, some of the principles are useful.

[Robin Petterd - Host] 10:03:38
And the two principles, I'm going to work with today are essentially authenticity that sense that our assessments, need to be really catered exactly to what we were doing in our, in the workplace.

[Robin Petterd - Host] 10:03:47
And they need to be reliable, and it's been a spot where they actually even work with them from multiple people, and that can actually be in a spot where they get the same measurable results.

[Robin Petterd - Host] 10:04:00
This is one reason why people actually quite often love with it because it feels this so measurable, and so it's so easy, easy to work with.

[Robin Petterd - Host] 10:04:10
Now, Essentially, just to flesh that out a little bit more.

[Robin Petterd - Host] 10:04:16
By authenticity. I mean, we're not knowledge so assessing the work tasks and getting multiple bits of evidence, and using multiple, multiple tactics.

[Robin Petterd - Host] 10:04:30
In terms of reliability probably needs to be over time, and observations.

[Robin Petterd - Host] 10:04:46
It's actually one of the things we can learn from education because quite a good education, good education programs will focus on your assessment first and build up from that.

[Robin Petterd - Host] 10:04:51
No question how many people that as assessed as learning designers in the in the group, sit there and actually write their assessment test.

[Robin Petterd - Host] 10:04:56
If we sat down with okay.

[Robin Petterd - Host] 10:05:09
So, this don't go to business think this is what our business impact is, this is what the behavior outcomes, we're going to do we need people to get our business outcomes are we going to measure, measure that just shifts.

[Robin Petterd - Host] 10:05:12
How are we going to measure, measure that just shifts. And how are we going to assess that shifts what you then do in the actual learning and the prep space for people.

[Robin Petterd - Host] 10:05:20
Now, getting back to core digital learning.

[Robin Petterd - Host] 10:05:24
I use the word digital in a fairly expensive why.

[Robin Petterd - Host] 10:05:31
And we are we talking about things that will not be particularly about learning management systems or offering total.

[Robin Petterd - Host] 10:05:41
I think about digital as our technologies we have around us in our workplaces, not just your learning management system. And this becomes a little bit gives you a lot more flexibility to think about different things in a different way.

[Robin Petterd - Host] 10:05:57
So, at the core of also when it comes to selecting what your types of digital learning assessment strategies and tactics might be. I think it's about risk.

[Robin Petterd - Host] 10:06:11
So, this is the list of general high level, what I think about the continuum of digital assessment at one end, we've got something that's very low impact very easy for everyone to do a knowledge points.

[Robin Petterd - Host] 10:06:32
And this should probably be only used in a spot where you're working with low risk behaviors, things that are going to have a lot of impact on your people on your on your business.

[Robin Petterd - Host] 10:06:47
And then at the completely, the other and we actually have a spot where you got evidence and meeting data and observation, because it's a high risk behavior.

[Robin Petterd - Host] 10:07:01
Safety Training and got better at this in terms of business, but where if you actually in a smart way you are had to have people doing high risk behaviors.

[Robin Petterd - Host] 10:07:11
You probably really do need to be in a small way you actually observing what they're doing less so around some of the other types of magic traits, really interested for multiple reasons in the way we're rethinking what used to be equity around us to be

[Robin Petterd - Host] 10:07:31
a mandatory around, equity, and discrimination, how we're now seeing this as a sense of sense of a bigger picture about the fact we want to build out workplaces, to spaces and places where people want to belong and want to feel included and want to be

[Robin Petterd - Host] 10:07:48
part all, they don't want to be excluded from things. So becomes less about the legislative requirements and more about the environment. We build for our people in our organization.

[Robin Petterd - Host] 10:08:07
The classic. Is that a mandatory has been classically, allow me to quit and moving forward is different, the core part of keeping our people might be unsafe.

[Robin Petterd - Host] 10:08:14
Is this really in a spot where it's probably not something you're going to be using for quiz anymore, probably in a spot where you're moving into a very different type of assessment strategies for these types of things.

[Robin Petterd - Host] 10:08:30
So Amanda a little bit of an explanation for the rest of the session on each one of these, these types of strategies strategies.

[Robin Petterd - Host] 10:08:41
Dive into them. Some of them more than others and some of them lists that well.

[Robin Petterd - Host] 10:09:01
So that's an interesting one. It depends on the nature of it. So I'm actually going to pick up a example that talk a little bit about later in the podcast.

[Robin Petterd - Host] 10:09:02
How does one develop the risk impact.

[Robin Petterd - Host] 10:09:08
It was the medical header required legislative requirements, with a letter that goes out to their members every year.

[Robin Petterd - Host] 10:09:18
That was one legislative requirement that if they didn't meet, they would be find that one legislative requirements names.

[Robin Petterd - Host] 10:09:30
At changes to their processes to get that one letter out, and the process change those labels requirements sounds like a lot like a low risk activity to start with.

[Robin Petterd - Host] 10:09:45
But when you actually put it in terms of in terms of it changes the dentist is always apparent go. Actually, this is quite a high risk to the organization will need to get this change.

[Robin Petterd - Host] 10:09:55
Exactly right, we need to we need everyone to understand actually what it is and what the what what they're impacting.

[Robin Petterd - Host] 10:10:04
So it's about that the decision of outsourcing they're going, is this.

[Robin Petterd - Host] 10:10:08
What's the impact on the business if we get this wrong.

[Robin Petterd - Host] 10:10:12
What's the impact on our people. If we get this wrong.

[Robin Petterd - Host] 10:10:18
That's what what's lovely about the, the shifts that's happening safety is essentially gone from being a mandatory to a whole, actually we need to care and make our workplaces are safe for people.

[Robin Petterd - Host] 10:10:30
So it's that thing about thinking that's true.

[Robin Petterd - Host] 10:10:34
I'm, I'm probably going to do a follow up session to this, that's about those early discussions that you have with your stakeholders.

[Robin Petterd - Host] 10:10:44
Around assessment and kind of get more buy in for higher level assessment and thinking through what does that does types of risk think that we are as well.

[Robin Petterd - Host] 10:10:54
Okay.

[Robin Petterd - Host] 10:10:58
So, I knowledge quiz at the end at the at the bottom, bottom in the back continuum, is not very authentic but it does actually work in each one of these days I'm probably going to highlight what's that standard tool, we're going to work with as well.

[Robin Petterd - Host] 10:11:14
Now, I'm starting to switch into it into time when Patti Shanks was meant to be talking so I'm actually going to share some of what I've worked with Patty shakes shakes about doing knowledge quizzes and multiple choice questions.

[Robin Petterd - Host] 10:11:29
First of all, being your learning objectives, need to have action verbs.

[Robin Petterd - Host] 10:11:37
If you have knowledge in your learning objectives, you're going to end up with knowledge, assessments.

[Robin Petterd - Host] 10:11:45
If you once you get your learning objectives to a spot where you've got action verbs in then align your questions directly to those action verbs and people practicing those decisions.

[Robin Petterd - Host] 10:12:02
And

[Robin Petterd - Host] 10:12:12
I'm just trying to think about a nice example from something we've been working, working on recently.

[Robin Petterd - Host] 10:12:20
Bloom's Taxonomy yeah it's about getting getting a little bit higher up the adoptable opposite Bloom's taxonomy.

[Robin Petterd - Host] 10:12:26
And it's this is actually a well step process.

[Robin Petterd - Host] 10:12:31
So when the client came to us it was the whole, the learning objectives was to have knowledge of the 12 step process but don't don't.

[Robin Petterd - Host] 10:12:44
We didn't realize that, to AA, be able to follow and apply the 12 step process process.

[Robin Petterd - Host] 10:12:58
So that 10 minutes, the multiple choice questions were quite often scenario based questions about a single scenario that was in a guy. So you've got to this particular spot in the process, what what do your actions need today.

[Robin Petterd - Host] 10:13:15
By the way, none of this was essentially a spot where they could refer back to their process guidelines to nothing, and then documents as well so awesome died out having to remember this, but they were just having to show that they couldn't could apply.

[Robin Petterd - Host] 10:13:33
One of my questions when this knowledge scene comes up is, if an employee has the knowledge. What does that mean they need to do.

[Robin Petterd - Host] 10:13:43
What decision to them and to make classic questions that come from Catherine was it impact modeling work.

[Robin Petterd - Host] 10:13:53
So, it's a spot where it's actually yet, quite often easy to do if you start to think about the vision.

[Robin Petterd - Host] 10:14:02
Now, improving your quick quiz questions. I'm not sure if Patrick's

[Robin Petterd - Host] 10:14:09
really interesting moment recently.

[Robin Petterd - Host] 10:14:12
Patrick I hope you don't mind me mind me sharing that sharing this Patrick and I were talking and he was talking about some quizzes, it was done recently wasn't particularly happy happy when it was a really obscure area.

[Robin Petterd - Host] 10:14:39
And then the ones I did had no idea at all. Oh that she answering see that sidebar.

[Robin Petterd - Host] 10:14:45
and I start to sit there and get them right and I was like, no, not only just a scientific area with this assessment, because like I said there will long response right.

[Robin Petterd - Host] 10:14:46
So, just be really careful about actually what you're doing with some of these things. It's very easy in our, in our assessment work to pick a spot where we keep on making these types of mistakes and going back to them.

[Robin Petterd - Host] 10:15:05
Cool.

[Robin Petterd - Host] 10:15:08
The.

[Robin Petterd - Host] 10:15:08
One of the things we can also do to improve back on the quizzes as well as this type of putting more effort into into writing them is actually what I'm thinking of as Markram assessment.

[Robin Petterd - Host] 10:15:21
So, essentially quite often with assessments and all these quizzes people do the module online. And then they're in a spot where they do the quiz afterwards and then that gets certificate at the end with this notion and idea of micro self assessment,

[Robin Petterd - Host] 10:15:37
it's about separating the learning and learning and assessment over time.

[Robin Petterd - Host] 10:15:43
There's lots of things someone actually talked about spacing things over a period of time.

[Robin Petterd - Host] 10:15:49
We aren't spot where that that's quite often we think about in terms of learning but it's also be employed in assessment.

[Robin Petterd - Host] 10:15:59
So I'm using this example from from SEO Stevens podcast, the ball again for this one, we're actually talking about data analytics and storytelling.

[Robin Petterd - Host] 10:16:06
So, think, but to be able to still tell stories

[Robin Petterd - Host] 10:16:13
with data he needs.

[Robin Petterd - Host] 10:16:28
Three times after the learning first time three weeks after the another three weeks off afterwards. So what they were doing was the net guy had people being able to retain and lie this into their work into the, into the future.

[Robin Petterd - Host] 10:16:33
And what they've done with people is that this legislative change is that separated the assessments and the learning. So the assessment actually happened.

[Robin Petterd - Host] 10:16:44
And what was interesting from a data point of view, they actually found that in that first assessment.

[Robin Petterd - Host] 10:16:52
A huge percentage of people were getting some steps that set of questions wrong.

[Robin Petterd - Host] 10:16:56
And then when they drilled into that and actually realize that actually wasn't a problem with employees but there's actually a problem with the learning experience.

[Robin Petterd - Host] 10:17:09
It wasn't quite clear, and that then they rework that and then rerun out training to remediate. Well, I think this is really in spot where you need to stay with knowledge and quick questions.

[Robin Petterd - Host] 10:17:19
This shift to micro assessment, could be a powerful strategy.

[Robin Petterd - Host] 10:17:24
In chat. And click back some suggestions about when you think that stuff and micro assessment might be useful, and I've noticed there's a few things that come back up in chat as well so I'll try to get my head back into as well.

[Robin Petterd - Host] 10:17:43
Cool formative assessment, think Joe Cox talking about that particular online, later.

[Robin Petterd - Host] 10:17:58
In the end of each topic.

[Robin Petterd - Host] 10:18:06
knowledge chicks sure out.

[Robin Petterd - Host] 10:18:06
And she interesting because sometimes as people are talking so maybe one of the mindsets about this is almost that formative assessment, especially when I'm thinking about with this it's actually separating things across time as well, that you might do

[Robin Petterd - Host] 10:18:22
something as small as in one day. And then you come back to the next time.

[Robin Petterd - Host] 10:18:37
She interesting Monica and pick up on that.

[Robin Petterd - Host] 10:18:43
We've actually seen a couple of times where people will actually be more able to engage, they're more likely to engage with pre work and post work is to do is the thing that people will say that they all got knowledge on by now knows how to use the knowledge

[Robin Petterd - Host] 10:19:05
If you shift your expectations to this whole, actually, there's work to be done in the workplace to embed what you're doing. You can get get more buy in, and people are well.

[Robin Petterd - Host] 10:19:20
Yep. Cool.

[Robin Petterd - Host] 10:19:22
David, yeah, essentially, that's what I can do i do as well I've got, I've got an extremely good short term memory. My classic example is I got 1996 98% on my diving course.

[Robin Petterd - Host] 10:19:38
And then I went into the ocean the next day and I was incompetent was quite scary incompetent because of my nose with women, and I stopped diamond, but on paper, I was always starts June, in the pool.

[Robin Petterd - Host] 10:19:52
I was actually really quite good as well, but I just really didn't couldn't relaxing in the open sea, and to me it's a really good example where, if you've actually got that sense of what you're being asked to do is not a particularly good thing.

[Robin Petterd - Host] 10:20:07
Now, the next strategy, which is a way of increasing your your

[Robin Petterd - Host] 10:20:14
your delegated sensitivity.

[Robin Petterd - Host] 10:20:19
A lot is using your quiz thoughts a bit video to build scenario based questions. You can do this judging using a standard quiz tools as well. A couple of people's hinted at graduates in our hands.

[Robin Petterd - Host] 10:20:32
I'm not going to actually dive into this for a moment, because. Coming up after me is Jenny Saltzman, and she's going to actually dive into this a lot, she's done some really interesting work around eating scenarios process, perhaps that process as well.

[Robin Petterd - Host] 10:20:51
The next level of complexity is written assignments.

[Robin Petterd - Host] 10:20:56
So, it's some sort of spot where borrowing from agitation again, that you're asked to do an SI. So I don't know if that's the best time.

[Robin Petterd - Host] 10:21:14
So, even what some of Ray was talking about was using and using their online platform, doing recent work. But, reflecting on things. This idea of a piece of evidence is maybe a stimulated thing is quite us quite useful.

[Robin Petterd - Host] 10:21:21
Best mindset for to talk about.

[Robin Petterd - Host] 10:21:34
Time and texts a barrier. So some of the ways we can we can think about different types of activity.

[Robin Petterd - Host] 10:21:43
The more like what we're actually doing in a world, but it's things like reflections like what Ray was talking about planning for change.

[Robin Petterd - Host] 10:21:51
And then strong knowledge base work doing the work report writing a stimulated activity to right where a writer a writer report can be a really powerful way of working, or if you're getting preparing people for a spot where they're doing work in policy

[Robin Petterd - Host] 10:22:11
writing, getting people to actually write policies and then giving them feedback on things.

[Robin Petterd - Host] 10:22:19
Getting a few things to get over the, the time to time, challenges with this feedback rubrics extensively you then education.

[Robin Petterd - Host] 10:22:31
We should have screen grab of one at one of these score workplace where essentially instead of doing recent feedback. What actually happens is there's a grid, when people are actually the obsessive and just sitting there going, you don't do that well

[Robin Petterd - Host] 10:22:44
we've done that well done that well and it's just really quite quick and easy to be able to give them feedback.

[Robin Petterd - Host] 10:22:51
Um, he is getting feedback. The process of giving feedback and then she also, and then self assessment or many, and then maybe your six months, as well.

[Robin Petterd - Host] 10:23:01
I think this will probably come up later in the day, if you actually spot where it is a high risk assessment, working towards a qualification, or working towards someone doing a new type of job that actually has risk and responsibility, you might find

[Robin Petterd - Host] 10:23:30
evidence.

[Robin Petterd - Host] 10:23:31
Seems people actually starting to do things live role play

[Robin Petterd - Host] 10:23:39
eventually caught up and people sit there and go in virtual classrooms, we can't really do many types of types of assessments.

[Robin Petterd - Host] 10:23:49
You can you can replace a client you pre prepared Dimmick was doing assessments and feedback on doctors in rural remote Queensland, using Skype.

[Robin Petterd - Host] 10:24:02
And that was in a spot where that we're doing a live role plays with someone sitting back in Townsville rather than having to having to travel pretend for 10 hours to get there.

[Robin Petterd - Host] 10:24:13
We can this be a really powerful simple way of doing things.

[Robin Petterd - Host] 10:24:18
Evidence as well as projects projects are really useful when something's new to an organization. It's whenever new practice a new way of doing things, not all people in a spot where they can undertake a project that's when that sort of sentences something's

[Robin Petterd - Host] 10:24:36
new, you can sit there and go okay let's plan this let's make a change, let's do this different differently.

[Robin Petterd - Host] 10:24:43
There's lots of things where you areas spot where you want things to move forward. A lot of our upscaling that's happening in digital and rethinking about human skills in the workplace and that could fit into this, this new ways of working for being well,

[Robin Petterd - Host] 10:25:02
so the projects can be a really powerful thing to take a bit more effort on paper.

[Robin Petterd - Host] 10:25:08
If it says a practice that actually existing

[Robin Petterd - Host] 10:25:14
work samples examples of people actually doing things is probably one of the most powerful things we've got videos quite often extremely useful for this.

[Robin Petterd - Host] 10:25:27
So, working with someone at the moment, who, during the pandemic.

[Robin Petterd - Host] 10:25:33
They were working in Kenya and working in the technology education.

[Robin Petterd - Host] 10:25:41
And basically sat there and like I can't, we're locked down pants can observe you, you're still working. Can you send me a video of you doing Dr.

[Robin Petterd - Host] 10:25:53
And that was a really powerful way of being able to capture the actual evidence of someone actually doing something.

[Robin Petterd - Host] 10:26:01
And that's in a spot where they're really, really nice way of being able to collect things.

[Robin Petterd - Host] 10:26:06
So, you're in a spot where you can sit there and say to people.

[Robin Petterd - Host] 10:26:11
So, this is a new behavior, we want you to have three examples of when you've done this new type of process, and then put those into an assignment Dropbox and once they're there.

[Robin Petterd - Host] 10:26:26
That's great.

[Robin Petterd - Host] 10:26:27
In terms of risk as well.

[Robin Petterd - Host] 10:26:32
You can do things like sit there and go, actually we want you to upload times. And then, when you've uploaded them.

[Robin Petterd - Host] 10:26:42
It was complete. And then do spot board at something, so that you say to people. Look, we're not going to give feedback on everything, and we're not going to check everything, but we're going to give spot order.

[Robin Petterd - Host] 10:26:53
This is not really high risk, completely a high risk activity because in that spot that sort of in the middle, the switch and where's the pink as well.

[Robin Petterd - Host] 10:27:06
So the next thing in terms of works that all data.

[Robin Petterd - Host] 10:27:11
During the spot where you actually collecting a metric on someone. So you're sitting there going, you're, you're a customer service person working in a call center.

[Robin Petterd - Host] 10:27:27
Your, your MPs score has gone from dot.to dot dot because of what because of these changes.

[Robin Petterd - Host] 10:27:39
The Chroma sprout labs team members talking a little, little bit later about how and how x API can be used to integrate this as well.

[Robin Petterd - Host] 10:27:51
So, essentially this is also this sense of flows through and things are we have your learning experience.

[Robin Petterd - Host] 10:28:00
Employees contribution to that business metric

[Robin Petterd - Host] 10:28:04
organizations that work with object objectives key, key results.

[Robin Petterd - Host] 10:28:12
Okay, as we're doing this extremely well, they sit there and say this is a business objective, this is how it flows down his team and it's an individual contract contribution to this.

[Robin Petterd - Host] 10:28:22
And then that individual contribution is quite often talked about in terms of your performance development there's no reason why it can't actually be linked to two different types of learning as well.

[Robin Petterd - Host] 10:28:37
Generally, in terms of of assess assessment and this is getting to the spot where data is an automatic way of being able to observe what people are doing or not doing.

[Robin Petterd - Host] 10:28:51
It's only one dimension, by the way, it is actually true geez people down to the data is not a pleasant spot for any of us to be in.

[Robin Petterd - Host] 10:29:09
Is that observations can be a really useful way of working through things though observations.

[Robin Petterd - Host] 10:29:25
time consuming type of assessment. The complexity is really high.

[Robin Petterd - Host] 10:29:27
And the paperwork that quite often involved in doing me quite often renders.

[Robin Petterd - Host] 10:29:37
Yeah, my wife started in learning for me and when she was actually doing lots of workplace assessment that she actually started, they gave her a number depend on the organization because essentially they sat there and like, oh, you're going to be doing

[Robin Petterd - Host] 10:29:56
a lot of writing. This is going to go that possibly don't have quite an effect on you.

[Robin Petterd - Host] 10:30:02
Um, so actual patents Bell Labs, what we've done is develop a online assessment assessment app that just takes the paperwork, out of this automates the process of things being assigned to people and being a spot where it just becomes a lot more automated

[Robin Petterd - Host] 10:30:21
as a process and reduces down some of the paper, add, add photographs and things things to it as well.

[Robin Petterd - Host] 10:30:30
Couple of activities going really well for time everyone.

[Robin Petterd - Host] 10:30:36
So we've got a scenario where we need to uplift cyber security people who are not in Northern Australia. We've just this week in Australia we've had a breach from the telecom telcos.

[Robin Petterd - Host] 10:30:55
That's what actually to people's.

[Robin Petterd - Host] 10:30:59
Id like driver's license or passport numbers into, into the public domain so it's a fairly big topic.

[Robin Petterd - Host] 10:31:07
So, we have two things. And this is actually one that's quite often happening, where all employees need to do some cyber security training, and then the digital staff who are responsible needs to do another sentence to set all

[Robin Petterd - Host] 10:31:24
of digital digital training as well, emphasizing that, because in actual fact there's like a two layer, the digital stuff hadn't had more of an impact than it did everyone, the exact opposite example was actually essentially a other side of a software

[Robin Petterd - Host] 10:31:43
developer, something wasn't a generic wasn't a standard employee employee, it was a systematic fall down or something.

[Robin Petterd - Host] 10:31:54
So, in chat.

[Robin Petterd - Host] 10:31:57
I do you think the digital team could be assessed as this type of experience.

[Robin Petterd - Host] 10:32:09
Interested seats to see what type of ideas we come back with.

[Robin Petterd - Host] 10:32:17
Cool.

[Robin Petterd - Host] 10:32:18
Okay, it's interesting I Jenny you brought up the, by the way, on the digital assessment of witness. The reason other than this a little bit as well as you need a lot of samples to be able to use so AI and machine learning for that to work.

[Robin Petterd - Host] 10:32:39
Cool.

[Robin Petterd - Host] 10:32:45
In here, number of breaches before and after.

[Robin Petterd - Host] 10:32:52
That's an interesting one, hopefully, by the way, in a perfect world, you don't have any breach of by the way they see it.

[Robin Petterd - Host] 10:33:05
E is the big metric is probably the best way to put on.

[Robin Petterd - Host] 10:33:11
Interesting flowing in from that invite the external artisans and people doing testing. How many tests they find how many issues are gone before and after.

[Robin Petterd - Host] 10:33:34
assessing the fishing programs on diverse monsters responses is a really good one, and there's lots of that type of work,

[Robin Petterd - Host] 10:33:43
get a reading sample breaches to trial security measures before and after. That's a really nice one.

[Robin Petterd - Host] 10:33:53
Given areas in code are configured software. I think she and lots of code is pure assessed.

[Robin Petterd - Host] 10:33:59
So that could actually be a potentially a peer to peer assessment of individuals on what what whether or not. someone's actually now developing code, more and more secure way on the bus, just shared a bit of jargon which I wish I didn't know I can't quite

[Robin Petterd - Host] 10:34:26
remember what it actually stands for.

[Robin Petterd - Host] 10:34:29
But essentially, it's the list of the Top 10. The Gru problems that happen but the things. So, you could actually do something where.

[Robin Petterd - Host] 10:34:39
Yeah, absorb essentially assessment self assessment, peer assessment around those 10 things before and after.

[Robin Petterd - Host] 10:34:51
Could be a really could be a really nice 111, as well. That's actually a really nice one of the many, many, many different different lives.

[Robin Petterd - Host] 10:35:03
Yet scenario activities so you practice things for people to walk to work to work on as well.

[Robin Petterd - Host] 10:35:12
So it's interesting so no one's suggested in quizzes here.

[Robin Petterd - Host] 10:35:19
So now you guys have become quite popular popular and also this whole idea of peer and collaborative things as well.

[Robin Petterd - Host] 10:35:28
The other one is interesting, lose mess with your particular one is, you've taken something that is actually a workplace and industry standard and turned it into the idea that's actually used as the assessment tool, rather than being a third party.

[Robin Petterd - Host] 10:35:48
I am internal thing as well. So that's an intro, as a useful one, because people will appreciate that that that in terms of their own skills as well.

[Robin Petterd - Host] 10:35:57
So let's flip this to employees. So, digital team we have less people and while we can have high activity.

[Robin Petterd - Host] 10:36:09
And that's a different different type of thing.

[Robin Petterd - Host] 10:36:12
If we've got thousands of people going through the course, how could they be assessed.

[Robin Petterd - Host] 10:36:38
Cool. So this is this moment, very much like oh yeah that whole nother way you got thousands of people doing a mandatory and it's quite a fun thing coming mandatory in different in organizations where you've got lots of, lots of people having to do do

[Robin Petterd - Host] 10:36:54
these types of courses. How can we upped our assessment energy for these types of things

[Robin Petterd - Host] 10:37:02
without creating too much to work with people

[Robin Petterd - Host] 10:37:07
is one of those moments where I'm also the reason why I keep on using this one is I'm seeing some fairly average billing practices happening.

[Robin Petterd - Host] 10:37:21
Yet, fishing, simulation sending sentence and sending things to people

[Robin Petterd - Host] 10:37:30
getting people to practice things over time. Actually, it's interesting if people are not familiar with this the number of platforms that after people finish your course.

[Robin Petterd - Host] 10:37:42
They get sent email messages, essentially fishing, that the data.

[Robin Petterd - Host] 10:37:51
There was one particular day when I looked at my email and I just went the password reset from Facebook's possibly the best um phishing email message I've ever caught.

[Robin Petterd - Host] 10:38:02
Because it was actually it was sitting there saying that had a breach of my account, and I had to reset my password.

[Robin Petterd - Host] 10:38:08
And for my moment I just end it wasn't a choice that.

[Robin Petterd - Host] 10:38:27
When, when, where's this coming from the word. Ah, yes, it was just it was a really good example of that that how easy it is to be caught by the simulation.

[Robin Petterd - Host] 10:38:28
Could people do their own risk assessment. So essentially, the completion of a module could be is, it could be in a spot where people have to do their own risk assessment of what type of data that they have access to, and where they took them they might

[Robin Petterd - Host] 10:38:44
be vulnerable and then make a plan for that.

[Robin Petterd - Host] 10:39:00
active situation.

[Robin Petterd - Host] 10:39:03
Okay.

[Robin Petterd - Host] 10:39:05
Things back up again.

[Robin Petterd - Host] 10:39:08
towards my wrap up everyone's.

[Robin Petterd - Host] 10:39:11
A couple of people have seen to that some of these things as a bit of a wrap up I want to do in a bit of an exploration about bow greatest barriers and improving assessment practice.

[Robin Petterd - Host] 10:39:21
The me some extra time design skills, stakeholders like a platform functionality.

[Robin Petterd - Host] 10:39:29
I'm going to do something I don't do very often I know launch a poll, actually technically goes okay caught on to the production webinar during this launch.

[Robin Petterd - Host] 10:39:39
Cool.

[Robin Petterd - Host] 10:39:41
We just going to give this a few minutes for people to be able to access that one person accessing it so far.

[Robin Petterd - Host] 10:39:54
If you can't see the polls in a spot, you might you might need to go out and haven't haven't popped up might need to click on the actual whole button down the bottom.

[Robin Petterd - Host] 10:40:17
Or if you want to get in the chat as well that was he was originally going to do this.

[Robin Petterd - Host] 10:40:27
Cool, thank you scripting remember whoever you are kind of bad job of mine mine instructions again.

[Robin Petterd - Host] 10:40:36
Okay, David.

[Robin Petterd - Host] 10:40:38
So, essentially, just to just to reiterate that what someone is put into chat, it's sounding like I haven't popped up automatically which I thought was going to.

[Robin Petterd - Host] 10:40:49
So, essentially, if you click on the three buttons on your toolbar that says more, you should be able to go to the polls and come back through that if not just actually put them in chat chat as well.

[Robin Petterd - Host] 10:41:02
So, essentially, as people have been joining I've been asking what joining and signing up.

[Robin Petterd - Host] 10:41:10
People have been listing some of their challenges and what they see as they see as an assessment problems, and what often is is actually just getting their internal clients engaged the stakeholder problem.

[Robin Petterd - Host] 10:41:25
People don't want people to do more complicated thing.

[Robin Petterd - Host] 10:41:29
And they don't want employees to do more because they already might be doing say 10 hours worth of mandatory training repetitively the same training every year.

[Robin Petterd - Host] 10:41:45
I don't want it to become become more complicated complicated.

[Robin Petterd - Host] 10:41:45
Please don't take offense but he spent a lot of people who are instructional designers.

[Robin Petterd - Host] 10:41:51
We don't know even knowing that in my own program that I run a running up strong instructional designer, I don't focus on assessment that quite often, learning designers don't actually have more sophisticated understanding of different ways of assessing

[Robin Petterd - Host] 10:42:08
beyond quizzes as well.

[Robin Petterd - Host] 10:42:10
And then sometimes platforms like functionality.

[Robin Petterd - Host] 10:42:15
I've come across many learning management systems that don't have anything other than quiz tool with tools that really just that.

[Robin Petterd - Host] 10:42:25
And I've tried to keep the technology fairly simple when I've been talking here. Here they don't actually have assignment drop boxes.

[Robin Petterd - Host] 10:42:34
Cool. yet, Rebecca, as your requirements that if you are not to do you can't do what I just randomly didn't rewrite the, the key principles of assessment you'd have to work within it within, within our structure.

[Robin Petterd - Host] 10:42:52
It has to be incredibly well documented because you are working with it's in a different spot.

[Robin Petterd - Host] 10:42:57
Cool. All right, on and two minutes early.

[Robin Petterd - Host] 10:43:01
And I'm going to wrap up and say, thank you very much everyone read some great participation in chat. And

[Robin Petterd - Host] 10:43:14
I'm going to stop my share and hand over to Jenny.