Ray Jimenez - Two Case Studies on Workflow Learning Impacts - An Assessment Method
Links from Ray Jimenez - Two Case Studies on Workflow Learning Impacts - An Assessment Method
- The Miro notes from this session
- Visit Ray’s Website
- Connect with Ray on LinkedIn
- Connect with Ray on Twitter
- Check out Situation Expert
- Read ‘25 Thinking Tools’
- Learning While Working Podcasts with Ray Jimenez
Automatic Transcript of Ray Jimenez - Two Case Studies on Workflow Learning Impacts - An Assessment Method
[Ray Jimenez] 09:17:48
Thank you, Robin You're so generous with your phrase I'm the world expert, by all means no. I'm just a humble guy from a small part of the world, trying to shout help you know we need to do something about the one thing that we do in learning and development
[Ray Jimenez] 09:18:06
so thank you everyone. Appreciate the opportunity. I'd like to share to you a couple of cases that might be interesting for you to provoke to you, because as you can see I've lost my hair banning against the wall, trying to figure out how to do better
[Ray Jimenez] 09:18:33
what we're doing. So I'm going to share with you a cases I'm going to have some chat questions so feel free to be in the chat. That way we can have a little bit more interaction with our ideas. So I'm coming in and let me start sharing my screen here.
[Ray Jimenez] 09:18:42
I'm coming in from a world where in the business. And when you are in the business, you try to justify your existence, because otherwise you won't be around very long if you don't justify your existence.
[Ray Jimenez] 09:18:56
So what I'm going to show you here is a process whereby learning starts with a situation, a problem, a challenge, a real work, issue that either the worker the participant, or the trainer facilitator might want to start.
[Ray Jimenez] 09:19:19
So in case number one I just wanted to show it to you and see what it connects to the concept of assessment. So we start with the situation and in this situation.
[Ray Jimenez] 09:19:30
This particular lady belongs to a department that they're really losing sales. And let me play just a 10 Minute.
[Ray Jimenez] 09:19:37
10 minute 10 second video that you can see the highlight of the problem. Yes, but, but unless I have a specific date or billion number I can't.
[Ray Jimenez] 09:19:49
Ma'am.
[Ray Jimenez] 09:19:49
Ma'am.
[Ray Jimenez] 09:19:50
What I'm trying to tell you if you listen for just a second, is I have no way of. So that's sort of tells you that a lingering problem very very very small problem is we're faced here.
[Ray Jimenez] 09:20:03
So we start with a problem, and then we asked the learners to go in and begin to do a thinking process and say, let's analyze this problem. Now they are familiar familiar with the fish bone tool so they could type in their problems and say no empathy,
[Ray Jimenez] 09:20:21
or they could say poor instruction, or they could say there's no trust in the customer or defective product, while they are doing this one or two or three or four or five participants or workers can also share their ideas of the conversations that are
[Ray Jimenez] 09:20:36
going on so they do something a little bit the peer learning. The idea is we're asking participants to bring in their experiences, no matter how small into the conversation so analysis and then we do solutions.
[Ray Jimenez] 09:20:50
We ask them to reflect and say what do you think and how you going to solve this problem. So now you have guidance better guidance, some people will put notes in quotations and continue to really learn from each other you leverage the knowledge of people,
[Ray Jimenez] 09:21:06
so you can begin to solve the problem. And then the final step is this, we asked them to present what you call impacts. Now this is not scientific ROI high psychological PhD degree.
[Ray Jimenez] 09:21:24
This is a practical connection over how the worker is trying to analyze solve and then report them in anticipated.
[Ray Jimenez] 09:21:33
Kind of an idea of what the impacts of work that might be. So analysis solutions and impact. So my first question to you to reflect on this is kindly go to the chat and answer this question in here, how this impacts serve as a work based learning assessment.
[Ray Jimenez] 09:21:51
Share your thoughts in the chat, I'll read it up and then we'll continue our conversation. Are you able to see the slide in here that they inserted with a question, makes it real Kara, thank you Hey Kyra, you're my favorite person do you know that.
[Ray Jimenez] 09:22:06
Thank you for that. So, okay, gotcha makes it real. It is more hard hitting because it's a problem that they're facing.
[Ray Jimenez] 09:22:16
Okay, thank you. Keep it coming. Topic coming in the in the chat to share your thoughts in here haha relatable. The issue of the problem becomes relatable.
[Ray Jimenez] 09:22:27
Good. You give them the way of training gives away the training Thank you, real meaning and purpose. I love that barrier. Because, Penny because now you're trying to bridge the knowledge into real life situation so making connections between learning
[Ray Jimenez] 09:22:42
and the workplace Josephine right empathy kicks in. Okay, then retail value add on we're doing tied to outcomes now fail, that idea you mentioned about tied to our outcomes is very essential in here.
[Ray Jimenez] 09:22:58
There are at least two ways. I know that we can do an assessment. One is assessment by gauging the depth of recalled knowledge. We call that memorization which is a pretty standard in what we do.
[Ray Jimenez] 09:23:13
But it's getting difficult to be able to make an assessment of how people learn and apply.
[Ray Jimenez] 09:23:20
And what they had discovered by experimenting and trying other these things together, it is near impossible.
[Ray Jimenez] 09:23:28
You might invest a lot of millions of dollars to connect it this near impossible to connect knowledge retained the real impact at work. There's so many variables as you might see there's so many variables.
[Ray Jimenez] 09:23:40
So one of the methodology in here to connect assessment and results and performance is to narrow down, narrow down in a very small way to identify a problem.
[Ray Jimenez] 09:23:53
That is true to real life to the learner so that they can take advantage of this real problem, and wrestle with it they go to analysis to go to solutions and then the report an impact.
[Ray Jimenez] 09:24:05
At that moment, you could see the correlation between the knowledge of the application in the problem solving and the impact at work. So, for me that's assessment now this is not rocket science or this is that again scientific and say that you can do.
[Ray Jimenez] 09:24:19
Go to the finance person and say these are the outcome.
[Ray Jimenez] 09:24:25
But the most important value I realized based on many what you're saying in the chat is that we are training our learners to actually do the assessment themselves.
[Ray Jimenez] 09:24:37
We're asking them to reflect, and more importantly we're asking them in their thinking process. How would you bring in what you know, to impact your work, which is really what's important.
[Ray Jimenez] 09:24:50
Nonetheless, now of course in the education world there might be a different standards, but in corporate world, where you have to deliver something we make sure that what happens in there is that area so the impacts therefore is the utility, a byproduct
[Ray Jimenez] 09:25:05
of once thinking, a byproduct of one's reflection, a byproduct of one's application of experience and knowledge and showing that they have the impact that they greet them.
[Ray Jimenez] 09:25:19
Doesn't make sense to you so far can you type a yes in the chat.
[Ray Jimenez] 09:25:24
Thank you so much. Great, great idea or raise your hands there. Thank you. Great. Good, thank you so much. So, this is case number one that I'd like to express to you, not when you're looking at this, therefore, the reflection is the process of assessment
[Ray Jimenez] 09:25:41
in itself.
[Ray Jimenez] 09:25:43
Now, that's how I understand it to be. The more you reflect, the more you assess the more you think the more you simultaneously assess assessment in my own world of understanding is not a report card that you would show to school and your parents and
[Ray Jimenez] 09:26:00
your teacher hey I get an A, because that's really a historical view of performance. That is a historical view of assessment, what we want to do is to be able to help them to think through it so that very very quickly.
[Ray Jimenez] 09:26:14
They're assessing the work, and therefore they are reflecting and they're redoing their iterating themselves. So in this model you could see, We could add basically a different way of looking at things for example.
[Ray Jimenez] 09:26:29
So there are a few different tools in here. And one of the other tools might be as simple as deciding the pros and cons of things. What is by favorable and what is not favorable or another set up thinking tools we call them is primarily in the area of
[Ray Jimenez] 09:26:47
really consequence thinking, like, if you are facing a problem, you making a decision, what would be the positive outcome, or what would be the negative outcome.
[Ray Jimenez] 09:26:58
Now this is where the real assessment in my own view that's happening is assessment if I'm learning. If I'm applying the idea, if I'm discovering an answer that satisfies me.
[Ray Jimenez] 09:27:11
So assessment in this model. Therefore, as work in that area so Sherry the chat with this particular case that I'm showing you right now what is something that raised into your mind, or if you have a question or thought what grabs you here in this first
[Ray Jimenez] 09:27:26
case study that I'm sharing with you.
[Ray Jimenez] 09:27:29
Another follow up question what they so different with his approach, compared to what you're familiar with. can you share your thoughts in the chat.
[Ray Jimenez] 09:27:49
I love that the gate assessment is part of the learning process. It is actually simultaneous. Okay, they just were natural that way. Okay, thank you for that slightly off topic analysis tools a year grid analysis tool you want to show that the grid of
[Ray Jimenez] 09:28:07
so let's just put it up in here open for the meantime, I like to building self reflection. Yes, the self reflection is there. Good. Going reflection action reflection action good. So, this reflection for me has been a crooks.
[Ray Jimenez] 09:28:15
In many programs that we do, we had been a little bit limited because we had been focusing on instruction, which in the instruction, the only way you can do an assessment is to test or observable behavior which can become very difficult.
[Ray Jimenez] 09:28:33
So in the process of reflecting the built into reflecting is a self assessment. So this is really where the heart of that, and then you're connected with some impacts.
[Ray Jimenez] 09:28:43
So Christina thank you for that the building and employee engagement with their thoughts method in the solution process that Christina was one of the biggest important product that they've developed.
[Ray Jimenez] 09:28:55
Imagine this.
[Ray Jimenez] 09:28:57
When you ask the worker to do something.
[Ray Jimenez] 09:29:01
And they don't know how to do it, you train them, but they don't know how to do it but then you gave them the tool. Okay, do it.
[Ray Jimenez] 09:29:08
And while they're doing it, they are actually diagnosing the problem.
[Ray Jimenez] 09:29:13
They're fixing the problem. They've tried to test something. At the end of that. It is so natural. It is so organic, that they must assess.
[Ray Jimenez] 09:29:23
So we are already, you would be things with a constant organic way of assessing things. We cannot separate the two with us. So instead of artificially creating an assessment, away from that natural assessment process.
[Ray Jimenez] 09:29:40
I would urge, some of you most of you like what I've tried to test, look at that and see how you can build in an assessment process. And the biggest outcome is this, you get learners.
[Ray Jimenez] 09:29:52
One thing to learn more engaged with a problem, because they can influence the outcome of their assessment, which is a different outlook right influencing the outcome you'll be assessment is more important for the worker and the learner, then having a
[Ray Jimenez] 09:30:08
scorecard.
[Ray Jimenez] 09:30:09
I don't know if you agree with that but do you think I will repeat it.
[Ray Jimenez] 09:30:13
Having to influence the outcome of the assessment is far more important than having a report card. What do you think, Share your thoughts in the chat.
[Ray Jimenez] 09:30:24
Having the ability to use the assessment in changing your process changing your improvement in order to learn more is far more important than getting a report card.
[Ray Jimenez] 09:30:37
Of course you have to have a report guide. But it doesn't stop there. Right.
[Ray Jimenez] 09:30:41
More learner centric Thank you much more empowering. Thank you for that. Brenda Lisa mastery versus performance Yes, very good. Yes, it is so much meaningful to the learner so mastery business performance.
[Ray Jimenez] 09:30:55
So this is case number one and I'll scroll down below here the other tools. There's a troubleshooting tool. There is a mapping tool. This a contribution tool right.
[Ray Jimenez] 09:31:06
So let me show to you. The other example that I wanted to I prepared for you here. This one is a bit more technical.
[Ray Jimenez] 09:31:14
It's starting again with the concept that somebody reported there's a problem.
[Ray Jimenez] 09:31:20
assessment done and set up by a teacher is almost like external foreign for the thinking process of the learner, because they don't want to be assessed based on somebody standards.
[Ray Jimenez] 09:31:36
How many of you like to be assessed based on someone's standards. I don't want to be assessed by a teacher, because the teacher has a different standard.
[Ray Jimenez] 09:31:44
But if I am facing my own problem, I set my own standards. Therefore, I begin to learn from my own feedback and assessment. So this is a second model which is malfunction of an avid hf system.
[Ray Jimenez] 09:31:59
So, this is a area where there's a problem of the system. The situation, talks about a certain clog, there's a certain problem with the sensors. So let me just play a very tiny video here to give you a feel of what this problem is and I will skip the
[Ray Jimenez] 09:32:14
so you can see it forward that catheter is placed in the pulmonary artery, to determine baseline chemo dynamics and along with a selective angiogram to confirm the target implant vessel.
[Ray Jimenez] 09:32:26
The center delivery system is advanced over a guide wire to the identified. So this is where the problem is. There seems to be a loose connection or something that the reading cannot be seen through the digital display so small problem brought up by technician,
[Ray Jimenez] 09:32:43
and now they develop a simple tool that they will reflect on the problem situation, they discover where the errors could be likely. It could be an error in this area.
[Ray Jimenez] 09:32:55
It could be an error in that area, it could be an error in that area so they go through this process of discovery. and then they go through what you call a troubleshooting process.
[Ray Jimenez] 09:33:05
Now, let me just play this very tiny little video here 15 second pa sensor is calibrated during manufacturing.
[Ray Jimenez] 09:33:11
During the implant procedure, the sensor and PA catheter mean pressures are matched. After the implant procedure, the center provides accurate pa pressure measurements, and is, so they're talking about the problem this is where the grace of the corrosion,
[Ray Jimenez] 09:33:27
but they have no problem, they have no understanding of what causes. Now how many of you have encountered work, whereby most of the time you have an answer, but there are those moments where there's no answer.
[Ray Jimenez] 09:33:38
Even if the policy is very clear the procedures very clear one to 10 is very clear. There's always some point in time where something goes wrong, and nobody has an answer.
[Ray Jimenez] 09:33:48
So, how do you make an assessment in that area. So what they do next in here is they go to a troubleshooting activity.
[Ray Jimenez] 09:33:55
They go to troubleshooting activity they ask the questions What did they do wrong we can we test. What can we do in this area, and so on and so forth.
[Ray Jimenez] 09:34:04
They follow this process until they figure out a trial and error component, and they make a conversation among themselves.
[Ray Jimenez] 09:34:13
Virtually Of course because there's a thought as a virtual on what are their conclusions and they're finding. And then finally, we go back to reporting that impacts.
[Ray Jimenez] 09:34:24
Over here the impacts is now calibrated or configured to ask questions like, which one of this really helps in low cost of maintenance, saving of the replacement of that particular part safety with patients and reliability in us, because these are the
[Ray Jimenez] 09:34:39
standards and expectations they want, when they're trying to maintain that particular product again when you're looking at this I wanted to just ask you a question.
[Ray Jimenez] 09:34:49
That really in measuring impacts and put it up in here by solving a specific problem. A useful form of feedback and assessment right but using a specific problem.
[Ray Jimenez] 09:35:00
Okay. And why is it so useful, that when you assess something you need to identify a specific problem. Can you share your thoughts in the chat for me.
[Ray Jimenez] 09:35:11
Why specificity is so important. can you type your thoughts in the chat.
[Ray Jimenez] 09:35:26
Stage tool of psychological safety learner safety. Very good, thank you. Why is safety Why is a narrow problem really useful in forming feedback, and some form of assessment.
[Ray Jimenez] 09:35:41
Okay.
[Ray Jimenez] 09:35:42
baselines are important to understanding growth. Good input there thank you, Maria, it is more actionable, a, I love that. It is more actionable. This is where we go.
[Ray Jimenez] 09:35:56
You know, when we convert our knowledge into something specific, we can act on it.
[Ray Jimenez] 09:36:00
When you don't have something specific, it's so broad, you cannot act on it. So that's one of the essence of here. Thank you so much, Maria. I rain you can give feedback that is meaningful meaningful because it is concrete.
[Ray Jimenez] 09:36:14
It is a specific. It is a real life consequence problem it is not a theory, it is not knowledge to be memorized. It's something that is real, real consequence right.
[Ray Jimenez] 09:36:24
The problem must be clearly defined in order to make a solution Ralph fantastic good are Peter provides just in time learning for a specific problem that is very very key.
[Ray Jimenez] 09:36:29
So, again, the format in here as I'm showing it to you, at least from my own experimentation and working with clients and we have some pretty large FedEx and other companies using the system.
[Ray Jimenez] 09:36:49
The idea being that we cannot assess impacts.
[Ray Jimenez] 09:36:53
If you don't use a specific problem, that's the general observation that I have made. So, in one excellent point in here, in certain work there's an expectation that people can generalize learning the range of situation.
[Ray Jimenez] 09:37:08
It can make specific specificity difficult I agree with you i agree in that there are those conditions like those as well, right that they exist.
[Ray Jimenez] 09:37:15
Okay, good. Prevent circular conversations due to too many possibilities one point in time. So you need to make it so narrow, so the conversation becomes useful.
[Ray Jimenez] 09:37:26
So when you're looking at this Therefore, a new reflecting on this. One of the things that we discovered as we move ahead, is that although the assessments, the rating of efficiency savings and quality by every problem that they notice although it is
[Ray Jimenez] 09:37:43
not scientific that you can detect all the numbers, they're all estimation.
[Ray Jimenez] 09:37:50
Many of the people I work with the clients and leaders begin to respect an idea that it is not necessarily the numbers that will impact the bottom line, obviously.
[Ray Jimenez] 09:37:59
But the fact is that learning is connected to impacts makes a huge difference is almost like before you memorize something before you have a lesson. The first thing you do is is the learning objective what's the impact, we will use it.
[Ray Jimenez] 09:38:14
Okay, what's the value of that, when are you going to use it. What is so narrow that you can apply it with. And that's the same premise in year. So you're closer, the gap, you put the gap together, you tighten them together, you don't make it separate
[Ray Jimenez] 09:38:28
like the typical approach that we have a lesson and knowledge, and then the assessment is too far away. It will lose its meaning. If we lose its value to the learner, so that's what's impacting here.
[Ray Jimenez] 09:38:39
Another thing that I have discovered is that if we want that assessment to be a reliable way to get the results. I also have discovered that in the typical lesson, and being careful here to make sure that they are accurate in year.
[Ray Jimenez] 09:38:57
In a typical lesson we watch knowledge recollection, and to some extent application, and behavior change right.
[Ray Jimenez] 09:39:06
Ideally we could capture that. But the realities are near impossible to do that.
[Ray Jimenez] 09:39:12
In this experiment in this case study, we watch carefully the concept of whether people are doing the analysis, the solutions and the impacts, as a collective activity.
[Ray Jimenez] 09:39:24
Because the more they do the analysis the solutions and the impacts the likelihood of them getting deeper into an understanding of the topic and the problem is a lot higher.
[Ray Jimenez] 09:39:38
The probability is that they get to understand it because it becomes, they own it, they solve it. If they only do an analysis and they don't continue to solutions and impacts, it sort of loses the steam so from a assessment from a trainer designer point
[Ray Jimenez] 09:39:51
of view, we carefully look at this and simply say, are they completing, what we call their thinking process, or they completing their thinking through process as a result of that.
[Ray Jimenez] 09:40:04
So my question to you and help me out in the chat in here in tracking the depth of analysis solutions and impacts how is learning deepened share to be your experience how is it that when you spend time trying to wrestle yourself with a problem that it
[Ray Jimenez] 09:40:19
deepens your understanding of that, and therefore your assessment would be completely different. Share, share your thoughts. How does it deepen when you go to this process of analysis solutions and you backs.
[Ray Jimenez] 09:40:31
Share your thoughts in the chat. Thank you so much.
[Ray Jimenez] 09:40:48
That's a good question. Similarly, how the space repetition. Let's say okay, learn on, they do a chance to retrieve what the chat in the like. You have to research think and spend time to the.
[Ray Jimenez] 09:41:01
When you analyze solution, thank you so much for this Alicia.
[Ray Jimenez] 09:41:05
When learners invested they tend to get more drawing upon requisite now. So, the earlier comment area was a really interesting one and I have a few minutes more to just digress a little bit and certainly when you believe in.
[Ray Jimenez] 09:41:21
Rick knowledge retention, your assessment methods are different.
[Ray Jimenez] 09:41:26
You talk about mastery memorization reflection you talk about that area. When you look at your assessment as a natural part of work, then your approach is different.
[Ray Jimenez] 09:41:36
So I think that's the major takeaway that unlike to share with you today. And as a result of that I just like to ask you one question in here. What is the one idea that you thought interesting and you found useful today, and I would put in here in the
[Ray Jimenez] 09:42:01
chat, a link to a brochure about this for your own reference later on, what's the one idea that you found interesting and helpful and interesting to you within the chat, please.
[Ray Jimenez] 09:42:07
Josephine Thank you How are you surfacing student analysis, the same question. When you put everybody into a solutions mode and analysis mode of their own problem.
[Ray Jimenez] 09:42:17
There's a natural surfacing Josephine right chain of consequences reflection Thank you, crowdsourcing solutions via self assessment I love the term their government going to borrow it from you simply making assessments feel organic more natural.
[Ray Jimenez] 09:42:33
With that, I would like to end and turn it over back to Robin, and the team. Thank you so much for the opportunity to work with you today, and I look forward for some questions,
