Using chatbots to increase learning transfer, with Emma Weber

Emma Weber is an expert in learning transfer. This is actually the second great conversation with Emma on the Learning While Working podcast; the first conversation focused on the 70:20:10 learning model and learning transfer.

This interview starts with Emma giving an overview of her approach to learning transfer. One of the key features of her approach is what she calls ‘learning breaks’, which are coaching conversations that focus on personal accountability for making changes and reflection.

In the interview, Emma talks about how she struggled to find the right technology to support her approach to learning transfer. She is now using two platforms that work together: an action planning tool called Turning Learning into Action and a chatbot built with the Mobile Coach platform. The Turning Learning into Action platform is free.

What she achieved is a great example of using digital technologies to automate the learning process. It might be not a perfect replacement for a personal learning transfer coach, but it is scaled and affordable.

Download the how artificial intelligence is changing the way L&D is working eBook

Subscribe using your favourite podcast player or RSS

Subscribe: Apple Podcasts| Spotify | Amazon Music | Android | RSS

Useful links from the podcast

• Find out more about Emma
  
• Find out more about Lever – transfer of learning
• Find out more about Coach M – the learning transfer chatbot
• Start using free Turn Learning into Action tool
• Find out more about learning transfer

Transcript - Using chatbots to increase learning transfer, with Emma Weber

Robin: At the Learning While Working Virtual conference, you talked about the work you've been doing with chatbots and learning transfer. Before we start talking about the chatbot bit, I think we need to explore your philosophy to learning transfer. What do you think are the key factors for learning transfer to happen?

Learning transfer is about behaviour change – putting learning into practice

Emma: I think learning transfer is such an interesting challenge, because basically we're talking about the behavioural change part of learning, so how do we get someone to actually shift their behaviours after or as part of the learning intervention?

There's a lot of talk at the moment around the Ebbinghaus forgetting curve, which of course has been around for ages, but I think neuroscience is giving it a resurgence and helping people remember what they've learned. For us and the type of learning we work with, which is often soft skills, leadership, or sales, it's less about whether someone can remember what they've learned and more about whether they actually put it into practice.

Learning transfer – the key is reflection and personal accountability

Emma: So Robin, sorry to choose a long way around to share a philosophy. For me, learning transfer isn't about remembering, it's about helping someone reflect and hold themselves accountable to putting into place what they've learned. The way that we tackle this is to help people reflect and hold themselves accountable.

It's not about sending out reminders. It's not about giving them additional information. It's not about even just helping them reflect on the content. It's about helping them reflect on what they've said they're going to put into place as a result of the learning.

Robin: This shifts learning to focusing on behaviour change rather than memory. Which means a different sense of personal ownership is needed.

Emma: Yes.

Robin: I really have issues with the idea of learning being about memory. It's actually about behaviour in most situations.

Learning is not just about remembering

Emma: It's a very easy trap to fall into with transfer because it would be so much easier if it were about memory. But I think the people we work with, they're clever people and they can find ways to help themselves remember or have access to information. It's much more fundamentally about why do we do something or don’t do something, even if we know it. That's where the behaviour part really comes in.

Robin: In the past, with this sort of dual focus of accountability and reflection, how have you been making that happen?

Learning transfer conversations

Emma: So really it's through a series of follow-up conversations that are held over the phone. The reason we hold them over the phone is because it's about helping the individual have that conversation with themselves, and then there's a direct link between their words and the phone headset and it kind of goes directly into their brain and their body through sound. So at the moment a lot of our work is delivered over the phone.

We work with organisations either to deliver those learning transfer conversations for them, and we've got a team of people around the world. I think at last count we were working in 12 different languages. We can also help companies create internal teams that do these conversations. So we also help upskill internally around learning transfer, particularly learning teams.

That's been really interesting work, and that work has really then led us on to the development of the chatbot, which is of course what we're here today to focus on.

Robin: Before starting to talk about the chatbot, can we explore this phone business a little bit, Emma? Because essentially, working remotely most of the time, it's interesting how over time I've grown to love just interacting with people via voice. That's maybe the reason why podcasting works so well for me.

Was this focus on using a telephone a conscious decision? Or did it happen through the logistics of having to deal with diverse organisations and diverse coaches, and then you realised that was an asset?

The power of voice

Emma: So logistically it's really handy. We can't get away from that fact. The joy of it is that often it’s really powerful and I think there's a big piece around vulnerability. People will often be more vulnerable when they can't see the person. I don't know if you can relate to this Robin, but perhaps if I'm travelling away at a conference and I have a phone conversation with my partner, we'll often have a much deeper conversation over the phone while I'm away then if we were sat at the dinner table opposite each other. It just takes the conversation to another level when you're speaking by phone sometimes.

It can be used to be very tactical, but we find that people will become really honest and open and vulnerable when they're working over the phone. We don't use video calling at all. We really are trying to help that individual have a conversation with the internal dialogue, which is a very personal thing.

Often when we're working with someone over the phone, they'll share something that they may have never voiced outside of their own head before. You can help shift behaviours the minute you get someone really at that level of awareness, reflection and deep diving into what's really going on for them around what they're trying to achieve.

Robin: It almost becomes like the internal monologue that you have in your own head being supported by someone else.

Emma: Yes, and supported or challenged in a very supportive way.

Robin: Fascinating, because that also then helps shape the fact that you weren't jumping from working with face-to-face coaching to working with chatbots. When you first talked to me about this, you said you'd gone out and spent a period of time trying to figure out what could be the best type of technology to help with learning transfer and that took you a while to decide which was the right technology. I think even in that conversation you didn't say which way you had gone, because you weren't sure if this was going to work at that moment. You were just saying, ‘Oh, I'm doing something with technology.’

Emma: Yes.

Robin: Why did you end up moving towards chatbots as a possibility?

Emma: Well, we spent a good two years investigating different types of learning transfer technology. What I was really searching for was the technology that I would have designed if I could have been bothered to. I am not a technology person, but of course everyone, my mentors and other business owners would challenge me and say, ‘I mean, you've got to get tech savvy. How are you integrating with technology?’ and blah, blah. And, ‘I'm investigating, I'm investigating.’

But everything I saw was really around reminding, prompting – and the whole kind of prompting thing for me, it just strikes of nagging. I think we call them boosts or some people call them boosters, I think. No, Robin, that's not quite the right word is it? I'm lacking my vocabulary.

Robin: There are a few different types of thinking around some of those approaches, and they are often focused on memory, e.g. retesting someone over a period of time.

Emma: Yes.

Robin: Sometimes they're reconnecting and giving people nudges in a certain direction.

Emma: Yes. Nudges is the one kind of approach that comes easily to mind. But I think for me, a nudge could almost be so close to nagging someone to change their behaviours. Personally it really gets my back up if someone reminds me, ‘Emma, you said you were going to do this. Have you done it?’

So for me, I was looking for the technology that really was core to the way we believe humans operate and the way we want to support people in changing their behaviours, rather than nag them into changing their behaviours. I was on the search for that, and it wasn't until I came across chatbot technology that it really started to fall in line with what we were thinking and what I believe is the best way to create behavioural change.

One of the things I often say is that learning transfer is really about helping the individual have a conversation with themselves. Someone said to me, ‘Well Emma, if that really is the case and you're just trying to get that individual to talk to themselves, why can't you do it with artificial intelligence?’ Of course, I tried to come up with 101 reasons why you couldn't, and then sort of said, ‘Okay, well let's try it.’ And Robin, I did keep it quite under the radar to begin with because I really wasn't sure if it was going to work, and I don't want to be putting anything out into the world and marketplace that doesn't create really great results. So I was kind of hedging my bets a little bit. We were keeping it as a very quiet pilot.

The first person who I showed the demo to, said, ‘Emma, I absolutely love it, I want it within the next month.’ That was the first person we'd had a conversation with about the technology we've been working on.

Robin: Yes, it's great that you got that great excitement so quickly as well. Can you give a bit more context? How does it sort of work, what happens? We're talking about a chatbot, but what's the experience for the learner? What were you showing that person in that demo, may be a good way of thinking about it.

Learning breaks

Emma: The idea is that we want people to sit down and have what we call learning breaks. A learning break is just when you sit down and you reflect on the goals that you've set yourself at the end of a training program or learning intervention. In essence, when we're working with someone over the phone, that's what we are doing. We actually sit down with them and have a 30-minute conversation based on their action plan; they reflect on the action plan and then we help them move forward.

With the bot it's the same premise: that the individual is going to sit down and have a learning break with their mobile phone, and have a conversation with the bot. Now I've got to say here, Robin, before we go too deep into bots, the conversation with the bot is a different level conversation than the one you would have with a human, but we have to be realistic that it is a lower cost than having a conversation with a real person.

If you're looking at your learning transfer strategy, there are going to be some strategic things that need to have human contact, and there are some things that don’t need human contact. The high stakes strategy pieces need human contact and the lower stakes pieces can be dealt with by the bot.

Robin: I know when we do coaching work, there's a set of questions that you constantly ask with the clients we work with. It's about automating those bits of the conversation and templating of the reflective structure.

Expertise from over 18,000 learning conversations

Emma: I think one of the things that has helped us, particularly as an organisation working in this space, is that we've had over 18,000 one-on-one learning transfer conversations. There's been an awful lot of learning and peer learning, about how to move people through to create the behavioural change. What we've done is we've taken the learnings from real conversations and put those in the chatbot.

Robin: In one of the other podcasts in this series, Mike Sharkey was talking about the way that an academic who'd been working with students for 10 years summarised some of his deep learning into a chatbot. He'd taken 10 years to get to that point. To be able to build something that was really profound, you need to know what the questions or what the responses are going to be.

Stage one: turning learning into action

Emma: Robin, let's go back to your question around how this actually works. The first stage is the preparation stage and this is all part of our TLA: Turning Learning into Action methodology. Whether it's over the phone or with the bot, it's preparation, action and evaluation. The preparation is when the individual makes the commitment as to what they're going to put in place from the learning that they're attending or the learning that they're experiencing online.

Essentially, that's the stage where you make a decision that you're going to implement or you're going to experiment with certain aspects of the learning. Then the action is when the bot actually steps in, and the evaluation is of course when we collect the data.

Now in our process we've actually got two different parts of the technology. So not to confuse things too much, but as part of our mission worldwide is to make a difference to learning transfer, we've created a free online action planning tool and it’s at turninglearningintoaction.com. Anyone can use it, and it's a tool that really easily distributes action plans.

Stage two – Automating the learning breaks

Emma: One of the places it can distribute your action plan to is the bot. The bot is not free, but the basic action planning tool can go out to either a colleague, a peer, or your manager. It's a really effective and easy way to do that, so that is standalone technology that people can use for free.

We use API to send the learning into a bot so that when someone is following up and having their learning break, it's based on their specific commitments that they've made as part of their learning journey.

Robin: So the bot's actually asking about those actions and about how people are tracking on those actions?

Emma: Yes, it's loaded into the bot. What that enables it to do is to be really personalised, because it's based on what the individual has committed to and they want to be working on.

Robin: Now – maybe we’re jumping forward a little bit – the learning conversations are quite often 30-minute sessions. Are the bot sessions that same length?

Emma: You know, it was interesting, Robin, because we were thinking they would be 10-, 15-minute conversations. During our pilot we were seeing that for nearly 20 to 30 minutes people were really sitting down, thinking and reflecting. They would be replying with whole sentence answers, all by SMS. The bot can work either by SMS or any kind of messenger platform that people are using within their organisation.

What we are trying to do is have the learning break in a conversation format where you would normally talk with your friends. So if an organisation uses WhatsApp, then it's a WhatsApp. Whichever way people interact together as humans, that's where we want to drive that chatbot relationship.

Robin: Okay. So it comes to the platforms you're working on already and that creates a sense of intimacy that some of those platforms have as well.

Emma: Yes.

Robin: How have people responded? Have they been positive or negative? Is it odd that they're talking to a bot rather than a person?

Learners’ response – people we engaged more than expected

Emma: The thing that really surprised me is that people would give full sentence answers and would also be really friendly and polite, and do things like wishing the bot happy Christmas. So there was that real relationship piece with the bot, even though they knew it was a piece of technology.

I've been chatting to people about this since, and it really does play to the fact that as humans we still want to have that connection. It's part of our own value system to be polite, whether we're being polite to a piece of technology or a person. That in itself is quite interesting. But I think what's great is that the people were really relaxed and didn't feel it needed to be a sort of a programmed response because they were talking with technology.

Robin: It's really fascinating. Really interesting. It's now common to chat with people using technology. It’s an engaging and intimate personal medium.

Emma: Yes, and people knew this was a chatbot. This was not a covert operation of trying to convince people that they were chatting with a human. Everyone knew the playing field.

Robin: Track right back. I do personally keep a journal and I do think of myself as a reflective person, but it's reflecting inward – not something that people often do in workplaces. So what's really interesting is that your model forces people to have time to reflect by putting structure in it, rather than making time to reflect. It's interesting in terms of giving reflection structure.

Igniting the power of reflection with technology

Emma: Robin, I think you're completely onto something there. I find it quite upsetting actually, that we are so busy in today's world that the art of reflection, I think, is being lost. Even when people perhaps used to sit on the bus staring out the window, perhaps reflecting on their day, you sit on public transport now and everyone's on their phone. I walked past the coffee shop on my way home last night and everyone was with a device, either their phone or their laptop.

What I love about this process is we're actually helping people get used to reflecting and almost re-stimulating those pathways in the brain. It’s actually a means to reflect and think things through. I think we just get lazy and it's much nicer sometimes not to have to reflect and think things through and rather just zone out, watch another clip on YouTube. Sometimes even in the guise of learning, it's much easier to take in fresh content than it really is to work out how you're going to apply the current content you know.

Robin: It's lovely to see that you're trying to bring those sort of reflective moments back into that technology space and in an intimate way as well. So you said a couple of times to me that you've worked with a technical partner to build this bot. What's been your experience around the complexity of it? Is it a really complicated process?

Emma: Good question, Robin. Let's think about that. Is it a complicated process? Yes and no. The partner has been fantastic. I'm really lucky with the people that we have on team, and to have experts that will help us in this space.

With the conversation that we have created, there's a three-question sequence and of course we needed to take all our methodology and put it into the bot. So even with a three-question sequence, we have 127 different sorts of permutations and combinations of how people could answer these three different questions. When we're actually creating the path for the bot, it sounds really simple, but actually it’s complex. I've sat on long haul plane journeys sketching out this process. It gets really complicated really quickly.

Robin, I think I've shared, when I was part of the conference, that I've become quite obsessed with Chatbots Magazine, which is a fascinating read if anyone has spare time in their day. I was reading an article the other day that – it sounds so simple, but it's complicated when we think about human language and how words and nuances and different pathways – yes, it's not simple stuff. So I don't know if that quite answers the question Robin, but yes, there's complexity within it as with everything nowadays.

Robin: I think what you're saying is that your technology is not complex; it's the planning side of figuring out the way it’s going to be used that is complex.

Emma: Yes. That could be it. I could also say maybe it’s one of those things that our partner kind of makes look easy, given that I'm not from a technology-based background and that's what they supply with this.

Robin: Now, let me spring a question on you. I was actually struck by something you were talking about – about the criticality of the conversations. Sometimes in chatbots that are being used in marketing and sales, there's a moment where you switch from talking to the bot on a website, to an actual person. Is that something you're doing or thinking about doing?

Emma: Yes, absolutely. We have our bot set up where you can actually text a certain kind of a code word and we will then call the person and help them. They have proper live human backup. Now, one of the lessons from the pilot was our code word was far too common. We set up the code word of ‘coach’: that if you are stuck and you need help and you want to speak to a person, just SMS the word ‘coach’ and you'll arrange a time to have a chat with someone. Now of course we were actually working with sales managers who were developing their coaching skills for how they were going to support their staff.

We learned pretty quickly that the word ‘coach’ quite often came up in conversation as part of a conversation about what they were going to do with the actions, and not because they're in distress and needed to speak to a real person. So yes, it's absolutely set that if you get stuck, you can speak to a person. We're just doing a bit of refining around what's the specific word that we use for that.

Robin: That's funny, as you were saying that I sat there thinking, ‘Wait, I think I know what word they might have used.’

Emma: You could've preempted us, Robin.

Robin: This is shape shifting work you're doing. It's really exciting. It's an interesting, deep, complicated way of using technology in a different way, to transform some of the real possibilities around learning transfer. I know you're not a chatbot expert, but what do you think some of the other applications are in L&D?

Cutting through the content noise

Emma: When people think chatbots, they often think about customer help and customer service interfaces. Ours is a little bit different in that we're not giving out the information. But I do think that chatbots can be used in a broader learning way to actually share information at a time of need, or to really cut through the whole noise of content.

I think bots are a way to share curriculum. In our bot we have videos that are about reflective thinking. So you can actually use the bot to send you a link to a video. For us, it's part of a learning transfer conversation, but of course it can be used in a different way.

There is also the whole idea that you could use the chatbot to help people at the moment of need, when they're trying to do something technical or when they need support. Which is the same way as you would use a chatbot that's on a travel website, when you're having a problem booking something or you want some information. So I think there's a lot of implications for learning across chatbots. Yes, I think it's quite broad.

Robin: Yes. To me it seems like chatbots are one the most accessible bits of smart AI based technology that could be adopted really quickly by L&D today, with real practical outcomes.

Emma: That's the thing I love about it, is that for me it's about thinking, ‘Well, what are you trying to achieve, and does this piece of technology help us achieve that?’ As opposed to, ‘Oh, we want to get into AI and we want to get into using chatbots and let's do that.’ It's like, ‘Well hang on a second, what is it you're trying to achieve?’ Really coming back to the end outcome or the user experience all the time, rather than it being about the technology.

Beyond engagement – seeing depth in chatbot conversations

Robin: I just realised that I haven't asked you about what kind of outcomes are you starting to see, and the results you're starting to see from the trials you are doing.

Emma: So there are a few things that really surprised us, and one was the level of engagement. Now of course, if we're looking at a behavioural change outcome, a level of engagement isn't an outcome in terms of behavioural change, but it is really an indication of how engaged and how deeply people were reflecting.

I was anticipating this by the time we got to the third learning break, remembering this is the third time someone will have sat down and had a 20- to 30-minute conversation with the bot. I was thinking it may even be only a 20% engagement from a group, but 67% of people had three to four conversations with the bot. 87% had two, and 100% had one conversation.

The other interesting thing to analyse was the depth of the conversations that people were having, through looking at the actual sort of scripting. You could analyse the number of words in sentences in responses. We could actually just read the responses, but there are ways that you can use big data to crunch and understand what people are saying. We also did exactly the same measurement process as we would do if someone had a series of conversations over the phone and asked them to recalibrate the shifts in their goals on a scale of 1 to 10.

Checking my notes here, Robin, because I wouldn't want to fire a percentage at you and it not be 100% accurate. There was a 53% uplift in terms of their scale of 1 to 10 on their particular goals, and a 35% uplift in their ability to achieve objectives with or without the follow up. So this is a lower stat than we would get from talking to a human, much lower, but then in some ways you'd have to expect that.

I still think though, the level of engagement actually shows how far people have come and that they were still reflecting after three learning breaks.

Robin: Yes. This is also interesting in terms of the fact that there might be a lower outcome, but a lower cost as well.

Emma: Definitely.

Robin: A personal learning transfer coach could be expensive. Whereas the chatbot is more scalable to achieve worldwide results. You also talked about transforming learning transfer globally.

Emma: Yes. Yep. Agreed.

Robin: Thank you so much for joining me. We had a longer conversation than we normally do, and I'm off to change our Slack Bot, which does a check-in each day about what we've done that day, to include some reflective questions.

Emma: Good stuff, Robin, I'm glad I'm inspiring you to turn your learning and put it into action. Really appreciate that.