xAPI and Totara
Hamish Dewe from Orion Health joins the Learning While Working podcast to outline some of the practical applications of xAPI and Totara Learning Management Systems in a company with a global reach.
Subscribe using your favourite podcast player or RSS
Subscribe: Apple Podcasts| Spotify | Amazon Music | Android | RSS
Sign up to our email newsletter to get podcast updates
Links from the podcast
Transcript
Robin:
This is the Learning While Working podcast. Today I'm talking with Hamish Dewe for Orion Health in New Zealand about the work he's doing with xAPI and Totara LMS. This podcast is a lot more detailed and focused than most podcasts are done. LND now seems to recently be seeing the potential of xAPI and its adoption is increasing. It's incredibly flexible and powerful. Now, there's still more learning that needs to be done around its implementation. As well as the work that Orion Health is doing around Totara LMS, Hamish also talks about really great group based content and link sharing system that they're building using the xAPI bookmarklet. In the blog post that goes along with this podcast I'll include links to the bookmarklet, Learning Locker and the Moodle log store plugin that Hamish and I talk about as well. Hamish. Welcome to the Learning While Working podcast. It's great to have you on today.
Hamish:
Thanks for having me.
Robin:
Just to give a little bit of context to what we're talking about, what is Orion Health, and what does Orion Health do, and what's your role there?
Hamish:
Right. Orion Health is a provider of software solutions for managing patient health information. We're basically a development shop. I work in the education team, so our role there spans everything from the usual internal process and compliance training through to internal training on our bespoke software, and that software stack, through to end user training for those who are going to be using our solutions out in the field. My role is learning technologist, so I look after our systems, everything from the LMS through to our authoring tools.
Robin:
Okay, so that means it's internal and external. Sort of an interesting-
Hamish:
Yes.
Robin:
If often use tech companies, as an example, quite often of learning driven organisations that have lots of self-directed learners. It's also especially when people are in startup mode. What's the head counter for Orion Health?
Hamish:
I think at the moment it's 1,200 plus.
Robin:
So a considerable size organisation. It's not small.
Hamish:
No, it's not small, and it's quite geographically distributed. The bulk of our staff are development staff, and a lot of them are here in Oakland, but there's also development centres globally, plus of course there's quite a distribution of our sales and support staff.
Robin:
Yep. I imagine your clients, that you're then doing work with, is well distributed globally.
Hamish:
Yes.
Robin:
Yep. So you're doing some really interesting work with Totara and xAPI. What's the overall learning technology stack that you're using?
Hamish:
Right, so we run Totara as our LMS. We've actually got two main ones. One for staff use and one for external use. The primary reason for putting in xAPI was to link those two LMSs and give us central data store showing the basic things like course start and finish, and pro-program starter finish in a Totara environment.
Robin:
Okay, and then are you authoring your learning management systems, or are you using another xAPI ready authoring tool?
Hamish:
We don't use xAPI ready authoring tools except for Storyline, but we run those as typical SCORM packages rather than XA API. Because in our case we don't really need that extreme granularity of data that you can get from an xAPI package. On a course, really, we just need when does it start and when does it finish?
Robin:
Yeah. So this is all really interesting. So what are you doing is using xAPI to essentially string together to lots of data sources into one particular spot, and you've been really selective about the data you're actually collecting.
Hamish:
Yeah. We could have collected a lot more, and there's always scope to add that in later if called for, but at the moment we have no real need for that data and we're happy for that granularity to stay within the LMS that generated it in the first place.
Robin:
Yeah. Okay. Couple of quick more tentative questions exploring this a bit more. Which learning record store system are you using?
Hamish:
We are using Learning Locker.
Robin:
And then for Totara, you're using the log store plugin, or are you using something else because the log store plugin might give you too much data.
Hamish:
Yeah, at the time that we were starting work on this the log store plugin wasn't quite mature enough, so we went with a homegrown solution, which has its good and bad points. We have to maintain that, of course, but we also get to pick and choose what data we're going to get. If I was to go at it again I would probably just branch off from the existing log store plugin and configure it to our needs.
Robin:
Yeah, okay. You would probably reduce down the amount of data it was actually collecting and making sure the data was coming through in the right formats for that.
Hamish:
Yes. And there's actually quite a bit of data enrichment that happens in that pro process as well, so that's another area with standard plugin wouldn't have worked for us. Things that we have added in that are important for applications of xAPI that we put in later on are things like adding extensions to reflect the user's position and organisation at the time that the statement is sent.
Robin:
Ah, okay. So taking some of that organisational structure elements, putting that through into the statements, and then you can report on that in Learning Locker?
Hamish:
Yes. We don't really do a lot of reporting in Learning Locker for public consumption. It's pretty much internal to the education team. The reason for that is, of course, if you have access to your LRS reporting you don't have control over which user records are presented to the reporter and which ones aren't.
Robin:
Yeah, and this is a really interesting thing that essentially I know as a Totara partner, we've got very used to that sort of granular level of control of our who gets access to what reports. If you did part of this exploration of doing this podcast, and series of podcasts about xAPI, is exploring what the different learning record stores actually can do, which ones are starting to get the better reporting working, because we're quite often working with Learning Locker as well. But yeah, it’s got some real pros and cons about the sophistication of its reporting tool.
Hamish:
Yeah, our workaround to that was to create a custom report in Totara itself that will extract the data only for the users that you have access to and present that in a report in Totara rather than having a login to the LRS.
Robin:
Yup. So that actually pulls back into the xAPI, back into Totara.
Hamish:
Yeah. And that relies on one of the features that I like most about Learning Locker, which is the fact that, and this is quite a technical thing, the fact that it exposes the underlying MongoDB aggregation, which is a lot more powerful than relying on the standard xAPI queries.
Robin:
Yeah, it's a nice feature of something be a open source. It gives you this different set of flexibility that quite often a closed source software doesn't have
Hamish:
Right.
Robin:
Yeah. If you're going about it next time, what would you do differently?
Hamish:
There's a few quite specific things that I would do differently. One is in how we are actually pushing statements across from the LMSs. What we should do is actually listen for the events and send a statement, or cue up the sending of a statement at the time that the event is triggered. What we are actually doing is querying for course start and finish entries that haven't been sent yet. So that change, I think, would be a lot more efficient. That also means that you need to instal time workout whether you are going to send the historical data across or not. We ran into a few roadblocks with ingesting that historical data.
Robin:
Ah, that's interest. What sort of roadblocks ... It's a very common roadblock, Hamish, when you're moving between management systems, and it's always a big decision about which data goes, comes across, or whether or not the data actually goes into something other than a learning management system. It is one of those decisions that sometimes is difficult. Yeah, so what did you guys end up doing?
Hamish:
Our issue was really about timestamping, because in the statement there are actually two timestamps. There's the timestamp that you can set on the statement itself, and there's the timestamp that gets added at the point when the LRS receives the statement. And of course for all of these historical statements, should be timestamped in the past. Learning Locker would rely on the later timestamp when it came to filtering and ordering the statements. Which is not the behaviour that we wanted, so we had to go directly into the database to override all of those timestamps in order to get it to work the way that we wanted it to.
Robin:
It's really interest, Hamish. I'm enjoying this little detail because the actual facts is some of them make or break activity around xAPI at the moment is actually not. People are getting the power of it, but then it's actually incredibly flexible about how you use it, but there's a whole lot of subtleties around that flexibility.
Hamish:
Yeah, and I think a lot of people see that flexibility and they kind of assume it's so flexible that we can do whatever we want right now, and later on we can tweak it to fit our needs. That isn't quite true. I think that you need to have a really good idea of where your end point is before you start doing these things. I'll go back to the idea of enriching the statement data that got sent through. We have a bookmarklet that we based on the standard bookmarklet that I think is supplied by Watershed software. It extends that. We're using that to tap into the statements that are there to power a recommendation engine.
The recommendation engine relies on knowing what kind of person you are so that it can find statements from similar people for resources that you haven't looked at and recommend them. Because we knew that we were going to be doing that, we had add-in things like the position and the organisation. If we had decided to just add that stuff in later then we wouldn't have had access to recommendations from that whole quite large bulk of historical data. We could have, but it would have been quite difficult to later on enrich the data that was already there, but that is a technically quite demanding and time consuming thing to do.
Robin:
Yup. That's just a really nice example of using the xAPI bookmarklet, and I'll include some links in the show notes to that one for anyone who hasn't heard of it before. Developers are interesting because they're always constantly finding resources and sometimes not sharing them. Essentially you're doing group content curation through using the bookmarklet and xAPI.
Hamish:
Yeah. The bookmarklet that we have had has a couple of sections. The standard section is just sending a statement based on the page that you're looking at at the moment, and we've really limited the verbs that can be used. From memory, I think there's just read, watched, played and a custom one, shared. Shared, we kind of overload to implement an actual sharing mechanism, so if you see shared you can tick a box to say you want to share it with all the people who have the same manager as you, and you can also search for people in our organisation and add them to the sharing list as well. Those statements go into our LRS, too.
The next bit is a presentation of the recommended learning, so this is the recommendation engine output. You can tick a bock to opt-in to a weekly email of that, if you like. After that, there is a standard activity string. And then lastly, we've actually wired the bookmarklet into the Totara goals framework. You can upload a resource to use as evidence against a goal. If you're using the bookmarklet on your phone, or your tablet, that will also allow you to use the camera on your device to add a photo as evidence too.
Robin:
Cool. That's really quite an elegant set of possibilities in flexibility. Especially that last one, being able to add something to a goal, because as you're surfing around on a phone you're not going to go back into your learning management system and add something to that. You want to just quickly do that. It's also really interesting. I'm a huge fan of learning logs as a way of developing reflective thinking. It's almost what you're talking about is using these logging resources and sharing those, and collecting those as evidence. It's a really nice learning just in time, sharing, collecting evidence across xAPI as well.
Hamish:
Yeah. Reality is, as a technology firm, we use a lot of technology that is not in-house, that's part of our open source stack. We don't generate content for learning about that, we don't put that in our LMS. All of that stuff lives out in the wild. If you're going to attract that, really the only logical thing to do is to allow people to track it via something like the bookmarklet.
Robin:
Yeah, okay. It's a really powerful example, the bookmarklet. And also, it's interesting, the few organisations I talked to have had trouble with adoption of the bookmarklet. To me this sounds like it’s got a huge value proposition for learners and employees because they get back other people's resources, so the more they use it the more ... And everyone that uses it, the more everyone gets good content being fed through them by the emails in the strings.
Hamish:
Yeah, adoption here is also a bit of a challenge. Adoption is much better with the staff that are coming on now than those who have been here for quite a while. And of course the data set really only is as good as the usage.
Robin:
And it's always interesting. Only a small amount of people actually do share and publish online. It’s not everyone's cup of tea.
Hamish:
Yeah.
Robin:
Yup. It's interesting. At different job roles at times I've shared a lot more than actually as I'm working on at Sprout Labs I actually share less and am really conscious now about what I share. Whereas when I was actually in other roles I've shared everything, but now I'm actually just really quite conscious of what might actually be client intellectual property, is the best way to put it, to make sure that that's not shared.
Hamish:
Yeah. There is one more thing that I'd like to call out about changes that we've made to that standard bookmarklet. Our infrastructure is such that our LRS actually lives inside of our firewall and it's not directly addressable from outside, which means that the bookmarklet does not have direct access. Because one of the issues that I had with the standard bookmarklet was that you had to add the client's credentials in clear text, and anybody could self-identify as anyone that they want. What we do is we provide a draggable link in the LMS itself that you can click and drag into your bookmark bar, and that link is customised to you. Everyone has a different link that does the same thing and nobody knows the direct access credentials to login as a client into Learning Locker.
Robin:
Yeah, that's actually really nice little security feature for that for everyone in terms of guarantee for users and guarantee for the organisation as well. We know you're running out of time, Hamish. Just one more question I do want to try to get in. If an organisation was getting started with xAPI what would be your greatest piece advice for them?
Hamish:
I give this advice in a lot of different contexts, which is start at the end. Look first at what you want to get out of this information.
Robin:
And that's a really nice line. Really nice way of thinking about it. Start with the end in mind and then all the bits in between flow quite nicely.
Hamish:
Yeah. The configurability is so high that you really do need to know what your output is before you can even make a start.
Robin:
Yeah, so you need
Hamish:
Configuring what you want xAPI to collect.
Robin:
It's almost like you need to plan the reports and then plan data you need to get
Hamish:
Yeah, I think of it more as a typical instructional designer look at things. What are the learning outcomes that I want? What do I need to do in order to ensure that that happens? It's the same kind of mindset.
Robin:
Yeah, the same kind of mindset and just applied to seeing outcomes and then how are we going to measure that with data. Thank you so much today for joining me on Learning While Working podcast.
Hamish:
You're welcome.
Robin:
It's actually been really nice to talk about details with xAPI. I think on the keys to adaptation of xAPI is more sharing of knowledge around, how to go about the nuts and bolts of it all.
Hamish:
Yeah, and xAPI, I think, is at a stage where there isn't a lot of use cases yet. If you're going to be using it either like us, need to have a fair amount of technical internal resource, or be working with a partner who's able to supply that.
Robin:
Yup. I think that's a nice bit of sentiment around the whole xAPI area not being plug and play thing yet. Takes a fair bit of technical knowledge to get started. Actually, it’s a combination of technical knowledge and learning and business knowledge to be able to bring all those three things together. Thank you so much.
Hamish:
You're welcome.
