Adam Avramescu: [00:00:00] It’s one of the first times where I didn’t feel. Depressed state of, learning management.
Dave Derington: [00:00:08] Oh my gosh.
Adam Avramescu: [00:00:09] I actually felt hopeful.
It’s October two 2018. Welcome to episode three of CELab the customer education lab, where we explore how to build customer education programs, experiment with new approaches. And exterminate the myths and bad advice that stopped growth dead in its tracks. I’m Adam . I lead customer education at Checkr
Dave Derington: [00:00:41] and I’m Dave Derington.
I lead customer education at Azuqua
Adam Avramescu: [00:00:46] Awesome. one of the most common questions that we get asked from people who are starting customer education programs is do I need an LMS, a learning management system?
Dave Derington: [00:00:57] It’s a good question. an LMS is like a standard part of a customer education program.
And it’s usually the first system right at them that is implemented, but isn’t right for everybody.
Adam Avramescu: [00:01:08] Yeah. A lot of what I hear is that when you’re first starting a customer education program, you can’t always get the budget for an LMS. You can’t always justify the ROI. I hear people get asked that a lot.
And even when you work at a more established company, you often find that you have an LMS and it’s maybe the first system you have. But it doesn’t always work for you.
Dave Derington: [00:01:29] That’s right. let’s tear into this. So let’s get into our hypothesis section, right? This is a scientific venture. So I think we agreed that it’s not.
That an LMS isn’t important. It’s rather, it’s a part of your, and we both love this word in customer education. It’s a part of your stack. I’m doing air quotes, stack. it’s part of it’s part of that suite of tools. What’s that
Adam Avramescu: [00:01:55] like a pancake,
Dave Derington: [00:01:56] like a pancake? Absolutely. it’s every bit as tasty, just a little digital.
All right. So it’s part of your suite of tools that you use on a day-to-day basis. And here today, let’s go ahead and talk about those big thoughts that what does an LMS actually do when might you need one or, in, why do you need one? And then what can you do before. You have one. So that hypothesis we’re going to test is the first piece of technology for a good customer education program is a learning management system.
So with that, let’s kick it off.
Adam Avramescu: [00:02:35] That’s a really interesting hypothesis, Dave, because I think a lot of the hypotheses we’ve been doing so far are a little more oriented around myths, right? Like we’ve been talking about things like, Oh, you don’t need a customer education program until you’re later in your journey.
And we know that’s false. I think this is the first hypothesis that we’ve really stumbled upon that, there isn’t necessarily conventional customer education was demand. So I’m excited to jump into this.
Dave Derington: [00:03:02] Yeah. Let’s tear this down. So why don’t you lead it off?
Adam Avramescu: [00:03:05] All right. So let’s ask that first question.
What is an LMS before we know whether we need one, we need to define what it is, right? not to belabor the point, but when I talk to a lot of customer success executives, they may have heard of an LMS. But they don’t actually know what it does. They don’t even know what it stands for sometimes.
So let’s talk about what an LMS actually does in the context of customer education. Cool.
Dave Derington: [00:03:30] LMS of course stands for learning management system. and let’s just go through some of the points here. I could start off saying, it, LMS is there too. fundamentally track student activity in that learning journey, meaning, Hey, I know what courses you took.
I know what grade you got on it, particularly when it’s on a, like an on demand format, but also for live e-learning or I’m sorry. But also for instructor led learning.
Adam Avramescu: [00:03:58] Yeah. when LMS was first started out and I believe that they started in the world of academia before they even came into the corporate world, they were really more about managing enrollments in live courses.
And so when we talk about what an LMS does, I think it’s important to realize that they’ve come from a completely different world from the world that they’re currently being used in. they came from academia and there were a way for people to track, enrollment and progress in those courses.
and then they spent a long time in the world of internal L&D being able to track, what, internal learners at their organizations were doing, maybe going to live classes and maybe completing some activities outside of class. so you had these LMS, like Blackboard and canvas, and then eventually, Moodle is another one that people were using a lot back in the day.
And then they started to explode. It went from there being maybe like five or 10 LMS is out there too these days. I think the figure I hear is over 700 or 800 LMSs.
Dave Derington: [00:05:01] Good Lord. That’s a lot.
Adam Avramescu: [00:05:02] Yeah. So they’ve gotten more sophisticated over time. And they’ve also become more specialized. And I think that’s why we’re seeing so many, because LMS has have gone from this thing that just lets you track enrollments and completions in a course to things like letting you actually author content directly in them, giving you portals to share that content and actually presented directly to learners.
So it’s not just a backend system anymore. And a lot of them have all these bells and whistles in them like micro learning or gamification or whatever the, the buzz word of the month
Dave Derington: [00:05:34] is. And that’s actually getting somewhat confusing and it makes it, I think evermore difficult to really say, this is the one I need, or this is the one I want.
And it’s that fact that they’re becoming more specialized. and you
Adam Avramescu: [00:05:48] have, you have an LMS now for customer education and that’s different from an LMS for, internal education and that’s different for an LMS for. for lifelong learning for professional organizations, everyone has different needs and the market has expanded to meet those needs.
And then
Dave Derington: [00:06:03] there some market confusion in between, but Hey, I’ve got a question for you. Yeah. so I think at one point we were talking about the TSIAand there was a quote and I hate to put you on the spot here. I guess you could spit ball it with, with an approximation, but did you not tell me at one point that for education services and LMS is.
The most purchased system, but not just that. And I think he said this it’s also the most disliked.
Adam Avramescu: [00:06:31] you did hear that from me. So TSIA the technology services industry association. They do a lot of really good work with education services organizations, and they do a, I believe it’s an annual report just on the state of the industry.
And one of the things that they look into is the tech stack. And so they break down by percentage. Do you have an LMS? Do you have a content management system? Do you have a, certification platform so on and so forth? And LMS is, at least the last time I looked at that report were the most adopted, but then they also go in and measure.
Are you satisfied with those systems and LMS is, I believe we’re good last at the time.
Dave Derington: [00:07:12] we’ll have to break into it. Maybe that’s an entirely new podcast right there in and of itself.
Adam Avramescu: [00:07:17] I think there are some out there about that.
Dave Derington: [00:07:21] so maybe one more point on this, baby, go back to the past here.
And I recall you were talking about how in the past, you really needed to comb through some random LMSs to find the one that was the most customer centric, because again, HR internal is a little bit different from that customer to get you to. Can you tell me more about that?
Adam Avramescu: [00:07:41] Yeah, I don’t know about you Dave, but.
The first time I ever looked for an LMS for a customer facing organization. There wasn’t really the wealth of customer LMSs that you see today. So really what you were doing was you were looking at the internal, HR L&D based LMS, that would stand the best chance of being able to be used for customers.
So I remember the first one I ever picked when I was working at a company called bank view. Is, it was one that was part of a larger talent solution. So they had, a performance management system and they had an LMS and they had a talent management system and all those good things. but really you could tell by that suite of products, that this was something that wasn’t designed for external or what you’d call extended entrance price learning.
it really was something that was designed for internal learning. So there were just, weren’t a lot of systems at the time that really handled that extended enterprise use case. And that’s changed.
Dave Derington: [00:08:38] Yeah. and what is it about those internally focused systems? Is it they were inherently less polished or they had to have the benefit of being on a corporate land at high speed.
W what was it that differentiated those from what we want to surface with the customer?
Adam Avramescu: [00:08:54] Yeah. at the time, maybe I’m going to be a little snarky here. ,
Dave Derington: [00:08:59] it’s funny, like
Adam Avramescu: [00:09:00] 1995, Microsoft.
Dave Derington: [00:09:02] Oh, goodness.
Adam Avramescu: [00:09:03] Yeah. And this was not 1995 when I was looking for these. I want to clarify it actually wasn’t that long ago.
So there wasn’t necessarily that, that Polish placed on the user experience or on the interface, there are a lot of bells and whistles that just weren’t necessary for an external customer situation. And you could also tell by the types of integrations they did. When you talked about user management, for example, in the LMS, they were really touting being able to sync with your HR information system or other internal systems that really tracked your employee base.
And that’s just not really going to work for customers, And customers, the source of truth. Isn’t your HRS or like an LDAP integration or something like that. It’s your CRM, your customer relationship management software. It’s, it might be your product itself. So even on a data level, these systems weren’t necessarily reflecting the reality of how you train and how you measure customers.
Dave Derington: [00:10:08] It was all internally focused with different kinds of probably similar, but different nuances and different kinds of end points, which that’s really cool. and also a lot of the good, the really good customer. focused ones. I would also say have more Polish because you’re working with people you want to impress, you want to integrate well that’s
Adam Avramescu: [00:10:28] between the first time you were ever looking for an LMS and today’s market.
Dave Derington: [00:10:32] I do see that a lot more of them are starting to get, and we probably can get into this a little bit later where they’re starting to get a lot more integrative with your product. For example, like you look at products like, I’m trying to think of the word, walk me, the ones that are embedding training within your product itself.
and that’s a different nuanced approach because you’re almost micro learn, but that didn’t exist a long time ago that coaching you through it, it’s a different modality of training, but I’d still argue that it’s part of that whole enablement and education process.
Adam Avramescu: [00:11:04] Yeah. And one other one that I thought of Dave and I remember this being a huge limitation at the time.
Was that you had so many of these LMS is a pricing set up per seat because you knew how many students were going to be taking your courses. There wasn’t this variable demand like there was for our customer education.
Dave Derington: [00:11:22] Yeah, absolutely. Cause at one point, I recall working with, the last LMS I was on, we had something like 33,000 people had enrolled in the system, which is just a staggeringly large number.
Adam Avramescu: [00:11:36] Yeah. maybe we can actually use that as an anchor point to get into our, the next part of our discussion, because we really need to think about when do you need an LMS? and so I’d love to hear some of your stories about how you were able to get LMSs implemented in some of your past roles.
Dave Derington: [00:11:53] Cool. that’s a good bridge into it. This is hard. This is harder than you think. And I think, to caveat this, if you’re listening to this, you don’t have an LMS. One of the things that I’d encourage you to do is just use this podcast as a time to reflect. Do you need one? we’re walking through all these things cause we’ve been there.
So here’s one of the lessons and I’ll tack onto what I had said before. one of my last roles I was at Gainsight and we had implemented, learn DOT’s LMS. it’s a product from surface rocket. And initially we had offered training for customer success managers on thought leadership.
So that is how do you be a, how can you become a great customer success manager and all the thinking that goes around that it wasn’t so much technical product training. And then I came into the picture somewhere in that process, and now I had to adopt. this platform for more, a different cohort of customers, right?
So we had a tremendous volume of customer interest. we had some basic customer material and then basically we dropped all that material into the system. And then, it’s like a backwards process. We had this, we had to use it and was I ready for it? I don’t know. because.
Because at first, if I didn’t have a whole lot of material, I was more adopting to what the platform could do for me rather than me spearheading and leading that adventure. So what are your thoughts? What kind of experiences have you had?
Adam Avramescu: [00:13:21] Yeah, I think my experience has been a little bit different in the sense that I’ve never actually walked into an organization that already has an LMS where I’ve had to either adapt to it or rip and replace.
for me, I’ve usually walked in during the fakest stage. and then later we have to figure out how to make it. for instance, the company I was talking about before, bank view, we didn’t have, we didn’t have e-learning, we didn’t have a way of really tracking the trainings that we did. I think some of them were going into the CRM.
if a trainer went out and did a training, they would log it as a training record in our, in our CRM. But that’s not really going to be able to do much more than track activity, right? what a super old school LMS would have done. So right. But we have to figure out was, Hey, now if we want to have all these customers coming on and taking e-learning and we really want to be able to do reporting back to them on what all of their individual users have done, we need to start figuring that out.
the fake it stage for us was we continued using the CRM, but we also. Started using the system that, I believe still exists, called articulate online. And so it was using the e-learning offering software that we had, which is called articulate. It’s pretty common one. they actually had a hosting platform where you could, where you could post the learning online and they would give you reporting on what people were doing in there.
And then eventually we were able to use that kind of as the beachhead to eventually implement a real learning management system. And this is the one that I was saying earlier was a little bit more of an, a, an HR focused one, an internal one that we were just somehow able to use as well, external LMS.
Dave Derington: [00:15:06] Oh, that’s cool.
You know what, let me expand on that because I like to think that you and I are lucky in those opportunities where you come in and you could fake it. w you’re building it out. I’m doing that right now, again, with Azuqua. And I actually tend to prefer this because we have decided to build out all of our material on essentially a document portal and.
And so I’m basically focusing on the different modalities of training. I have a video for those that like to watch and listen, I have the script or, the notes and then I have workbooks. And so I basically have all of the assets you would otherwise have in a learning management system.
I just don’t have them in a learning management system. yeah. Yeah.
Adam Avramescu: [00:15:47] I see companies doing this more and more, and in fact, that’s what we did at Optimizely. When I first came in as well, we created a. A version of an LMS that presented all the contents and all the assets and all the yeah. Paths basically, but it was built on top of a Zendesk help center.
And so while it looks like in LMS, it didn’t have some of that basic functionality in terms of tracking student progress or being able to show you what courses you had or hadn’t completed. We were doing all the analytics on it in Google analytics. And so finally when our customer success team. Really wanted to have that in-depth tracking of what their accounts were actually doing.
We were able to say, we should probably implement a learning management system to do that.
Dave Derington: [00:16:31] But I
Adam Avramescu: [00:16:31] see this pretty commonly, right? Like startups are always asking about the ROI of the LMS. Have you run into this
Dave Derington: [00:16:37] constantly? and does this come up a couple of times at a recently of how much would it cost us?
And it could be expensive at first. You don’t know, if you’re looking at a price tag of upwards of $30,000 for a platform, it’s a little jarring. yes, you could get cheap ones, but. You’ve really had to think about that. We’re in that mode right now where we’re thinking of, do we want to build out all of this training on our own platform and spend all that time, which is non-trivial or do we want to drop this amount of money and then have the added complexity of working with a vendor, getting this product in and it, there’s a lot to think about the build versus buy argument is, do we really have the budget for it?
What’s it really going to give to me? Or, and then there’s that risk too, when I start thinking about it. Yeah. I probably need a learning management system now for tooling, for reporting for all of that stuff. there’s a lot under the hood. and you have to like really lay out what, when you’re going to need this.
I don’t think I need one quite yet. I think I’ll need one going into next quarter, because I’ll have all my content developed and then it makes it really easy. What else would you think of?
Adam Avramescu: [00:17:45] I think a lot about total cost of ownership or TCO as
Dave Derington: [00:17:49] you hear it called. Yeah.
Adam Avramescu: [00:17:51] So when you think about build versus buy a lot of the time, you’ll see companies shutter at this.
Oh, we’re not going to pay 25, K 30 K 50 K, whatever it’s going to be. that’s what I would estimate the cost of an LMS for kind of a small to medium sized company being. but then when you start to go the alternate route, you don’t necessarily think of the hidden costs of doing it yourself. So either in terms of losing functionality.
So for instance, you might not be in a situation where you can track learner progress the way you want. Like we had it optimized early, right before we implemented an LMS, or you might not be thinking about the costs of. Custom development. you talked about added complexity from working with an LMS vendor, but a lot of the time I think an LMS vendor can help you simplify some of that complexity because they’ve already thought through the product development and you don’t have to recreate that yourself.
Dave Derington: [00:18:44] Certainly, of course you have to have the right fit and the right pairing with that LMS product. That’s good for you that if you get something and I’ve done this several times, if you get something perhaps that isn’t flexible as you’d want it to be meaning, I want it to look like my product. I want it to have the same styling, all that jazz.
And I have to fight that’s difficult, but if you have something that’s a little more attuned to your specific needs, maybe you need to have integration or something like that. So all these things. You really need to be thinking about, and if you’re starting to ask those questions, why you need an LMS and you’re starting to have answers to them, then you’re ready.
Adam Avramescu: [00:19:22] So I actually think that’s a really good transition because the next thing we wanted to talk about was what do you actually need in an LMS? What should you be looking for? So what’s on your high priority list, Dave.
Dave Derington: [00:19:35] Oh, my gosh. I think this is a really good case to bring up what happened.
it was a good experience with Gainsight because Gainsight was very focused on that customer success outcome. the journey because we want people to adopt, and this is what we want is customer education, people that Adam, I think you keep the mantra up. customer education is a pillar.
Of customer success, maybe that’s not for all people, but I believe it holds true. So number one, I would say, at the adoption phase, once you have customers using your product, a customer success manager, whether they’re at zero touch, low touch all the way up through scaling to an enterprise grade, I want for them to be able to hold a customer accountable and do things like reporting.
Now, what it mean by that is let’s say I’m at a particular account and I’m monitoring that account and it’s a little Rocky and they’re not picking stuff up. And somebody calls me. Repeatedly and repeatedly. And they add, they asked me the same kinds of questions as a customer success manager or anybody within the company, engaging with them.
I’d want to know that they’ve at least looked at the training material, particularly when I say, Hey, we’ve got really good training material. You don’t have to call me all the time. And then to be able to expand upon that as well. And let’s say I have, I’m working with a project manager at. at an organization and they’re trying to scale the adoption.
We want to be able to give them a reporting and to have them know, yes, there are efforts to help train and get people to adopt all that material is there. So that’s one thing that I would tap on.
Adam Avramescu: [00:21:12] Yeah. I’ve definitely seen a lot of customer facing. LMS platforms, place emphasis on that, where they have a dashboard that CSMs can use, and then maybe they have even a separate level of permissions where you can put your point of contact from your customer side, into the LMS as well.
So they can self-serve and report on their team’s progress.
Dave Derington: [00:21:32] Cool.
Adam Avramescu: [00:21:33] you also want to give that user level progress, right? So as
Dave Derington: [00:21:37] totally.
Adam Avramescu: [00:21:38] As a customer or as a user from your customer side is going through the learning paths or going through the courses. You really want to make sure that the LMS is doing a good job of showing them their level of completion.
And this might even tie into email nurture where a lot of them, a lot of platforms will let you send emails based on incomplete courses, or maybe you visited the site, but you haven’t enrolled in anything yet.
Dave Derington: [00:22:06] Yeah, that’s fantastic. And playing on that, I’d like to give props to Salesforce, Trailhead, have you used a Trailhead or at least peeked at it?
Adam Avramescu: [00:22:15] I have, it’s one of my favorites. Customer-facing learning platforms.
Dave Derington: [00:22:19] It’s amazing because it’s all there. there’s a tremendous amount of content, but that I love the badging. I love being able to feel like I’ve accomplished something and you’re making progress yet. I could do it on my own time.
So that makes you. That makes you, at least me a more motivated to learn.
Adam Avramescu: [00:22:37] Yeah. And I’ve seen this in the more recent crop of customer LMSs that have come out in the past few years, they have badging and gamification and things like that really make the experience a little more enjoyable.
and so it allows a lot of people to have that Trailhead like experience, even though they’re not necessarily going to build their own Trailhead platform. Totally.
Dave Derington: [00:23:01] what about, I had a question for you on e-commerce is I think you’ve mentioned this before. let’s get to the point now we’re actually charging for training.
What could you tell me about that need, that specific need within LMS?
Adam Avramescu: [00:23:13] Yeah, I’ve seen LMSs do this in a few different ways. whether it integrates with a system like Stripe or however you’re doing your payments processing, or it might even be. And integration with your ERP or figuring out how to do invoicing.
Things get a lot more complex when you start doing e-commerce. So I don’t want to under emphasized that it’s almost as complicated, I think, as internationalization and localization, which that’s a whole other thing. but two of the most complex things that you can do, I think the big question to ask is.
How are you going to be selling training? who’s going to be doing the selling and what does the actual transaction look like? So for instance, if you’re in a position where your, a sales team is going to be. Adding training onto the contract. And that gives you access to certain courses with a coupon code.
Then you need to make sure that the LMS that you’re purchasing handles coupon codes really well. on the other hand, if you have a model where you expect individual users to be coming on and buying, certification courses or just making those transactions themselves, then make sure that there’s a good integration with a billing system.
that lets them do e-commerce right there on the site.
Dave Derington: [00:24:28] And then you have the alternative to that is, I think you were alluding to this. I might have a sales person negotiate in your initial contract or renewal, a certain number of seats to training for your, for your employees. That could be like a token or some other thing where you type in that code.
I think you were already talking about this and make it
Adam Avramescu: [00:24:48] or training credits or something like that, so that your courses still have value. they still have a price tag, but that price might not actually be paid directly. It might get carved into the license fee for your software.
Dave Derington: [00:25:01] And I think, we’re talking a little bit that this was like that tail end of the small business, mid-sized business, we’re actually building the educational services team out. The other thing in there, and I’m missing the word right now for this, but it’s confirming that you’ve had an individual.
actually revenue recognition. That’s a big one for finance because I’ve had people come back to me and say, did so many people get training? Did they take it? How can we improve it
Adam Avramescu: [00:25:27] the same as you actually
Dave Derington: [00:25:28] have to do that at some level, if you’re not getting where
Adam Avramescu: [00:25:31] probably if you’re going to be doing a lot of paid training, you really want to bring your finance team in on that, vendor evaluation.
Dave Derington: [00:25:51] absolutely. Yeah, that’s a good call. I’ve got one other one Adam to shatter. this kind of expands upon what we were talking about before with just customer progress. and I’m sure you’ve experienced this too, when you’re. Working with, ISV, a partner that’s maybe extending what your company does.
Maybe they provide installation services, integration services, implement implementation is the word I’m really looking for. so let’s say I have a cohort, partners with those partners are a little bit different from a customer because a customer can self-help. At a certain level, with a low touch type environment, but a partner.
I really want these folks to know that they have the material to be able to get to it easily. But I also want to be able to go back at the end of the day and say, look, I’ve also vetted. You vetting means it’s not just badging. It might be another degree of certification where I make sure that partner is certified and I need to have a place where, You’ve done this, right?
You have a partner team. Like I know I’ve trained folks from Accenture and Deloitte and other big consulting firms and they say, Hey, can I have the report for all of my team members to make sure they’ve done this?
Adam Avramescu: [00:26:59] Yeah, for sure. I think that’s important. And one thing to think about is that something you’re looking for your LMS to do?
or is that something where your partnership team might have a partner portal? and that does some of the tracking as well. So you might either look for an integration there, although in my experience, those systems do not integrate very well with customer LMSs.
Dave Derington: [00:27:22] in a lot of time, Adam, part of that whole process is a, an in-person one where I’m having somebody prove to me that they really know it by demo.
Adam Avramescu: [00:27:31] Yeah. however, you’re actually certifying partners. It might be a matter of thinking about just how you imagine that use case would actually go either in your partner portal or your LMS. Totally. Yeah. Dave, you’ve mentioned, even in this session today, the importance of being able to customize and brand, the LMS.
Can you talk a little bit more about what you look for in an LMS there?
Dave Derington: [00:27:54] Oh, absolutely. And I’ll give you a practical example at the last system I work with and I would definitely give profits to service rocket for the learn dot product, because I think it was an awesome PR or it is an awesome product and it’s evolving.
one of the things that I sometimes struggled with, and this is good and bad, so I’m not criticizing or judging here. I personally like to be able to do modifications quickly in that means, let’s say, let’s think about your website. So we’re not talking about training, we’re talking about a website.
you need to add content to it. Yes. That’s the same thing you do in an LMS, but sometimes you need to move things around and lay things out differently. For example, let’s say our navigation panel at the top of the website, the customers aren’t finding something they need, or they can’t get to something very easily.
So that actually translates because often. And with Mo introduction of Mobilist isn’t this always the same anymore, but often your LMS needs to be a little bit more of a seamless experience. So for example, I found myself in situations where I wanted to have like markdown formatting or have multiple pages on some documentation that would associate with a video or I wanted to have a learning journey that assembled, Like the same content in different structures.
And again, I struggled with that and I found him in one case, I wasn’t able to make like simple cascading style sheet changes and I had to work through a third party. so things like that, like I want to have some for me. I want to have some fine grain control over things that I can change quickly and not have to open up like a ticket and spend a couple of days changing something.
Adam Avramescu: [00:29:26] Yeah. I think when I evaluate a learning management system, one thing that I look for is. Show me the level of control that I get over. Being able to modify the CSS or be able to add JavaScript, or be able to change some of the HTML of the individual pages. do you have an interface for that? How does that work?
But that, the other thing that I like to do is when I’m working with LMS companies, I ask them to show me a few examples of sites that their customers have built. And I always ask to see the most customized one. a moderately customized one and then one that looks pretty out of the box, just to understand the extent of the customization.
And then I like to follow up by asking questions about how some of those customizations were done or how extensive the CSS or JavaScript work was. And I also like to ask. Sometimes you see with LMS is that they have, a custom built landing page in front of the actual LMS. And that’s not even living on the LMS site.
That’s just something that they built somewhere else in a different system. So try to figure out when you’re actually looking at the LMS versus when you’re looking at a custom built landing page or something that lives outside of the LMS site.
Dave Derington: [00:30:40] Yeah, that’s super, that’s a really the golden kind of approach.
I like that where you’re looking at the low, mid high customization, probably even talk about the cost involved because the caveat is if you have something really awesome, then it might take a third-party to do, or even a web developer to help you with.
Adam Avramescu: [00:30:59] Yeah. And the more you customize the system, the more you’re going to have to maintain it because the system itself is going to update over time.
Things are going to change and you want to make sure that if you’ve customized that site enough, that you can also continue to maintain it as it changes.
Dave Derington: [00:31:15] Yeah. with that, why don’t we talk about, so we’ve talked about the. Other parts of an LMS. We’ve talked about the, why we’ve talked about the wind.
We’ve talked about the, what is an LMS to begin with. If you don’t even know, I’m sorry, let me say that again. Sending a route. and the, what if you don’t know already now we’re onto the last part of it, which is okay. I don’t have an LMS. And for those of you already do have one or, are happy with it.
Might, kick back and listen and see if you agree with us, but what can you do before you implement that LMS?
Adam Avramescu: [00:31:49] Yeah. So Dave you’re in that position right now, right? You don’t have an
Dave Derington: [00:31:53] LMS. Totally. I don’t have one. with the last one, I already shared that story, Adam, that I inherited the system.
I liked it. it did a good job, but I felt rushed and I didn’t feel like I was a part of the process. And in fact, I really feel like I did not know the requirements that I would have come six months later so that, the first thing I would recommend is before you go out and buy an LMS, really think about what you need out of it.
what is it that I have? what do I want it to do? but the point to that is don’t rush. Take your time. Yep.
Adam Avramescu: [00:32:31] Absolutely. And so before you do that, we talked about going through a fake at stage and then I make it stage. What can you do as a prototype to prove the need for a real LMS.
Dave Derington: [00:32:44] Oh, my goodness. So I’ve decided I, we have a system that we’re using right now. It’s actually called Hugo. Maybe you haven’t heard about this. It’s a, it’s very techie, very geeky. I love it. And of course I would. it’s a stack and it requires me to do a little bit more with get hub and I have to check in and check out, blew my mind at first, I’m an educator.
So I learned, that. Yeah. I was able to lay out a training and education and blocks of content as a developer would, which is actually a good, because I have to work with developers pretty regularly. So I have spent a lot of my time at first, I just built little video blocks and I put them on a regular webpage on our website.
And then as I was able to expand those and add to them, the exercises, the workbooks, all the different. accessories to learning. Then I started to relay them out into small modular components with landing pages and journeys. And so now I’m at a point where I’m pretty darn happy with this. I’ve got basically a map.
I’ve got a lay of the land. And I could share that with a customer. and, but then that starting to precipitate that need for, I want the system to actually help coach you through that and do things because a website can’t, so I’m really in a good spot. Come the end of the quarter, come the end of the year that I’ll be able to have really great discussions.
Say, these are all the things I need. This is all the stuff I’ve got. How can you help me?
Adam Avramescu: [00:34:05] That’s great. the other thing as you say that really makes me think is that if you’ve had a good trial phase and you’ve really experimented to figure out what’s actually important, you can start to circumvent this common mistake that I see people make an LMS evaluations where they just show up with this really long RFP or were they long features list.
And they’re trying to find the LMS that checks the most boxes. but then the experience. Of actually using that LMS or the experience of actually working with that company as a partner, is inferior because they’re trying to do Zayn their platform around checking boxes, not around creating a coherent experience for you as an admin or for your customers as users.
Dave Derington: [00:34:46] So are you saying not to have that checklist or are you saying don’t offer that as the number one key to succeeding in RFP?
Adam Avramescu: [00:34:55] I’m saying that you should absolutely have a checklist. and maybe in a moment we can get into a few other things that you might look for, even though I think we’ve covered the most important ones.
the key is to know what is most important to you and to really be able to prioritize that checklist. So for instance, when I do a feature requirement doc for an LMS, I actually like to prioritize the features as must haves, nice to haves or, this actually isn’t really important at all.
Dave Derington: [00:35:22] That forces
Adam Avramescu: [00:35:22] me to have an honest conversation with myself about what’s necessary.
Dave Derington: [00:35:26] It’s always going to be different. can you talk about some of your experiences? I know you’ve talked about. the Zendesk LMS and articulate a little bit. Can you expand on those with, with regards to going towards an LMS?
Adam Avramescu: [00:35:38] Yeah, absolutely. And in both cases, I think we knew that eventually we’d want to have an LMS, but we were using each of those, fake it LMS experiences to validate something that would lead us towards getting a better LMS later. So with articulate, for instance, all it’s really doing is.
Presenting the content that you’ve developed in the actual offering software. So you create the content in at the time, I think it was called articulate presenter, and now they have a new product called articulate storyline. and then it just posts it and it tracks student completions, super simple functionality.
So we were just using it as a way to deliver the content through these deep links. we didn’t even have. I think dedicated persistent users, we were constantly having to wipe users out of the system so that we can open up seats for new ones. So we were really trying to create this, kind of customer management approach.
We were really just trying to figure out is this learning that our customers will use? What feedback do they give us based on that? and if we start to move towards an LMS, do we want to make that investment in having a way to. Give self-paced learning to our users is in desk. It was almost the same thing.
We wanted to see whether people were using it, but we were actually trying to figure out something additional there, which was, we had all of our courses assembled into learning paths. So we wanted, first of all, to validate are these paths that resonate with our users. Do these actually reflect the optimized customer journey that they’re having?
And so there, even though we were just using Google analytics to do the measurement, we wanted to track drop-off through the courses. We wanted to see which courses and which articles and which videos got the most traction, so that when we actually moved to a real LMS and we did end up implementing a customer LMS after that, that we could actually validate the structure.
And whether that structure actually resonated with our real learners.
Dave Derington: [00:37:33] Oh, that’s really cool. And I know I’ve done a little bit about that with, with the Wistia video platform. again, it’s a fake it till you make it. I started to put videos up, but I could see hot spots and places where people are particularly attentive, therefore testing out with data, what things people are proactively consuming, what they’re really liking and where they drop off that drop off stuff is super cool.
Adam Avramescu: [00:37:56] Yeah. And I’ll say that, I think for a lot of companies, this is cyclical, right? they do the fake at stage for a while where they use something either home-built or repurpose a different platform to be their LMS. And then they do invest in a customer LMS. and then at some point for some companies, at least they get to the point where they need to home build again, because they’ve just.
Outpaced the functionality of a lot of customer LMSs that are out there. So we got to that point at optimized lead, for instance, where we had been using a customer LMS for a while. but we didn’t have the ability to customize it as much as we needed to for our customers. So we actually had to end up building our own.
Again, on top of a headless CMS,
Dave Derington: [00:38:40] Adam, I wanted to make a, a note here that this is really good because finally I don’t feel alone that I feel like, Like I developing the. Educational content for one thing is always an iterative process. you make it and you scrap it, you redo it, you make it, you scrap it.
And I think a lot of times, and I felt this, you get beholden to basically engaged to a platform and then it’s really hard to get off. And you’ve got to look at that. Like here’s an example, one prototype that I’ve done in the past. And again, this was more specific to like testing and certification is I use the Moodle platform.
Now Moodle is free. If you want it to be free, you can download it and install it on your own server. If you can. There’s also a Moodle cloud and the company that makes us is in Australia and it’s $250 a year, but it takes the best of the product and actually gives you some support. that was a really good experience because I was able to test and explore and then shut it down at the end.
And I didn’t want to have a lot of out-of-pocket. so that iterative, like you’ve got to be just like you are with being a bold and fierce educator. Do the same thing with your LMS realize. I’m probably going to scrap this. I’m going to get stepping stones and what I have today, isn’t what I’m going to have.
And that goes all the way to, like now you’re seeing newer systems. I think we could talk about, the LRS or X API. You can’t just run. You’ve got to crawl. You got to walk and then get to where you’re going to be. It’s an evolutionary process.
Adam Avramescu: [00:40:12] Absolutely. I’m glad you brought up. LRS is an X API is because.
At this point, you’re starting to see a lot of emerging technology that starts to move us away from the idea of the traditional LMS. So I think even the customer LMSs that are out there probably wouldn’t like us in this podcast, calling them LMSs. They’re trying to brand themselves as. Customer training platforms or customer learning platforms or things like that.
So maybe eventually there’ll be an entirely new category. but at the same time, there are other technologies that are starting to emerge. That if you don’t want to go the traditional LMS route, you don’t have to. LRS is for instance, that’s called a learning record store. And what it lets you do is basically create this.
this area, I think I’ve heard it described as a data Lake where you can basically pass different statements like through an API about learner activity, and be able to have those tracks so that you can do reporting on them. And so our learning record store is different from an LMS in the sense that it’s not just tracking the traditional learning completion and learning enrollments.
It’s really trying to build this database of all of your learner activities so that you can report on it in fundamentally more interesting ways. And I think at the same time, the customer learning platforms out there are also trying to move in the same direction through integrations with CRM systems and things like that.
Right.
Dave Derington: [00:41:41] and I think in, I think Rob and ADA hit on this at one point from surface rocket about how we want to reach the learner in their moment of need just in time learning. So you might even be on Facebook for work, w what do they call it workplace? You might be on Slack. I recall having a conversation with, somebody in Slack and in a AMEA, and they’re like, Hey, I just want it to be, and they do a good, you talked about this before.
Slack does a great job of that in, When you need it training on platform training. so that’s super exciting. I’ve really want to see how that market grows as we continue to evolve.
Adam Avramescu: [00:42:14] Yeah. I’ll just say anecdotally, this most recent LMS search that I did, for Checkr it’s one of the first times where I didn’t feel.
Depressed about the story of, learning management.
Dave Derington: [00:42:30] Oh
Adam Avramescu: [00:42:30] my gosh. She felt hopeful because there are interesting options that are starting to come out to meet the need of customer educators like us.
Dave Derington: [00:42:38] Yeah. how about you double down on that? Let’s talk about what to look for. You’re ready. What are you looking for in that?
And what are the options that you and I are going through?
kick it off.
Adam Avramescu: [00:42:51] Yeah. So let’s recap. A few of the ones that we said were important from our earlier discussion. First of all, there’s gotta be a focus on the user interface and the user experience. You have to be able to make sure that the out of the box experience is user-friendly and that there’s enough options to be able to customize it through CSS or maybe Java the script.
Second of all, you’ve got to have contract content, authoring capabilities. that doesn’t necessarily mean that the LMS needs to be able to host, or build interactive courses and things like that. You’re probably still going to be using other software to create videos and to create e-learnings and things like that.
But you should at least be able to author and publish some content directly within there. So for example, text lessons or pieces like that, maybe quizzes. and the third piece that we had already said was really important was reporting and analytics. it’s imperative for your system to be able to track what learners are doing in there, and to be able to report back on that so that you can continue to not only deliver that information to your CSMs and your customers, but also, so you can refine the content over time.
Yeah,
Dave Derington: [00:43:59] can I extend on that? Yeah. So what to expand upon is one of my biggest gripes with, with the platform is I’d like to have a lot of fine grain control of what I do with that data. And sometimes it’s not often enough just to have some basic canned reports. sometimes I’d really like to be able to send a targeted report to a team leader.
Over a bunch of learners. And a lot of this stuff doesn’t quite exist in this modality yet, but I’m starting to see, for instance, again, with Learndot they added in an analytics layer and I can actually set up some rules and do some really cool things and send somebody an email, send out reports either that it will get to integrations in a minute, but I want access to my data.
Adam Avramescu: [00:44:41] Yeah, absolutely. we ended up choosing Skilljar at Checkr and one of the big reasons. Why we chose them was because of the way that they set up their data integrations. So my philosophy has always been, instead of using the native reporting, that’s directly in the LMS as the source of truth, or even instead of using the CRM as the source of truth.
I like to build data directly into our data warehouse. So that we can start to do better reporting of our learning data against other systems. So whether that’s coming from the CRM or from the product itself, or from another system, I like to be able to merge all that data directly, where it’s being stored in our data warehouse.
And so one thing I appreciated about the way that Skilljar built their data connector was that it made it easy to pull those tables directly into our data warehouse. So I think. Whether you’re using integrations or whether you’re doing a direct, data export, really think about where that data is going to be going and how reporting and analytics are gonna work.
Dave Derington: [00:45:40] Oh, yeah. Can I add, I would expand on this one as well. I’m going to jump ahead to integrations because at Azu CWA, our big thing is integration is integration, particularly with an API. And one of the things that I love more than anything, and this is how I actually went to work for Azuqua to begin with is that I was taking my data out of my.
LMS. I was aggregating it, rolling it up, but I didn’t really have a good way to, I had an integration, but it was really painful to use to get the business rules around it, to get it in a format I could use, basically I wanted to have one row in a database per learner, per course, with. All of the associated information about, how long they took on it, what their quiz scoring was.
And that was painful. So with me and API, I was able to hook into one system and hook it into Gainsight and have everything I needed. So those kinds of things are stellar. And you may actually want to think about a product like Azuqua, if you don’t have an integration, but you have an API, those are the kinds of things that you can do to connect the point to point, because like Adam, you were saying, I think one of the biggest.
Assets to you is to connect into that customer success team and the bigger picture from a corporate level to say, is my, was my work doing something? Is it affecting the outcomes that I want? We’ve talked about that in a whole other episode, but that’s super important.
Adam Avramescu: [00:46:58] Yeah. And that’s one of the reasons why a Checkr, we actually moved directly to an LMS without going through that fake at stage, because we actually had a pretty clear idea at that point of.
How we wanted learning to roll into the customer life cycle. And we already had that buy-in so it wasn’t this complex ROI argument. It was really more, a matter of needing to understand how the data was going to flow between our systems and what reporting we wanted to offer to our customers. one thing that I did at Checkr that I wish I had done at previous roles as well was we brought our data engineering team into the discussion with potential systems.
So that we could evaluate their API APIs and so that we could actually understand their approach to the data. And that was really cool.
Dave Derington: [00:47:43] I’m super impressed that I’m totally going to do that.
Adam Avramescu: [00:47:47] Yeah. and you know what? They might not have time always, but at least in this case, they were really happy to be involved and they were thankful in a way that we were bringing them into this discussion, because if you’re a data engineer or if you’re a business operations person, or if you’re a.
Sales enablement or, roles like that, where you’re dealing with a lot of data and a lot of integrations. A lot of the time you get systems just plopped on you and their APIs are terrible, and you have no idea how to work with them. So if you can actually be part of that evaluation process, I really appreciate hearing their insights on how difficult this is going to make their lives later.
Dave Derington: [00:48:25] Yeah, that’s awesome. So let’s start to wrap up. What are the other things I think we talked about, e-commerce that? if you’re doing, if you’re selling your training, if we’re thinking about selling your training, you’re not even doing it today. Definitely be cognizant of the options you have at your disposal, does it integrate with, what you just mentioned before Stripe?
Are
Adam Avramescu: [00:48:44] there other common systems like that
Dave Derington: [00:48:46] other systems.
Adam Avramescu: [00:48:47] Another integration to think about is SSO or login. So if you’re not planning to have your students actually create accounts in your LMS, if you want to use your products log in or something like that, look at how they handle single sign on SSO, or off is another way that people do it. So you can either identify people by their actual product accounts. So for example, at Checkr, we’re having people log in using their Checkr dashboard accounts. or I’ve also seen people use OAuth where they use a third-party account, like Google or Facebook to log in to a different system.
Dave Derington: [00:49:23] That’s one of those things that people may forget. Get or pass upon because one of the biggest complaints that I had. One of the few complaints I had was that, Hey, this is too hard. I have to remember another password. Of course I use LastPass and I never remember a password anymore, but, we don’t prescribe that for everybody.
And nor would we expect them to have that. So that’s stellar. it’s about that you’ve tethered it to Checkrs login dashboard. That’s super cool.
Adam Avramescu: [00:49:48] Yeah, that alone might save you from having to get a training admin later.
Dave Derington: [00:49:51] Yeah. And trust me on that. I had to have a training admin from our last system
Adam Avramescu: [00:49:57] and, the last one, the thing that I really look for is good partnership.
again, it’s not just about checking those boxes on the RFP. It’s really about making sure that whoever you partner with approaches it in a way that is intelligent and thoughtful, and that this is going to be a partner that grows with you over time. Yeah.
Dave Derington: [00:50:14] And the last part of it is, and I think you’ve done this, full demo before you buy, click the buttons and try some stuff, loads and modules
Adam Avramescu: [00:50:22] and environment.
Dave Derington: [00:50:24] Totally because you might be, feel rushed and you might want to move into it. And a sales person will definitely encourage that, but goodness, you have to know what kind of, you’ll never get to the point where you’ve uncovered all the gotchas in advance, but you could surely enough beat down some of them before you get too far into the process.
Adam Avramescu: [00:50:43] An excellent closing thought if I’ve ever heard one. So Humboldt what LMS is, but, probably not our last.
Dave Derington: [00:50:51] Yeah, and we’ve gone quite long, but I think this is all fantastic. And now it’s my favorite time of the podcast. Adam, do the honors
Adam Avramescu: [00:51:00] welcome. You’ve got mail it’s time for our mail bag. today’s letter is from Barry from Panama city and he asks, my boss is obsessed with the idea of microlearning.
She keeps telling me that what we need to do is make our content more micro. Because our customers have short attention spans, but I’m worried that if it’s too short learners won’t have the right context. what do you think of this whole micro learning thing? They’ve
Dave Derington: [00:51:26] Oh, I love this topic because it’s quite painful right now.
I’ll give you my 2 cents and you can address as well, but, so here’s the deal. The first. Few weeks that I dropped into Azuqua and started to build out some content. I had this resonant cheer, like we’re going to make micro training and I go, I didn’t sign up for that. but I hammered them. What, in my impression of micro training may be, may close, more closely matchers.
Is that when people say micro training, they’re saying, Oh, nobody has time for training and that’s gotta be really small so I can get bite-sized chunks. Yeah. True. but that trend is not completely validated right now. and Adam, you and I are going to totally agree on this. And I fight myself on this all the time.
this podcast in itself is quite long, but. most of our training, our learning is just too long in customers respond well to what we’d call chunked learning. Let me tap on that a little bit. I’ve taught game design for almost 10 years. And one of the things that you hear at here about when you learn, I think rave Coster has a really good book on those that we as humans.
Chunk information into related content. and also you could really only grapple with probably between five and nine things in your brain at any given time. This is not a slide on anybody. This is psychological research that tells us this. So what our goal really should be rather than going micro, like somebody said, you’ve got to make an under a minute and I go, it’s not going to happen.
three to seven minutes is probably a good target before switching on to something else. Things that are related, get material that makes sense worked packaged together. It doesn’t mean that you can’t go and make longer form material or workbooks or other documentation, but that core, that meat, that you’re really trying to communicate should be small.
I’m going to actually defer or try to stay away from the micro because micro might get too small.
Adam Avramescu: [00:53:22] Yeah. how micro is micro? I couldn’t agree with you more, Dave. I think that. When we think about micro-learning, I actually like to look at it as a healthy challenge to get all of that excess stuff out of the training, because a lot of the time, the reason why we’re doing hour long training, or sometimes even more is because we haven’t necessarily thought about how to chunk it.
But even when we start to get into that, like 10 to 15 minute range, I think a lot of the time we’re sitting there because we haven’t really done the hard thinking about. What content is actually going to support the learning objective that I have versus what’s the list of things that I think I need to talk about.
So the person will know this and I’m putting the word no in my air quotes too. So I like to look at it as a challenge. If the goal is to get the customer to comfortably adopt a certain feature, let me spend maybe, 30 seconds, maybe a minute, depending on how long we’re really focusing on why it’s important.
Let me do a quick walkthrough of what it is, and then let me give them a call to action at the end. Love it. Yeah.
Dave Derington: [00:54:32] one of the thing. And I’ve got an example to share with us. And I think you actually wrote down the point and engagement and length aren’t correlated. you were talking just now about focusing on stronger openers, the problems, the stories may be use cases that help people drive through longer content.
the example I wanted to share is that, we have one or two modules that are super long, at Azuqua one. The first one is Azuqua one Oh one. That’s intentional and it’s my best performing piece of e-learning. It shocked me. And because everybody was saying, Oh, micro. And I go, I have to set a learner up to actually consume that micro training.
And this one is my top all time, that I have the best stats on it. some of the smaller ones actually perform better at the end, but still this beats them all out as far as overall performance. So it’s not necessarily that you have to go to that micro. You just have to think about it. Another thing that would help, particularly with these longer ones is indexing.
So if you can index a video for example, or break up a document then, and make it searchable, and you mentioned SEO, get people to exactly what they need. And I know me, I might not have time in the moment. I might need to know one concept, but I probably was going to circle back.
Adam Avramescu: [00:55:42] Absolutely makes sense.
I think that supports the chunking idea really well, too, that even if you have a longer video, there’s still topics in there that people can jump between us as they need. Yeah.
Dave Derington: [00:55:53] Or skip entirely if they don’t feel that they are, they feel that they have a mastery of that concept.
Adam Avramescu: [00:55:57] Yep. Absolutely. And if the content is engaging in the sense that it really helps the learner reflect on what they’re learning, it helps them practice what they’re learning so that it becomes more real.
Then you’re right. Engagement at length are not the same thing. micro isn’t always micro and it sounds like we should experiment with different engagement, tactics and different lengths.
Dave Derington: [00:56:20] Yeah, definitely it experiments are something that both of both you and I, obviously we call this a podcast, CELabs, customer education laboratories, when it’s what we like to do.
so with that, let’s transition out. That’s a great question, Barry. and if you have a burning question about customer education, send us that question. We’re we’ll work to answer that in a future mailbag. Great.
Adam Avramescu: [00:56:41] So just to summarize this week’s lesson, let’s wrap up with a clear.
Call to action. So get out a piece of paper or download a template that maybe we’ll be offering on our site customer.education. And let’s get ready to answer a few questions.
Dave Derington: [00:56:57] Cool. first of all,
Adam Avramescu: [00:56:59] if you don’t already have an LMS, do you need one?
Dave Derington: [00:57:04] All right. And then if you’ve gotten past that hurdle, why is it that you need one specifically?
what is it that you want that product to do? Do you want it just to attract progress or hold those learners really accountable? And that means it’s attributable to customer success or even provide that baseline of tooling for paid offerings.
Adam Avramescu: [00:57:24] Yeah. And then think about whether you can justify the cost, do that total cost of ownership.
And to the customer, if you are actually able to track their learning and presented in a way that is more engaging and then also think about, do you have enough content to support an LMS or do you need to be doing more experimentation to
Dave Derington: [00:57:49] figure that out? Exactly. And then I think at the tail end of this, you’re determined, you’ve got to have an LMS figure out what is it that you need to be ready?
And then that means, and going to your last point here, do you have enough content? have you sketched that out? Have you done some trials? Do you need to do that? Are you, or have you done this before? And you’re good to go spend the time, do the research, fake it till you make it all those kinds of things.
And until, you’re absolutely you’re riding, you’re engaged and ready to go to them. The last portion, which is figuring out those requirements, right?
Adam Avramescu: [00:58:21] Absolutely. And so listeners, if you want to learn more, we have a podcast website@customer.education. no.com, no.org, no.net, no.ninja it’s customer.education.
There. You can find show notes and other material. And please, if you found value in it, this podcast share with your friends, your peers, over beers and your whole network to help us find it. The others who do what we do. So on Twitter, I’m @avramescu.
Dave Derington: [00:58:49] And I’m @davederington.
Adam Avramescu: [00:58:51] And as we say,
Dave Derington: [00:58:53] go out and educate experiment and find your people.
Adam Avramescu: [00:58:57] Thanks for listening.