Adam Avramescu: [00:00:00] We’re gonna do our vocal exercises next time
Dave Derington: [00:00:02] and blah, blah, blah.
Adam Avramescu: [00:00:03] So seashells by the seashore.
Dave Derington: [00:00:15] Welcome to CELab the customer educational laboratory, where we explore how to build customer education programs, experiment with new approaches and exterminate the myths and bad advice that stopped growth dead in its tracks. I am Dave
Adam Avramescu: [00:00:29] and I’m Adam Avramescu. Oh my gosh. I’m realizing that we don’t know.
National day of it is,
Dave Derington: [00:00:37] Oh my gosh. we’ve got to look that up. We
Adam Avramescu: [00:00:39] have just made the most grievous error that we’ve ever made on this podcast. Today is Brazilian blowout day. It’s also national spumoni day. whether you’d prefer a Brazilian blowout or a national spumoni well, I guess just a regular spumoni it doesn’t have to be a national spumoni.
You are good to go,
Dave Derington: [00:00:59] but now I love a good spumoni. I have to say that
Adam Avramescu: [00:01:01] I love a good Brazilian blowout.
Dave Derington: [00:01:04] I don’t really want to know what that is, but let’s just say it’s Pomone day and move on.
Adam Avramescu: [00:01:08] It’s not what it sounds like. All right.
Dave Derington: [00:01:13] All right. All right. Let’s get into it.
Adam Avramescu: [00:01:16] Red alert. Your CEO wants to start a certification program for your company, and she’s asking you to stand it up ASAP. What do you do now?
Dave Derington: [00:01:24] Certification? This can become a huge pillar of customer education program, but they can also be pretty daunting to think about don’t you think?
Adam Avramescu: [00:01:31] I do think.
Dave Derington: [00:01:32] How do you think Adam, about building the right certification program for your business? this is what we’re going to talk about today in this episode.
Adam Avramescu: [00:01:41] That’s right, Dave, and I think this is one of the areas that is most latent with myths and misconceptions. When it comes to certifications, there are a lot of them out there.
There’s a lot of, good information, but, probably an equal if not a greater amount of bad information. So we’ll try to tackle this one hypothesis by hypothesis and. Let’s maybe start with one around, at a higher level, what’s the business value behind certification programs.
Dave Derington: [00:02:07] I think that’s a really great place to start.
And you and I, and others that are listening in know this whole certification, let’s call it an onion. vis-a-vis Shrek, it’s onions, it’s layers. there’s a lot to think about. So starting with business value, I think is probably the number one, because we’re going to talk money. By tackling the business value of a certification program.
When you’re first asked to build this out. When you first asked or tasked with building a certification program, you might start off first by looking at what other companies are doing for their certification programs.
Adam Avramescu: [00:02:40] Incredibly common way to start Dave. I agree. and it’s a fair place to start because you want to see what format they’re using when they offer in terms of prep, courses, and exams, what formats they offer, certification courses and exams in what they’re charging and so on.
If you’re just starting a certification for the first time, you really want to know what other people are doing. So you can either model your program after that, or make it competitive with the ones that you see out in the market. And. So you might be tempted to really just model after another certification program that you see out in the wild.
But if your hypothesis is I should design my certification program based on how my peer companies do it. What do you think, Dave?
Dave Derington: [00:03:23] I think we can say that this hypothesis is proven false and here’s why. we go back to this unicorn thing. Every company is different. Each company makes different assumptions about how to set up certification programs.
And that’s going to be based upon the business rationale for having such a program.
Adam Avramescu: [00:03:43] Yeah. At what, what do you think are some of the major rationales? What are the reasons that you can think of for having a certification program?
Dave Derington: [00:03:50] just off the top of my head, what I’m thinking of is.
On one hand, it’s about qualifying people to use a company’s products and I work it out, reaching out right now. And I would love to have people who are certified outreach users in one capacity or another. You might also be qualifying a partner. A good partner is someone who ideally you want to go off and bless as, Hey, this person knows everything that I know they’re proxying as me.
And we want them to be able to service or deliver services on our behalf. You don’t want people using complex products without proving first that they know how to use them. And as an example, let’s think, Oh gosh, Salesforce, right? Most Salesforce admins have some sort of certification from Salesforce that.
Helps them prove that they can do the job. Let me give you a couple other examples. It’s not just only our market, the SaaS based market. but one good example would be Tableau. Tableau has an amazing program for certification. You can think of any number of certifications for, those higher end SaaS platforms, but then think Google has their.
A certified professional cloud architect, you might have Amazon’s AWS certified solutions, architects. And so there’s a lot of stuff out there that companies are offering these because they’re big a day, Shas large packages. And you just want to, don’t want to send somebody out with just a little bit of training.
You want them, and you want to know that they get the best practices.
Adam Avramescu: [00:05:22] Yeah, I agree. The more complex the product is, or the more technical it is, the more that there’s going to be some risk around someone using it, improperly, or really the more that it’s necessary just to give people a good way of building the skills that they need to be proficient and using it.
Dave Derington: [00:05:39] Exactly.
Adam Avramescu: [00:05:40] Exactly. And I see you have some, some figures here as well about, what a person who has these certifications can, can haul in. You want to talk about that?
Dave Derington: [00:05:49] Yeah. I thought that was interesting as I was doing my research and we always do a research or try to it just to give you an idea.
And I recall when I was at Gainsight, we started to thinking seriously about an admin certification. how do you bring somebody up with a complicated, sophisticated platform and what does that do for them? what’s the them, right? What’s in it for me. Just think about this. Google’s a cloud architect, certification, for example, you’re seeing a correlation, of salary income with that kind of certification.
So you could see people making in, roughly about $140,000 a year. On average, across the nation for the AWS certified solution architects, it could be about 132, and I listed one more in there. That’s not. A tech cert it’s the PMP, the project management professional assert that a lot of project managers are prone to get can net somewhere in the one 30, one 35 range.
So that’s nothing to shake a stick at that these certifications can translate out a certain degree of scale when everybody has them, it can be meaningful to your salary.
Adam Avramescu: [00:06:53] Yeah. And we’ll come back to this, but I think there’s a lot of value in offering certifications when they have a tie to your professional credentials or when they really signal something to the market about your skills.
but that’s not necessarily the case for everyone. So if you’re just looking at programs where really the output is by getting certified, I can perform a very specific job like the PMP or, by getting certified, I can use this highly technical product, like the AWS certification. you might be building your program upon some false assumptions.
If that doesn’t also apply to you. for all of those certification programs out there, I see, especially with a lot of newer businesses, they’re building certification programs for different reasons. So for some it’s about hitting customer milestones. so one example I can think of is a pro core, which is construction management software.
They use role-based certification to onboard the different roles that work with their software and provide a badge to people who complete it. So for them. It’s actually a way for them to monitor customer onboarding at a higher level of scale than just having their CSMs check in with customers continually.
And it’s something that people on their team can distribute to other roles in their company to help get them.
Dave Derington: [00:08:04] Plus you get a badge.
Adam Avramescu: [00:08:05] That’s pretty cool. A badge who doesn’t like badges. You’d like to share them on LinkedIn. Another example that I can think there is a HubSpot, which I sound like I’m being facetious, but I’m not people actually really do like sharing badges on LinkedIn.
but, HubSpot is another example that I can think of where. they’ve been in the business of creating a new category of software. but at the same time they’ve been creating a new category of professionals called inbound marketers because they’re inbound marketing software. So their certification programs are just as much about helping people learn the discipline of inbound marketing as they are about testing your skills with the software.
my old company Checkr. That’s another good example. we were not doing. Any sort of certification around using our software specifically. but we were building certification programs. They are still building certification programs to help HR and recruiting professionals to get the requisite knowledge and skills, to be able to do fair chance hiring and actually improve their hiring processes to make it more fair.
Dave Derington: [00:09:01] That’s really cool, because in some way it seems like we’re the first blush that I have with certification. You think about everything, but it really seems like what all of these examples typifies that we’re niching down into as bill shard might say, you’re niching down into a specific area in a specific cohort within your product.
an minister needs to know follow rooms more than an end user might.
Adam Avramescu: [00:09:25] Yeah, exactly. And for that reason, it’s not just roles. It’s also what format the certification will take. Is it just a course? Is it an exam? Is it proctored? And we’ll talk about some of those design considerations, but net.
One thing that I find is if you say the word certification to a room of 20 people, You’ll come out with about 56 different interpretations of what certification could mean. So it’s really important to start getting specific about what certification is for your company. And that starts with defining what the business goal that you want to drive.
Dave Derington: [00:09:56] Exactly. so let’s talk a little bit more about that. it’s helpful to ask yourself for the leader, who’s sponsoring this certification initiative. And then I’ll step aside for a moment to say that in almost every role I’ve had in customer education, someone has come up to me and said, And they’re a C-level or an executive to some, some level, some degree say where’s the certification program.
I really like a certification program. And I like to come back and say, what is the metric? What’s that mean metric for driving this? Why are we setting this up? What should we measure? What are the potential benefits behind that? Dive down into this a little bit more.
Adam Avramescu: [00:10:35] Yeah. Because ultimately, even if ultimately it’s going to have a lot of different, benefits to your business and a lot of different metrics that it will be associated with.
It’s also going to help you prioritize what you build and in what order. And some examples here could be, When you think about the impact, do you want to see fewer support tickets? Because users are more competent. Do you want to see more people whose job it is to use your software, getting certified, to become more skilled and become better advocates in the market.
Do you want to ensure a quicker and more sustained adoption of your product by getting customers certified? Do you want to see partners specifically? Not even customers, but partners getting onboard quickly to deliver services on your behalf. Each of those is going to inform what you create, how to develop it, how to deliver it and how to price it.
And you can see with some of the examples that we listed earlier, that everyone takes a different approach based on that high level business impact. But when you’re thinking about the way to structure this program, think about three different metrics. Think about reach revenue and impact. Typically it’s like the, do you want it done fast, cheaper?
Are those the three yeah. Fast, cheaper. it’s a triangle of project manager. Yeah. And right. you probably learned that if you take your PMP certification, but you can’t have all three of those at the same time. And the same thing is true about reach revenue and impact. you really have to decide where you’re going to focus.
Do you want the reach of getting, many people certified. Do you want to drive a lot of revenue from, offering your certification programs and prep courses? Or do you want to have an impact on some other external metric, like customer adoption or support tickets or things like that? that’s
Dave Derington: [00:12:10] cool.
And that also leads me to believe that once again, certification, that word is a sticky wicked, that it can mean a lot of things and what it’s really going to mean to us. And to what degree we’re implementing it. What’s the goal. What are you trying to reach? And I’ll do a bridge here. What’s your audience?
So let’s, let’s dive into that. Let’s talk a little bit more about the audience, right? Our second hypothesis would be this your certification program will be very different depending on who’s being certified. Just what I said, we have different cohorts, different personas, one of the most critical questions then for you to ask dear listener is who the heck are you certifying?
And if you have multiple audiences, which ones are you going to start with? Just to your point, Adam, a moment ago, like how do we prioritize this? So it’s helpful for you to get a clear understanding of who’s most important to certify customers, partners or internal glory while I can’t talk today or your internal,
Adam Avramescu: [00:13:13] we’re gonna do our vocal exercises next time
Dave Derington: [00:13:15] and blah, blah, blah,
Adam Avramescu: [00:13:17] seashells by the seashore.
Yeah.
Dave Derington: [00:13:19] Okay.
Adam Avramescu: [00:13:20] All right. So you’ve got customers, partners, internal teams, I think three really good audiences that tend to get mixed up. When people talk about what the point of a certification program is, customers and partners are pretty common use cases for customer education teams. See this all the time.
And once you create them, you can often use them to get internal team certified on your product as well or on whatever the skill is that you’re certifying on. But usually that’s a secondary audience. So a lot of the time we’ll see people build in the order of let’s build the most scalable version for customers.
And then let’s have something that’s a little more, intense for partners, but a little less scalable. And then let’s have, the most control over how our internal folks are also getting certified. But you want to pick which area you’re starting from.
Dave Derington: [00:14:06] Yeah, totally. And. One of the points we want to talk about here is priorities again.
So first you need to ask, what are those priorities? What do we need to deliver first? And I know at multiple different companies, and I think I’ve been in this recently, someone comes up to me and says, Hey, we want to cert certify our team. Okay. that can mean something very different from me having to go to a partner and I’m not going to spoil what we’re going to talk about really soon, but really engaging deeply with a system and a technology that is there to tool up certification.
But you can do something lighter weight internally when you have everybody at your disposal and they’re 10 feet away. L&D teams, sales, enablement teams create certification programs for employees that have. That learning content, but we usually go further in those scenarios, right?
Because we’re all here. We can do a demo and then we can challenge people on that. Or we can have somebody to do a pitch and improve their skills to us. And, that involves a lot of other things, but it’s less having a systemic approach that you would have through a system like criteria or something like that.
But it’s more, I have everything here. I have everybody here I can grade and coach and do what I need to do.
Adam Avramescu: [00:15:19] Yeah. So if you can get clarity, at least on what the goal is and what the audience is, then you can start to get a little more, visionary with your approach. so you could potentially have a walking deck where you go back and you say, Hey, here are three visions for what a certification program could look like.
in the next year, here are three things that I could build, which of these is most compelling based on the goal. And that will help you get a little more specific of the 56 things that certification could mean. which one are we really running towards to begin with? And exactly, even if your focus is on customers, get clarity on which customer segment is priority and which user personas are the priority.
Because for instance, if you want to take a cross section here of customer segment and user personas, an admin at a large enterprise, Is completely different from an end-user at a small business or a mid-market company admins at large enterprises need way deeper skills. They’re operating in a much more complex environment.
And typically they’re going to be able to devote more time to getting higher levels of certification and partners are the same way. You might have some like system integrator partners that you work with very closely and they’re highly managed and. They have strong relationships with you in terms of co-selling or co delivering services versus maybe having a really large distributed, reseller network, they are going to require a different level of intensity and hands-on certification.
Dave Derington: [00:16:44] That’s going to be a little, it’s going to take a little bit of a different form because as you’re onboarding and scaling those partners like an internal employee, you have access to them, but you’re really vested in making sure they’re successful. I’ve seen. Engagements go South really quick because Hey, I hired a partner.
I didn’t enable my partner screwed up and guess what? Now I’m getting complaints from my customer through the part around the partner saying, Hey, these folks let us down. It happens. And it happens pretty fast.
Adam Avramescu: [00:17:13] Yeah. And this is why it’s so critical in my opinion, to get clear on who you’re certifying and what certification is going to drive, because if the idea for instances.
we’re going to set up this very, low touch certification program, and we’re just going to run these multiple choice questions. And even if they’re proctored on a criteria or a platform like that, If the expectation is that by getting that certification, your partners are all going to be able to reliably bring in business and deliver 100% quality.
that might not be a deep enough certification to meet that need. You might need to go back and reset expectations with your business about, Hey, certification is a prerequisite for this. It’s a way for those partners to get a foot in the door and to maintain their partnership with you, they might need to have X, many people or X percentage of their delivery resources certified.
But certification itself is not going to solve that problem.
Dave Derington: [00:18:02] No.
Adam Avramescu: [00:18:03] So in doing that kind of brings us to our next topic, which is actually setting the stakes. So pretty quickly, you’re going to have to start asking the question. All right. if I knew who I’m certifying and what certification is supposed to do, how am I actually going to certify these people?
what do they need to learn? How do they need to prove that they have the skills to pay the bills? This is where you ask how rigorous your certification program needs to be. And is it high stakes or low stakes? So this is where we’re starting to get into a little bit more certification terminology.
You can look up high stakes versus low stakes certification to see a little bit more about this, but we’ll dive into it. So hypothesis number three here. This is one. I hear a lot, Dave. I don’t know if you hear those two, but
Dave Derington: [00:18:42] I hear those two.
Adam Avramescu: [00:18:43] Yeah. They’ll talk to people who run certification programs.
And they’ll say the only way to run a certification program is by offering proctored exams that are developed by a psychometrician. And if you haven’t heard the word psychometrician before, look it up. It’s a, basically a person who does assessments as their, as their career. So they know ,
Dave Derington: [00:19:03] wasn’t it, wasn’t this Tim, the enchanter from, Monte Python.
Adam Avramescu: [00:19:07] Yeah. It’s like that. That’s a cycle magician.
Dave Derington: [00:19:13] So the,
Adam Avramescu: [00:19:13] so yeah, Like having a psychometrician whose job is to develop assessments and to determine whether questions are fair and whether they actually reflect the job tasks. So they’ll go do a job task analysis for whoever was getting certified, and then have a certificate delivered at the end.
A lot of people will say that is the only way to run a certification program. If you’re not doing all of those things, don’t call it certification. It’s not certification. And
Dave Derington: [00:19:37] yeah.
Adam Avramescu: [00:19:38] Yeah. What do you think of that, Dave?
Dave Derington: [00:19:41] I think, and again, but part of what we’re doing here with the CELab podcast and in general is trying to reshape or mold a lot of these terminologies that we’ve been using in the industry and that industry, is that more?
I think John lay would put it as an extended enterprise. It’s a different modality of training. We’re here to help. Ramp up companies to get them producing educational content quick. So for us, that’s a traditional in it’s correctly. So what you’re saying is it can correctly be stated as the right way to do certification at high-stakes level, right?
Yeah.
Adam Avramescu: [00:20:15] But if we’re being prescriptive list, then yeah, that is the quote unquote right way to do it. But if we’re being disruptive asked about it and actually describing how people are doing certifications out in the world, doesn’t necessarily hold up. Does
Dave Derington: [00:20:26] it. No, not at all. And I think, okay, that’s the Jewish traditional way to do it?
If it were me, I know that has strong benefits. Actually. It’s a very rigorous, the it’s fair it’s routine. If somebody else can take care of the execution as well. so example for. For an example, when you use a proctored exam platform, like talking about criteria and it was one that popped up during my search,
Adam Avramescu: [00:20:51] there are others out there.
I know we keep saying criteria and there’s a bunch of other ones out there. There’s to be fair.
Dave Derington: [00:20:56] Yeah. To be fair. We’re not trying to single out. Let’s just say whatever platform you’re engaging with.
Adam Avramescu: [00:21:01] the list goes on.
Dave Derington: [00:21:04] you’re going to use that to deliver right. Deliver and for people to take them usually going to be working with those psychometricians those professionals, a project manager to develop an exam that it’s rigorous it’s going into testing the right things.
and that you’re going to have exam security. This is super important. And I’ll let you get into some of the reasons why, but that’s not quite intuitive for those of us that have jumped into this. Field this customer education. And some of you may be a customer success professional. That’s moved into this.
You’re not quite aware of all these things and where we’re going with that. Adam, tell me a little bit more about what you’ve learned on us. Yeah,
Adam Avramescu: [00:21:42] with exam security, a lot of people get hung up here because people start to get really paranoid. Once you have a certification. Assessment out there that assessment is reliable and valid, right?
And one of the ways to kind of tank the reliability and validity of the certification that you’re offering is by having the questions leak out onto the internet. So a lot of the platforms you’ll see will offer some sort of proctoring. Either onsite at a testing center or online via webcam. And some of them do all sorts of like knuckle detection or eyeball detection or things like that to, to see whether you’re a, looking at someone else, who’s holding an answer key up, behind the screen or something like that.
They can get pretty. Yeah, pretty hefty. can be a fig leaf too, if we’re being honest, there are ways for people to cheat that system. There are ways for people to still leak questions onto the internet, but you do want to be as rigorous as possible in terms of making sure that if you’re offering some sort of exam people, aren’t just leaking those questions out.
but that’s not all when it comes to developing an exam that’s rigorous or developing one, that’s sound from a legal perspective, because when you think about where a lot of this comes from a lot of the received wisdom around certification in our industry comes from one of two places. It either comes from the world of professional accreditation.
So like the PMP, which is again saying I can go out in the industry and be a project manager or from these highly technical software certifications. where again, it’s almost like a license to use that software. So right by default, they’re starting from a very heavyweight, very secure place. but a lot of the legal risks around certification, part of what I’ve learned is it doesn’t necessarily come from exam security.
It actually comes from the fairness of the question. So let me get into that for a moment. Cool. If you’re getting a professional certification to become a project manager, you’re taking the PMP. There’s an entire organization called PMI project management Institute. I believe whose job it is to educate test and credential project managers.
And so it’s essential for them to know that anyone taking their certification and going out into the market to say, they’re certified really has proven mastery. They really can do what they said they could do on the task, because if the test isn’t measuring anything, why did you take the test? Exactly.
So they want to make sure that the questions are written as fairly as possible because the legal risks for them. Is that an, if you work at PMI and this is wrong, please just write me. I would love to hear that I’m wrong. But unfortunately, the reality of the world is that the legal risk is that if the questions are biased against people in protected classes, for instance, PMI has to be able to say, Hey, we interviewed people who are actually project managers.
We know what their job task analysis is. We wrote the questions in a way that is sound and fair for anyone who aspires to be a project manager. And so they have to do that for legal reasons to really make sure that again, that test is accessible and reliable to the people who are trying to pass it.
And if they don’t do that, then they might get sued by people who fail the exam or who can’t get jobs in that field.
Dave Derington: [00:24:48] And that’s something that I think coming into this, and I know personally coming into this. I hadn’t even given thought to, wow, there’s legal implications for me saying there’s certification.
That’s why I usually throw up a red flag to anybody that mentions the word because there’s financial implications. There’s legal implications to you using a word. And it’s super important for you to know.
Adam Avramescu: [00:25:11] Yeah. But it’s not like the word itself is a curse, right? It’s not like you moderate and then Beetlejuice pops out of the.
under the floorboards, there’s something
Dave Derington: [00:25:18] like three times.
Adam Avramescu: [00:25:20] Now there are a lot of people out there who are using the word certification, but they’re setting the correct expectations with their customer about what it is that they’re actually receiving when they get certified.
So to speak.
Dave Derington: [00:25:30] something I’d like to talk about. And I’ve got a couple of little sidebar stories here, but I’ll try to stay on point if your certification is high stakes. And there is a risk of people not getting jobs because they’re not certified. Then everything then doing everything we talked about earlier is extremely important.
And it could also be a good idea to go with high stakes cert, where there’s a meaningful risk, And the risk of using your product in properly. One of the examples that I can get from practical, personal experience. I used to work at a pharmaceutical plant and I was a chemist in a laboratory.
However, when that plant would go on strike, they called upon all of us to do the tasks that those individuals would be doing. One of them was a forklift driver, which I got assigned to. So believe it or not, I had been a certified forklift driver of all things, but that was super hard, but it’s super hard.
Yeah, that’s a great sort of, and it’s super hard. I can’t tell you enough that machine, the machinery there is very dangerous. It’s dangerous because it’s top heavy. It’s imbalanced. It’s designed to do a certain job. And just prior to taking this training, there was an incident in our facility where a gentleman was.
For whatever reason, riding on a load on the blades and it slipped and fell and it crushed him, he died. and that, everybody that had taken that sort of, taking that training should have known better. And I think it was a case where somebody didn’t where they needed a refresher. So these kinds of things are super important to know.
There’s a lot of risks. I could flip it over and I could. Di, maybe you’re operating a train and you’re a trained engineer. You need to go through a lot more. So certification is very meaningful. There is a, you’re doing bodily harm here for us. That kind of criticality translates to. Maybe an administrator or a developer, and you’re working on a detailed configuration of an application then if you don’t get it right, guess what?
Your records to Salesforce integration through your product might not go over correctly. And then your business is affected. That’s not bodily harm, but that could be, you could lose your job over something like that. If something really, or I
Adam Avramescu: [00:27:35] feel like Workday is a good example of that, right? You have to be Workday certified to get.
A certain level of access to that platform because you’re dealing with people’s personally identical identifiable
Dave Derington: [00:27:44] information. Absolutely. And we talk about those because these are some lessons we’ve learned that there’s risks, there’s inherent risk in certification, and we want you, as you’re following this journey with us to step along and be cognizant of those risks as you’re going in.
risk is there. We want to help people learn. We’re trying to keep them from. Losing their jobs. We want to help them get good jobs. We want to be fair. There’s all kinds of things into that.
Adam Avramescu: [00:28:10] Yeah. and to the point of getting jobs, like part of the question that you have to ask is this a job market that already exists?
And we have to help gatekeep that job market to make sure that the people who are doing it are doing correctly, or is this an area where we’re trying to create demand and create a job market, like the HubSpot example. And that’s going to have very different implications again for how you set up your program.
But I think it’s not enough to say. if the CEO comes and says, Hey, we need a certification program because we want, there to be a huge network of professionals using product X, but product X doesn’t have a huge network of people even using it semi competently today, while a certification program isn’t necessarily going to fix that, you might want to start with some more traditional customer education modalities.
Dave Derington: [00:28:53] Exactly. so you’re saying there’s different levels or there’s there’s different weighting. We can apply to the term. So you can have really low stakes, lightweight cert all the way up to that, proctored exam, where you’re going to go to a facility and they’re going to test you and you’re going to do an oral review and all that kind of,
Adam Avramescu: [00:29:11] yeah.
I say it’s like dating a lot of the time. I say this in my book. This is my certification section in the book is it’s like a. There’s a difference between in a relationship when you’re just like talking to someone, quote unquote. and then, when you’re actually, putting a ring on it and it’s true in certification too, right?
it’s not that those are all, that there’s one right way to do it, or one right. Level of commitment, but you have to find the right level of commitment, relative to the relationship that you want to have, or the certification that you want to have. So you described it as an onion. I describe it as a relationship.
Dave Derington: [00:29:44] I like the relationship better get away from the Shrek motif.
Adam Avramescu: [00:29:48] So to that point though, like when we’re thinking about what certification means in the industry, it has expanded as a term. if we’re being descriptive, not prescriptive about it, it really spans everything from these very formal professional accreditation.
So I would describe that as like marriage to the lighter weight onboarding tools, which would be more like talking to or dating. and sometimes companies don’t offer Proctor tests. They do, what’s called an assessment based certificate or ABCs, which that’s usually a sort of a certificate that gets awarded for attending a class and passing an unproctored test at the end.
And those can still be called certifications just depending on what your risk tolerance is and how you’re willing to market a program like that. Others will have different tiers in their certification program where the lower end is something like micro badges that you earn. So a Salesforce Trailhead, this is a really good example of that.
You go through a course, you get a micro badge, but you’re not getting like a full certification, but then on the higher end yeah. Is something like a formal certification program, the people who are really, professionals or partners in this field, they’re getting that. Yeah. The point is don’t assume that you need to do the most rigorous program from day one.
If there’s not a need for you to do it. And I can actually recommend, another podcast called leading learning, which is basically for people who are in the business of lifelong learning. So a lot of professional associations and a lot of the content that they have is, really, akin to the content that we have as well.
Even though they’re talking about it, not from a customer education perspective, they’re talking about it from a learning business perspective. I would highly recommend that anyone who listens to our podcast also, Pick a few episodes out of that one, if, especially if you’re thinking about certifications.
Dave Derington: [00:31:30] Yeah. If you’re going to look that up, it’s leading learning.com. Pretty easy.
Adam Avramescu: [00:31:34] Yeah. Podcasts. Hey, and while you’re doing that, maybe leave us a five-star review on, on Apple podcasts.
Dave Derington: [00:31:41] Yeah, that’d be wonderful if you’ve been listening and we see a lot of you listening on a regular basis each every other week that we launch, go into the Apple store or go into Spotify, go into wherever you catch your pods and, give us some love, help share this podcast with everybody, So
Adam Avramescu: [00:31:56] that sounded like we were ending the podcast, but we actually aren’t. We have more,
Dave Derington: [00:31:59] not really. We’re going to do a lightning round. What do you feel about that?
Adam Avramescu: [00:32:03] Let’s do a lightning round,
Dave Derington: [00:32:04] do a running around. Okay. Let’s say Adam, you’re writing questions for your certification. Now we know.
we know this is the hardest thing quite often to do, because you’re going to have a group of SMEEs subject matter. Can’t talk, we’re going to have a group of SMEEs subject matter experts, your sages from, sitting down writing questions, thinking about this, and just because they’re skilled in their jobs doesn’t mean they’re good at writing fair, challenging questions for your subjects.
So let’s get into that. Let’s peel that apart.
Adam Avramescu: [00:32:41] Yeah, people write all sorts of crazy questions. And before we dive into the lightning round, let’s break down the different parts of a question. So we’ll do anatomy of a question for a moment. So in most questions, you’ll have, first of all the STEM, which is the prompt of the question and, I don’t want to take the wind out of my sails when I do the next instructional design one Oh one and we’ll do Bloom’s taxonomy, but.
In the STEM, you can basically be testing different things. You can be testing basic knowledge. So a, which of the following is a feature available in product X, or you can be doing something that is a little more sophisticated, like a more complex scenario. So then you have your answer options and you have your correct answer, but the incorrect answers are called distractors.
So if you hear us say distractor, we’re talking about the answers that are not correct, and that distract you from the correct answer. Each of these combines to make a great question, but we just want it to cover them before diving into the lightning round. So you will be familiar with the terms as we, as we move quickly.
Dave Derington: [00:33:37] Cool. Where are we ready?
Adam Avramescu: [00:33:39] Let’s go.
Dave Derington: [00:33:40] Let’s go. Okay. Let’s list out some good tips or writing assessment questions in a quick back and forth lighting round. So let’s go at them.
Adam Avramescu: [00:33:48] Ready? Okay. Cool. All right. First of all, make sure every option is valid. So that means even if you’re writing distractors. Don’t write joke answers.
So you can’t have D Bart Simpson. Okay,
Dave Derington: [00:33:59] cool. Don’t write distractors that are as valid as the correct answer. That’s just confusing.
Adam Avramescu: [00:34:06] Yeah. Good one. Okay. Write complete sentences in your STEM. Don’t just say you should colon and then have four answers because there’s no context for that. if you’re just listening.
Dave Derington: [00:34:16] that’s a good one. Also don’t make the correct answer noticeably longer. Then the distractors, this is actually a harder than it looks. It’s a dead giveaway because really good well-trained test takers, look at the lengths and a good psychometrician is going to say you should keep them all about the same
Adam Avramescu: [00:34:36] length.
That’s true. Okay. Ask one question at a time. So don’t write, what’s called a double-barreled question in the STEM where you’re asking multiple questions at once, because otherwise people are not going to know what the correct answer is.
Dave Derington: [00:34:49] Excellent. And stick to one item per answer, especially if you are including ranges, like option a is one to two, B is two to three, blah, blah, blah.
Those
Adam Avramescu: [00:35:01] overlap. Yeah. So you’re not going to know which one to answer. Okay, cool. Avoid writing question stems with negatives in them. So which of the following is not true and especially avoid answers with double negatives. So if you’re saying which of the following is not true and then option B is, do not do this.
That’s super confusing.
Dave Derington: [00:35:20] Yeah. And similar, confusing, and, or maybe easy to detect the right answer, avoid using absolutes. Such as always, never et cetera in distractors. like I said, they’re usually a dead giveaway that they’re wrong.
Adam Avramescu: [00:35:37] All right. Last one of the lightning round, if you’re using multiple select questions, so ones where there’s more than one, correct answer, indicate that in the STEM people don’t think to select more than one answer unless you tell them to.
So say, which two of the following options are. Correct. And if you want more great advice like this, Peter man, Jack, I hope I’m pronouncing his last name correctly has written about these for certification magazine and for CEDMA. and so you can look up tips for writing great certification questions, what we can link it as well.
and you can see some of those great tips as well as,
Dave Derington: [00:36:09] others. that’s fabulous. And I wanted to add one more note as I was looking through all of these, or we’re thinking through all of these Adam something. You particularly again, if you’re coming in from an area that wasn’t learning and development or education, I would recommend strongly.
Yeah. That you should start taking these question, building tips. Into consideration now, because as you go and you expand your program, if you’re building good questions from a sets from the get-go guess what, when you go to do certification style questions, that experience is going to translate. It doesn’t mean you have to be perfect.
You should aspire to really good questions.
Adam Avramescu: [00:36:47] Yeah. Yeah. I agree and start setting expectations with your business too. About how long good questions take to, right? Because if you go through this with a 10 question exam, for instance, that’s just part of your onboarding. And the people who are writing the questions, the subject matter experts, see how long it takes just to write 10 really good valid questions.
You’re starting to build awareness around what it’s going to take. If you’re building a, an 80 question certification.
Dave Derington: [00:37:10] Yeah. it will take a long time.
Adam Avramescu: [00:37:12] Yeah, that was a good lightning round. Dave
Dave Derington: [00:37:14] I’m exhausted. That was fun. yeah. Now that you’re exhausted, I’m exhausted. Let’s let’s continue this conversation, with our next podcast.
There’s a lot more to talk about as we know, when it comes to certification and we’re going to have with our next podcast. an interview with somebody who has been a practitioner, who’s been through all these things just recently, but for now, if you want to learn more, of course, we have our podcast website@customer.education.
That’s just customer.education. You got a fancy domain and you can find all of our content there and links to all those shows. Now on Twitter, I am @davederington
Adam Avramescu: [00:37:53] Derington. special. Thanks to Alan Koda for our theme music. And if this helped you out, you can. I’ll pass out by subscribing your pod, catcher of choice, Apple podcast, Stitcher, or whatever it is, or leaving us a review, please leave us reviews.
It helps us a ton. This is a labor of love. So please show us some love if you can. those two things really help expose our podcast to other people.
Dave Derington: [00:38:17] All right. And that brings us to the end of another great podcast and to our audience. Thanks for joining us. Go out, educate experiment, and find your people.
Adam Avramescu: [00:38:28] Thanks for listening.