Understanding employee engagement is critical to any organization. Listening to your employees is key to this understanding, but knowing what you need to ask them can be a challenge.
In this podcast episode, engagement experts David Long and Charles Rogel discuss what to ask in your next employee engagement survey.
Transcription
[00:00:16] Charles: Well, hello and welcome to our presentation today on “What to Ask on your Employee Engagement Survey.” My name is Charles Rogel. I’m the Vice President of Consulting Services here at DecisionWise and your host. Our presenter is Dave Long, Chief Operating Officer here at DecisionWise.
[00:00:32] Dave: Hello everybody.
[00:00:33] Charles: This webinar, along with all of our webinars, can be viewed on demand on our website or our YouTube channel. We won’t be sharing the slides from today’s presentation, but you can access the content from the recording. We do want to welcome your questions and your comments. Just use the chat feature to post those questions, and we will try to respond to those during and also after the session. With that Dave, let’s get started.
[00:00:59] Dave: Okay. Thank you for joining everybody. What we are here to talk about and hopefully what you’ve joined us for is to talk about what to ask in an employee engagement survey. And DecisionWise has quite a bit of experience with this. We have been around for 25 years. Started, I think, more in the 360 arena. Do a lot of 360 degree feedback. Even today, a lot of our business is one-on-one 360 degree feedback, coaching, that sort of thing. But one of the things that we’ve been doing for the last 20 years or so is to perform employee engagement surveys.
[00:01:29] And we’ve had a lot of data over the years that we’ve used and collected. We’ve tried this many different ways and through many different methods but also asked many questions over the years on engagement surveys and have kind of come to, through our research and through our analysis of the data have come to what we think is a very solid methodology for measuring employee engagement and everything that is around really the employee experience and that’s what we’re here to talk about today.
[00:01:59] I’m glad Charles is here with us. His team is very much on the front lines of designing surveys and designing survey items and really helping organizations make the most of their survey experience. So we’re gonna love his expertise on this.
[00:02:12] The agenda for today is gonna, be we’ll talk first and just kind of go high level, how to approach your engagement survey. Many of you may have already run engagement or org. surveys in your organizations. We would like to talk a little bit about how we think about it, the questions that we might ask as we’re moving into the survey, and these are really good questions to ask before you start saying, “Okay, let’s design a survey.”
[00:02:36] We wanna ask questions that will help prompt how we’re gonna do that. So we’ll talk about that here in a second. We’ll talk about key measures to include engagement surveys, some of the major measures that you might include, but also we’ll talk about some of the ancillary metrics that we’ve been including in engagement surveys over the last few years.
[00:02:53] In fact, part of that discussion will be; what are some of the trends that we’ve seen in employee experience and in these surveys and as we worked with clients to kind of tell you kinda what are other organizations interested in, or what have they been interested over the last few years, especially as we hit the pandemic in 2020. A lot of that shifted considerably. And so today some of the things that we do with our engagement surveys are a little bit different than what we’ve done in the past. So we hope to be able to give you some of that information today.
[00:03:20] We’ll talk about what’s the difference between an engagement survey and a satisfaction survey. As I know those words are both still used, I think on balance, most are now using the word engagement when referring to their organizational survey. But satisfaction is still used commonly and it’s still something that we believe in and something that we try to measure.
[00:03:40] Engagement, anchor questions we’ll talk about the direct measure of engagement, how to do that, how to determine engagement drivers and how to include items on your survey that would be typical drivers of engagement, so that you cover all of those bases. We’ll talk about a model for that. What do we look at when we look for satisfaction elements or measures of satisfaction?
[00:04:02] We’ll look at some other key metrics. And then we’re gonna look at common findings that we see in our engagement surveys, at least over the last few years, what we’ve been seeing in engagement surveys and what you could expect if you were to run a survey in your own organization.
[00:04:14] Again, my assumption is many of you have probably at some point or another, been involved in an engagement survey. So we won’t do too much definition of what is an engagement survey. We assume that most of you will know that information, and if you don’t, you’ll hopefully pick that up as we go through this.
[00:04:30] Okay. Talking about beginning with the end in mind, what are some of the questions you might ask yourself up front before you even start to design a survey? One of the key questions that you might ask yourself is, what do you plan to do with the information? What I really mean by that, who do you plan to have see the data? Will this be something that the HR team is mostly using? Is this something that you want executives to see? Is this information that you wanna share organization wide or you just want managers working in this information? So it’s an important question to ask yourself before you choose the questions you’re going to ask, you want to ask yourself, who’s gonna be seeing the data.
[00:05:09] Are there action planning expectations around the data? And who would be expected to do that? Are we expecting action planning to happen kind of at the HR structural level? Do we want executives to be involved in that and do we want managers to be involved with that?
[00:05:24] Who’s gonna be doing action planning on the survey will largely determine what you ask on the survey. If you’re gonna be having managers do significant action planning on the survey, we wanna make sure that we have questions on the survey that are directed toward managers or that they’re questions at least that managers can do something about.
[00:05:43] So, knowing your audience for the data is as important as anything as you’re kind of determining what are you gonna ask on the survey? Maybe you do not want to act upon the things that maybe you’re curious about, but we’re not ready to do anything about and if that’s the case, and we’ll talk more about what to do with those kinds of items that you think, Well, I’m kind of curious about this, but I’m not sure if we can really actually do anything about it. We’ll talk more about that as we get into this present.
[00:06:08] Are there any key metrics, key measures that you’re wanting to obtain? And this goes beyond just, we wanna understand the level of engagement. Do we wanna understand organizational communication or likelihood of attrition? Or do we wanna understand diversity, equity, and inclusion to a greater degree in our organization? And all those things are possible with an engagement survey, but you kind of want to have an understanding of what are the things that you would like to know from the survey before you even start.
[00:06:34] And the last thing that you would wanna ask yourself is, what sort of benchmark comparisons am I wanting to see? Have I done a previous survey? Do I wanna have significant comparisons between this year’s survey and the previous survey? That would determine in large part what you’re gonna ask on this year’s survey.
[00:06:49] Or am I looking more for external benchmarks? Am I looking for comparisons within the industry or companies my size or just want to see a global comparison between my organization and all the other organizations out there. Or, is it more important that I just get really specific with my own organization and just have a very customized instrument that I can’t really benchmark outside of my organization, but we’ll have internal comparisons, one department against another or one team against another?
[00:07:18] So, another question you want to ask yourself before you start the process is, are benchmarks important to me and especially are external benchmarks important to me because that will enlarge part determine how you design your survey.
[00:07:30] Charles: Let me qualify this too by saying what we’ve prepared today is geared towards this annual employee survey where we’re collecting kind of a larger amount of data to work with. And so we have kind of the ability, or at least flexibility to ask on several different topics and to really analyze the results and determine themes. This will apply to doing, pulse surveys, other types of surveys in your organization. But for the most part, best practice is to conduct a yearly, what we’d call maybe, an anchor survey to really give you a broad understanding of what’s going on in the organization and then you can focus specific initiatives around pulse surveys and other things ongoing.
[00:08:08] Dave: And that’s a really good point, Charles, cuz another question you might ask yourself is, how frequently do you want to ask these questions? Because if you’re wanting to do more of quarterly pulse or semi-annual pulse or something like that, you might think about trimming the survey a little bit so it’s not as long of a survey and maybe just trying to collect that information over time as opposed to trying to ask it all in one survey. Most organizations that we’re working with are still kind of on more of an annual model and well, they’ll do an annual anchor survey as, Charles referred to it, and if they’re doing a pulse or they’re doing several pulses, it’ll be. A shorter subset of questions that they’re asking on a quarterly basis just to, and usually on things that they’re working on, plus a quick measure of employee morale or employee engagement within that pulse. So they get kind of an ongoing understanding of where people sit in the organization.
[00:09:00] I just wanted to talk about, you know, since we’re sort of in a different world now, and certainly DecisionWise has experienced a lot of change over the last few years as we’ve worked with clients, that the concerns have evolved over time and in 2020, as you can imagine, we had quite a rude awakening in that every single organization that we wanted to work with, after about, I don’t know, March 14th, 2020, started to want to understand more about safety in their organizations, usually a topic that had been relegated mostly to manufacturing organizations or public utilities became at the forefront of almost every organization’s mind. And, we started to need to have items that would address safety in a different way than we’ve addressed them in the past, which would be more of a health safety. “My personal health is safe,” sort of questions in the survey. So we started addressing more of that in 2020, and that became the forefront of everybody’s mind. Charles, did you have something to add to that?
[00:10:02] Charles: I think you’re right. It was funny, you know, maybe 10-20% of our clients were asking questions about safety, and all of a sudden it was like 90% or more.
[00:10:09] Dave: Yeah. I mean, it went very quickly in the direction of safety. And the other thing that happened in the summer of 2020 is that the world became more, if they weren’t already aware, they became far more aware and far more concerned about diversity, equity, and inclusion issues. Especially probably around June of 2020. We had always, as an organization, we’ve always had measures for diversity, equity, and inclusion that we have encouraged. Most of the organizations that have run surveys with us, we’ve always encouraged them to include those within their survey and most organizations have at least on an implicit basis, included diversity, equity and inclusion questions in their surveys.
[00:10:50] What happened in 2020 is that organizations wanted to become more explicit about specifically calling out initiatives and/or commitment to diversity, equity, and inclusion within their own organizations in their survey. And we’ve gone, it’s kind of a similar thing, Charles, wouldn’t you say, about 20% were doing it before and we’re probably at 80 or 90% now of including diversity, equity, and inclusion in surveys, at least here in the United States.
[00:11:16] Charles: Yep. That’s right.
[00:11:17] Dave: Okay. Communication became a priority because people were working remotely, people were trying to understand the level of communication people were receiving.
[00:11:26] Interestingly enough, in 2020, scores around communication really spiked. The deliberate efforts that were happening in 2020 to communicate with employees really paid off. And most employees felt more communicated to in the middle of the pandemic than they ever had previously. But along with that, because people were working from home, they also wanted to know, you know, how do we make sure people are productive or feel productive or have the ability to be productive and also collaborative.
[00:11:52] As we kind of got into more of the pandemic and people started to get used to it, we started having different concerns. Certainly people were still concerned with safety. There’s still concern with DEI that’s still at the forefront of everybody’s mind, I think. But we also had things that started to come up, like work from home. What are we doing about this long term? Is this gonna come to an end when the vaccine is out? Or is this something that we want to kind of make part of our process going forward? What do we do about hybrid work arrangements? Et cetera.
[00:12:24] Workload became a huge issue. We kind of came out of this period of almost a mini recession. I mean, it was a huge recession, but for a very short amount of time. And then all of a sudden people became very busy again, and many organizations had and so they didn’t have as many people to do the same amount of work on the other end, and retention became a big issue because many people started to leave their organizations. We called it the Great Resignation and it is something that in large part, is still going on here in 2022. This idea of how do we retain our employees.
[00:12:56] DEI is still front of mind. And then we started to have more and more organizations interested in culture and in many ways culture in the way that it connects to DEI. And so in 2020, we have DEI and culture sort of in the same category. And that’s kind of like many… I’ve heard many people start referring this to as diversity, equity, inclusion and belonging. And belonging is kind of the cultural element that we’ve added to the DEI discussion, which I think is really useful, when we’re looking at our engagement survey and looking at what are the things that matter to us the subject of belonging or as we call it, a DecisionWise connection is a really important subject to understand if you’re running an engagement survey, it’s almost like if I understand the degree to which someone feels like they belong in their organization. I can understand a lot about their experience just by understanding that they belong or they don’t belong there.
[00:13:53] Flexibility a big issue. Again, it’s kind of the hybrid work arrangements that are still something that people are wanting to ask about on the surveys. Again, trying to understand retention through the survey. What are the key parts of our value proposition that can help attract new talent? We can understand part of that from the survey as well.
[00:14:11]And then still workload is a concern here in 2022. We would even throw in there the word stress. It’s not just workload, it’s people feeling stressed for other things, not just related to the amount of work that they’re expected to do.
[00:14:23] So just to kinda give you an overview we will talk about each one of these bullet points as we go through the presentation. Except for the first one. I’m just gonna tell you right off the bat, 40 to 50 questions is a typical anchor survey. Questions that we often get about this is, “well, can we ask fewer questions than that?” The answer of course is yes. You can ask fewer questions than that. Some people will ask, “Can we ask more questions than that?” And the answer is, of course, yes, but you wanna be careful with not creating survey fatigue by going to 60, 70, 80 questions. We start to get people going through that survey and wanting to sign out of it, and some of them do. They quit the survey before they even click submit. So we want to keep it between 40 and 50, which we think is the sweet spot, that allows people to feel like they can finish the survey within 10 minutes. And also where we can get enough information to not only understand the level of engagement in the organization, but also all of the factors around engagement that are causing it. And we’ll talk more about that because most surveys will include, not only a direct measure of engagement, but also engagement drivers and then other ancillary satisfaction elements that relate to the employee experience.
[00:15:31] We also design our surveys around a five point agreement scale. We’ll talk about that here in a second. We get a lot of people that say, “Well, why do we do a five point agreement scale? Why do we use an agreement scale rather than varying the scales throughout?” We’ll talk about that here in a minute. We’ll talk about frames of reference or dimensions that you can include in your survey, different aspects of the employee experience that we would ask about and kind of group our questions that way.
[00:15:56] The other thing that we’ll talk about toward the end of this conversation will be what about if we want people to be able to kind of free form some open-ended comments in the survey? How do we do that? And we’ll be able to address that as well as we get into this presentation.
[00:16:11] First, just to address the scale, we use a five point agreement. If you ask me kind of the main reason why we use a five point agreement scale, the main reason would be because most every other survey provider out there uses the same five point agreement scale. And the second reason would be we’ve been using a five point agreement scale for 20 years, and all of our benchmarks are based upon a five point agreement scale. And so we really do hold to this very strongly because we want the best comparisons we can possibly get from the engagement survey. A really good way to get that is by using a five point agreement scale, so I can compare now against other organizations in my industry, other organizations of similar revenue, similar size, similar geography.
[00:16:59] We are all based on the same five point agreement scale, and that’s why we use it. We give people the option in our survey to say, don’t know, not applicable. That way if we have newer employees that are taking the survey and they don’t feel like they have enough information to answer questions, they can say, “don’t know, not applicable,” and that really keeps that neutral area clean and what we’re looking for with the neutrals are people that have had a combination of positive and negative experiences which is why we really like to have a neutral option in our survey. And we don’t go with a four or a six point agreement scale. We love to have the neutral because we want to see the neutral, which we’ve found are the people that are most likely to be impressed one way or the other, either to fall into the more unfavorable category or to, in the future, be more favorable. And so we think that neutral is a really important thing to see and understand from a survey. So that’s part of the reason why we include an odd numbered scale as opposed to an even numbered scale where there is no neutral response.
[00:17:52] You would keep a consistent scale throughout a consistent scale also allows people to get through a lot of questions without having to read and understand the scale every time. That’s a really important point as well. If you’re gonna ask 40 or 50 questions on a survey, if you change the scale on each question, then people have to now stop after each question and not just react to it. Now they have to understand the scale and then choose something within the scale. That’s why you don’t do kind of do that multiple choice idea of everything has a different sort of set of responses. Everything has the same responses. Then now I’m reading statements on a survey and I’m reacting to them as quickly as I can or as quickly as I want to, I should say, and what ends up happening is people are able to get through a survey like that in 10 minutes and sometimes less than that. And if you had only 20 questions, it would be, you know, seven or eight minutes that they would be able to get through it. Varying the scale also hurts your ability to analyze the data. I want to be able to compare the responses to one question about tools and resources, and to another question about training and I can’t do that if those questions are asked on different scales. I wanna be able to understand very clearly when I look at the data, these are the areas where I scored high versus these are the areas where I scored low. It would be impossible to do that if we vary the scale throughout the survey.
[00:19:06] Someone asked, “Can you not include neutral if you do not have new employees? We have a small issue with that.” Really the new employee thing was the number six here, “Don’t know, not applicable,” that we include on our scale neutral. I want to include, no matter what I, it’s a big preference for me to be able to have neutral and the main reason is I wanna understand. How much am I gonna be able to change the thoughts and opinions around this subject? And a good way to understand that is to look at the size of the neutral. So I personally try to include neutral, no matter what.
[00:19:37] Okay. Let’s talk about frames of reference. And now we’re talking about kind of, as we build the survey, what are the kind of major categories? And I’ve seen this done a few different ways, and we do it a few different ways here at DecisionWise. One is to say, here are the subjects. Or the themes that I want to understand from my survey. I wanna understand about communication, I want to understand about training, I wanna understand about career development. You can do that. And then I can say, Okay, I can fill in questions beneath that.
[00:20:07] Another way to do it, and a way that we really like to do it, is by using dimensions or frames of reference. So my job questions, related directly to my job or the work that I do every day. Questions related to the people that I work with most frequently are my team. Questions related to my direct manager or supervisor, and then questions related to the organization. Now, we also include if you had within your area, a next level supervisor that you wanted to include questions about, we do that sort of thing. Or if you wanted a separate section about senior leaders, we can do that. But essentially what we’re doing is we’re saying, looking at somebody in their job and we’re saying, Okay, what are the frames of reference? There’s one forever reference that I just look at the things that I do every day. That’s my job. And then the people that I see very often, my team, and then the person that I have to interact with as a superior, that’s my supervisor. And then how do I feel about the organization. Interestingly enough, It’s how I feel about my job and how I feel about the organization overall. That mostly impacts my ability to engage. I can like my supervisor and like my team and not be engaged. But even as I say that, we still know that team and supervisor relationships are extremely important and I would. If I don’t like my team, I don’t like my supervisor, I’m not gonna stay in that job for very long. But if I can like them very much and still hate my organization and might still be engaged, is sort of the point that I’m making there. So it’s important to gather information from all of these frames of reference and we like these as our major categories. Not only because we think that there are important categories to understand, but also because when we do factor analysis on our survey items, this is how the survey items group, it’s by job factors, it’s by team factors, supervisor factors, and organization factors. So we really like to organize it this way in the survey, mostly because our research has indicated that’s the way that we should organize it.
[00:22:03] Examples of, and I’m gonna try to sprinkle in examples of survey items that we might use as we go through this, but job would be things like, “I enjoy the day to day tasks I perform.” A team question or survey item might be something like, “do people on my team treat me with respect?” Supervisor: “I trust my supervisor organization, I understand the vision and goals of this organization.” So those are just kind of a taste of what you might include in those sections.
[00:22:29] The other thing that you wanna include is a direct measure of engagement. In order for us to have this conversation, I just wanted to really quickly talk about the difference between, if we’re gonna say this is an engagement survey, not a satisfaction survey, what do we mean by that?
[00:22:43] Well, satisfaction elements and a satisfaction survey would really focus on the contractual or transactional side of the employer/employee relationship. So I take a job and I expect by virtue of the fact that I’m taking a job, that certain things are going to be in place for me. An example of that is if I take a job, I expect that I’m going to be provided with a reasonably safe work environment. I expect that I’m gonna receive tools to do my job, then I’m gonna have basic training in order to do the function of my job. And all of these things are things that I would expect to be part of my job. And by the way, satisfaction, I expect that I’m gonna be paid at a certain level that’s commensurate with other people in the industry doing similar things that I do.
[00:23:30] Satisfaction items, kind of what sets them apart is I could be very happy with all of these satisfaction items and still not feel super motivated to do my job. If I show up every day and I need a computer to do my job, and I show up every day and I have a computer, I don’t think to myself, Wow, what a great thing that I have this computer to do my job isn’t this wonderful, and how motivated do I feel? But if I show up to that same job and I need a computer to my job and I don’t have a computer one day, it’s broken or it is gone, someone stole it from my office, whatever… I don’t know what happens with your computers. I usually have a computer, but if I didn’t have one, I would be very frustrated that I couldn’t do my work and therefore it would lead to dissatisfaction. But even because it can lead to a great deal of dissatisfaction, it doesn’t necessarily mean that the reverse is true, that I would all of a sudden be engaged just because I have it. So what we’re looking for is going from a transactional survey, which is, all those transactional issues are still foundational. It’s very hard to be dissatisfied and engaged at the same time, at least over a long period of time. So we want to keep satisfaction as sort of foundation and move into more of a transformational space where we have people that are engaged and we have the words, heart, spirits, hands and minds written underneath engagement, really because we want people, their full commitment to, to the work that they’re doing, to the organization, to the team that they’re working with, we’re looking for that higher level of commitment, discretionary effort, whatever you want to call it. That’s what we’re looking for. We’re looking to connect with people on a deeper level in order for them to bring more of themselves to their work on a daily basis.
[00:25:13] In order to do that, really what we look for are things beyond tools, resources, safety, et cetera, where we’re looking for things that might connect with them deeper. We’ve done some research around this in order to understand that we’ve had many items, hundreds of items that we’ve asked over the years on engagement surveys, 150 to 200 items that we have that are fully benchmarkable for most industries, in the DecisionWise benchmark. And so we had a lot of items and a lot of topics to choose from as we went through and analyzed the data, and really what we were able to come down to is generally speaking, and we’re talking about across many organizations, we found that there were factors that lead people to be engaged and what luck, those factors spell out MAGIC, and those of you have followed DecisionWise for a long time, would understand that DecisionWise is very interested in making engagement accessible and understandable to everybody in the organization, not just to the HR team. So we use a framework like MAGIC to try to help us understand what are the key factors that lead people to be committed at a deeper level.
[00:26:22] And as we build our survey, we not only look to build a construct in the survey that would measure, directly, engagement, we also look at what are the things that are causing engagement and they’re these things: meaning autonomy, growth, impact, and connection. And I’m gonna get to those items as we get more into how we’re building out the survey. But let’s first talk about how to measure, directly, engagement. And we do that with what we call engagement anchor questions.
[00:26:48] Charles: Let me answer this one question. I know there’s different ways people kind of define engagement and some of these topics that, you know, Microsoft recently started using the term “thrive” instead of engagement or going beyond engagement. And they did describe it basically as to be energized and empowered to do meaningful work. And so we would say, well, that kind of fits or is a different way of describing what we’re showing here because the energized piece we say is the engagement piece. When you’re energized to do your work. The empowered piece would fall under this autonomy concept where we see autonomy as a driver of engagement. When you feel empowered, you have autonomy, you tend to be more engaged. And then meaningful work, you know, ties into the “M” here for meaning, where if you feel like your work is meaningful, you’ll tend to be more enthused about doing your work.
[00:27:38] And another thing I’ll say is, engagement really drives the performance of someone and satisfaction’s more about retention, where if those things are working, you’ll tend to stay at an organization longer, but you might not be as motivated to, do your best.
[00:27:50] Dave: A lot of these things are different versions of the same book. You know, Thrive. Microsoft calls it Thrive. It’s talking about the same thing that we’re talking about when we talk about engagement.
[00:28:01] Charles: Right. Right.
[00:28:02] Dave: I think a lot of that is, well, let’s pretend like there’s something out there that’s something that’s greater than engagement. And there may be some level of emotional commitment that people have that is beyond just feeling engaged and happy to do their work. But we’re just really talking about different versions of the same book, I think, when we’re talking about the difference between thrive and engage, at least for my part.
[00:28:24] Charles: Employee experience is the next level, employee experience is kind of the entire experience that we’re measuring on the survey.
[00:28:30] Dave: Right.
[00:28:30] Charles: But we’re trying to see then what is resulting. Is the experience resulting in people being more engaged? So is it a good experience that kind of motivates you to be more engaged or allows you to be more engaged? So we would connect it that way.
[00:28:43] Dave: Right. Very good point. In fact, if you were really to classify our surveys accurately, you would really call the employee experience surveys as opposed to just engagement surveys.
[00:28:54] Charles: Yeah.
[00:28:54] Dave: Contained within the employee experience survey is more of an engagement survey, but within it are many factors that influence the employee experience, and that’s why we categorize between engagement and satisfaction.
[00:29:07] Satisfaction items are a huge part of the employee experience and we don’t want to demean them or say that they’re not important. They are important, but what we’re trying to do is we’re trying to understand what is leading to that higher level of commitment within our employees that causes them to bring more of themselves and to perform at higher levels.
[00:29:27] Okay. So let’s talk about how do we measure that? We do use engagement anchor questions, and most of our surveys include about five questions that are measuring engagement. And they’re designed to measure, and this is where I think people have a hard time as we’re working with them in building the survey, they’re designed to measure outcomes of engagement.
[00:29:49] And what I mean by that is they’re not designed to be directly actionable, meaning when we’re asking these questions, we’re asking for a general feeling of where they are mentally, emotionally, as opposed to asking them specifics about their work. So people were like, Well, those aren’t very actionable, and that’s true, but what we wanna do is we want to take action on the rest of the survey in order to influence their level of engagement.
[00:30:16] Our engagement anchor questions are designed to address different aspects of engagement as determined by research. So there’s research out there that tells us that different aspects of engagement are the absorption that you feel into your work. How absorbed do I feel on a day to day basis?
[00:30:33] How dedicated am I to the tasks making sure that I can get to the end and finish a task? How much vigor do I bring? I don’t love these words and we don’t use these words anymore in our surveys, but we ask questions that kind of get at these topics.
[00:30:46] Charles: Yeah, we try to translate that into human speak on the survey.
[00:30:50] Dave: Exactly. Absorption, dedication, and vigor. Those are the words, though, that were determined through the research in this that happened in the nineties around engagement. And we still use those same concepts today because we still think that they’re very accurate and very valid and frankly, our running surveys for the last 20 years have done nothing but confirm that those are important factors of engagement. But we try to understand those things by asking, as Charles put it, putting it in layman’s terms. Like for example, this is one of our major engagement anchor questions: “Most days I look forward to coming to work.” Now actually that’s evolved over time because during the pandemic, and even still today, much of the time, we say most days I look forward to starting my work or beginning the day.
[00:31:30] Charles: Yeah.
[00:31:31] Dave: Yeah, exactly. So, it used to be we’re all going to work, but now we sometimes stay at home and start our work.
[00:31:38] But there are other questions, “overall I love my job” or “time passes quickly while I’m at work.” Things that help us understand the level of engagement. And again, I’m not gonna be able to act on “most days. I look forward to going to work.” I can’t just go out and tell everybody, “Hey, you should look forward to going to work more. Please do that now, please.” It’s not gonna be very effective. And so what I have to do is I have to create an environment where people would now feel that I look forward to coming to work every day. And so we’re acting on the rest of the survey and that’s why we’re big on understanding what’s actually causing engagement because we can address those factors directly.
[00:32:16] The cool thing about using an engagement index is now we’re able to classify your organization. This is sort of the outcome of using engagement anchor questions. We can index employees in an organization into four different profiles. We’ve got your fully engaged employees, and they’re everything that you think that they are. They’re very motivated. They’re very excited to be working there. They often come in with fresh ideas and innovation. They’re putting in more effort than they really have to get their work done. They’re often working on proactive projects and things like that. We love having people that are fully engaged. In most organizations, that’s only about 30% of your population.
[00:32:56] Key contributors would be people that are very satisfied, maybe sometimes engaged, but not really that fully engaged profile that we talked about just a second ago. We would put them more in a category of being your strong and steady population. About half of most organizations are comprised of key contributors. That’s not a bad thing to have key contributors. It’s definitely a good thing on balance. Of course, we’re looking for more and more people in that dark green, fully engaged group, but it’s really good to have a foundation of strong key contributors in your organization.
[00:33:28] The reality is if you have everybody that’s fully engaged in your organization, that can become exhausting to manage. Just people… everybody’s excited to do their work. I know that sounds awful for me to say, but you do want some people that are strong and steady in addition to having people that are fully engaged and a good mix between those two is really what we look for in on-balance, of course we want more people in the fully engaged category.
[00:33:51] Opportunity Group is a very neutral group and this is one of the reasons in the question that was asked earlier, “do we really wanna include neutral?” Yes, we do. And one of the reasons why we want to include neutral is to understand how many people are in the engagement group that we call the Opportunity Group. This is a group that will move. So as we analyze our data, we see we have people in the opportunity group. This is a group that’s either gonna move up in their engagement, down in their engagement, or they’re gonna leave the organization between now and the next time that we survey. And largely what we do with the data from our engagement survey and how we handle ourselves over the next year and the kind of experience that we create for our employees will determine if this group is moving up, moving down, or moving out. So it’s important to understand what is our opportunity group, that neutral group, and also what are the things that are keeping them on the fence? Are there items in our survey that are leading this group to be more on the fence? And that’s again, what we’re looking for when we’re looking for drivers of engagement.
[00:34:46] Fully disengaged. This is about 5% of most organizations. That’s the average that we have in our database. It’s exactly what you think they are, which is their board, frustrated. We sometimes call this the “quit and stay population” in the organization and you’re unlikely to be able to change their opinion. They’re not gonna wake up one day, no matter what you do, they’re not gonna wake up one day and say, “you know, maybe I had it wrong. I really do love this organization.” It’s just unlikely that they’re ever gonna get to that.
[00:35:15] So once we understand the engagement index and once we understand where everybody in your organization sits, and again we’re talking about confidential surveys so even though we’re classifying employees as engaged or disengaged or opportunity or key contributor, we’re not identifying who is what. We’re just putting them into groups in order to understand further drivers of engagement. We’re trying to understand what is leading people to be fully engaged, what’s leading people to be on the fence, what’s leading people to be fully disengaged? And we can do that most effectively with engagement drivers. Because we’ve known this for a long time in the last decade, we’ve really tried to simplify this into a framework that helps us understand, what do we need to include in our survey in order to understand what would be causing people to be engaged?
[00:36:02] And so when we’re doing survey design, we lean on this model a little bit to say, All right, we need to understand the level of meaning, autonomy, growth, impact, and connection that people have in their jobs. So what do I mean by that? Let’s give you some examples.
[00:36:16] When we talk about meaning, what we’re saying is your work has purpose beyond the day-to-day tasks you perform. And the way that we would test for that in the survey would be to say, this is one of the questions, “my job provides me with a sense of meaning and purpose.” Very simple. It’s a question we’ve been asking for years and years and years, and often shows up as one of the key drivers of engagement for most organizations.
[00:36:40] Second: autonomy. What do we mean by that? This is the power to shape your work environment in ways that allow you to perform at your best. A sample question would be, simply, “I have the freedom to decide how best to do my job.” Again, what we’re looking for here is, do people feel trusted to do their work? Do they feel the trust of their manager by the organization? Do they feel like they’re being micromanaged? All those things are important so that we understand, Do people feel like they have enough autonomy to feel engaged? As they feel more trust and more trust is placed in them, they’re going to react to that positively and become more engaged.
[00:37:16] Next one is, do they feel like they’re making professional progress. “Being challenged and stretched in ways that result in personal and professional progress?” is the definition that we have for growth. Do they feel like they’re progressing? And this doesn’t just mean that they feel like they have promotion opportunities, although that can be a part of this. I’m not gonna downplay that. People definitely engage when they’re building toward a promotion and right after they have a promotion. But what do we do in the other times to make sure people feel like they’re getting enough challenge, enough stretching in their work in order to feel like they’re making progress? Are we helping them add to their skillset? Are we helping them add to their resume? Really, I mean, in broader terms, that’s what we’re looking for. But we’re really looking for people to feel like they’re stretched and we look for that by looking for growth And we simply would ask people on an engagement survey, “my work gives me opportunities to learn and grow.”
[00:38:06] Impact. Do people feel like in the day to day work they’re performing, that they’re seeing positive outcomes, that they see that the work that they’re doing is effective, that there’s positive results from their work? And that’s really how we define impact and we might ask an impact question, which is, “most days I feel like I’m making progress on important work projects or initiatives.”
[00:38:26] And then connection, a very critical part and often what is the biggest driver of engagement in organizations is connection. And it’s simply stated, “the sense of belonging to something beyond yourself.” If you feel like you belong in your organization, you’re much more likely to be engaged. And a sample question for that, is very simply, “I feel like I belong.” Shows up very often as one of our key drivers of engagement. What do I mean by it shows up often as a key driver of engagement? Well, we’ve done research on all of our database that helped us come up with this magic acronym, these things that we know are very common drivers of engagement. But it turns out each organization individually has a life of its own, has sort of its own personality, its own culture.
[00:39:10] And so when we go from one organization to the next before we survey, it’s hard to predict what is actually driving engagement in this organization. It’s usually some mix of these five items, but it may not be. It may be from outside of these five items. And there may be one of these items that is more of a driver of engagement than the others. So we do driver analysis with most of the data that we perform. And really what we’re doing is we’re taking the anchor questions as the dependent variable and setting everything else as an independent variable and trying to understand, okay, all these things that we’re asking about the employee experience, what are the things that are driving engagement in that organization?
[00:39:48] So while MAGIC elements provide a good framework, it’s not always gonna show up as the drivers of engagement in your organization. It’s important to include the MAGIC elements because it’s a pretty good guess as to what is driving engagement in your organization, but it’s not always gonna be that way.
[00:40:03] So we conduct a broader survey with lots of different elements in the employee experience, and we put those all into this driver analysis. And it may include things from outside the magic framework. So let’s look at what that looks like. This is what a driver analysis looks like. And this is not the input/output. This is just the output that we create on our end when we’re presenting this information. But we’re looking for, what are the highest impact drivers of engagement. This is actually a real engagement driver analysis that we ran on an organization where 83% of engagement in that organization was explainable by these four items. And so we can see, “I am proud to work at this organization,” “My job provides me with a sense of meaning and purpose.” Both of those are meaning. “I feel like I belong here” comes from connection. I feel challenged and stretched. That is a growth question. So it did turn out that we had three of our magic elements that were drivers of engagement in this organization. But again, we now can prioritize the magic framework and say, We know of the MAGIC acronym. “M”, “G”, and “C” are the most important things in our organization, and that’s where we can place our focus in order to drive engagement the most. So Charles, did you have anything to add?
[00:41:15] Charles: I think that’s spot on. Yeah. And we do notice that some of these, like you said, the MAGIC drivers differ by organizations. So in general, magic, shows up as some of the drivers, but it is unique to each company,
[00:41:28] Dave: Right. And that’s the important thing to remember when you’re designing your survey. Even though you may think, “Oh, well we have the magic framework, so we can make a shorter survey.” That’s just an engagement measure, plus meaning, autonomy, growth, impact, and connection. Yes, you can do that and you’d be able to understand engagement to a large degree in your organization, but you also want to see, “Okay, is there anything else that’s out there that would also be driving engagement or even getting in the way of satisfaction?” So we include satisfaction topics in surveys. So again, building the survey we’ve talked about when we create frames of reference and we start filming our frames of reference. And we do that by adding first anchor questions and then we add probable drivers of engagement to that and now we’re filling in the survey even further with satisfaction topics.
[00:42:15] So what are the topics that we would want to understand? Well, do people have the resources they need to do their job? A simple question like, “I have the tools and resources I need to do my job well” would be appropriate for that. Safety. Do people feel safe in the work environment? “I feel safe in my work environment” would be a very easy way to test that. Training. I have the training I need to do my job well. Very simple.
[00:42:37] Remember if you’re writing your own survey questions, you want to be very direct about the thing that you’re trying to measure and put it in the simplest terms possible. And we’re not trying to measure two things with one question. Like saying, for example, “I have the tools and training I need to do my job well,” would not be appropriate because people might put “neutral” and say, “well, I have the tools, but I don’t have the training.” So we want to make sure that we’re separating each concept in everything that we’re trying to measure and if we wanna understand training, don’t try to fool people into answering the training question. Just ask directly, “do they have the training they need to do their jobs?” Communication. “This organization communicates well about what is going on,” would be a question that we would ask very commonly about communication. Collaboration. This case we’re talking about cross-functional collaboration. So “we work effectively across departments and functions.” Stress is a big one that we’ve been asking about a lot more lately. It’s always been kind of a part of our survey, but it’s been more of an emphasis over the last three years. “The level of stress in my job is manageable.” Same with workload. “The amount of work I’m expected to do is reasonable.”
[00:43:43] And then pay and benefits. We’re gonna talk more about pay and benefits here in a minute, but if you did want to include a pay and benefit question that would fit into the category of satisfaction. Pay can motivate people as we’re building up to a raise, and they’ll feel a certain level of motivation after the raise. But in the 12 months in between that or the 10 months in between that, we need something else to get them more engaged. They reach a point of stasis with their pay very quickly after they receive a pay increase. I mean, you’ll get some boost in motivation directly following that, but then you know you’re gonna need something else to keep them engaged the rest of the year.
[00:44:20] So now we’ve filled in the survey with engagement anchor questions, with drivers, with satisfaction elements. Another element that we include in our surveys would be this open-ended comment question, meaning we’re asking an open-ended question to try to get people to react and really write as much as they can about their experience. So most engagement surveys provide this opportunity to their employees. We usually include one or two comment questions. If we include one comment question, it would be simply a neutrally worded item. Something that says, “tell me more about your experience working at this organization.” And we’re not assuming that it’s good or that it’s bad, and we’re just letting them determine how they feel about their own experience. This can be a good way to understand where the sentiment goes, and we certainly do measure sentiment in some of our analysis. More commonly, I would say we use two comment questions at the end of the survey. And they would be one positively framed question and one more negatively framed question to understand what do they like about their job and what do they not. The great thing about comments is that we have the ability using algorithms and computers and systems, we are able to categorize and analyze the responses. So we know we had 600 responses that all mentioned communication and 200 responses that mention the manager and 150 that mention payer benefits. So we can categorize it that way, but the real value in comment questions is to be able to read them and understand them on a one by one basis. Sample question would be, “what are the areas that need the most improvement in this organization?” That’s a negatively framed comment question and you let people react. That’s not specific, and sometimes people want to get very specific with this comment. Questions are the easiest way to create survey fatigue so we usually don’t like to go beyond two. If we do, we maybe will go to three or four, but really not anywhere beyond that because people get tired of responding to open-ended questions very quickly.
[00:46:17] We’re not getting specific with this, because we’re trying to understand generally, where does their energy go when they want to talk about the things that aren’t going well. And likewise, we wanna understand, where does their energy go when they’re talking about the things that are going well. And I know I’ve been talking very quickly, Charles, do you have anything to add to that?
[00:46:34] Charles: The comment questions also point to sometimes low scoring questions from the other questions on the survey, so they really kinda highlight or draw attention to some of the main topics you wanna focus on.
[00:46:45] Dave: Yeah, and I would say oftentimes, if you read through the comments that are categorized, let’s say communication is an issue in your organization, you can go to all the comments that mention communication and see are there any ideas that are out there that can prompt our action planning discussion and it can be useful from that standpoint.
[00:47:03] So other key metrics that we might include. I’m gonna go through this very quickly. Employee net promoter score is something that we really love in pulses and for them to be valuable in pulses, we would include this question also in a net promoter score question in an anchor survey as well. EMPS is a good general metric to measure employee morale or even it could serve as a proxy, a one question proxy for an engagement anchor question. In fact, you can use it as an engagement anchor question in your survey and it can be if you wanted to, if you have customer NPS at your organization, you can place this on the same scale, a 10 point scale if you would like to. And then one nice thing about NPS is that leaders and executives are familiar with the format and the NPS question can be used, as I said, as an anchor question. So it is very simple. “I would recommend this organization as a great place to work.” You can also put it in the framework of customer NPS and have it say something like, “on a scale of one to 10, how much would you recommend this organization is a great place to work?” Something to that effect. So it’s a great way to kind of get a one question measure. And again, this works really well in pulse surveys.
[00:48:07] We also want to understand how likely are people to leave the organization. So we include questions regarding the intent for employees to stay. So we create an index that looks a lot like the engagement index, using questions that are related to attrition and that would look something like, “I would choose to remain here even if the job was similar, paying benefits were available elsewhere,” and we would ask other questions related to that item and then we would create an index just like we did with the engagement index, into category of fully committed employees, content, undecided employees, and employees that are likely to leave. So attrition is another thing that you can really measure with your engagement survey.
[00:48:47] We did a webinar last month. You go to our YouTube channel to look this one up. It’s something that we get into great depth. How do we include diversity, equity, inclusion in our surveys? And we went through all these constructs on that. But this is a one that’s a very common add and a common analysis that we’ve been doing over the last three years and so something that you might consider including in your survey. And what we do is we divide this into kind of their own dimensions. Diversity, equity, inclusion; important topics related to that would be the level of voice that I feel like I have, the amount of growth opportunities I feel like I have. How much do I feel like I belong here? And how much is this organization committed to diversity, equity, and inclusion? And we look at these items and really the whole survey through demographic comparisons. So we can look at it by ethnicity or race, and we can look at it by gender. We could look at it, if you have the information, we can look at it by sexual orientation or many other ways that you would wanna look at it to make sure that we’re being inclusive of everybody. And the factors that we’ve really found are important to that are voice, growth, belonging, and organizational commitment. So diversity, equity and inclusion is something we definitely have been including.
[00:49:56] Just really quickly, what to expect. Well, you can expect that there will be some areas where you score pretty well. Questions related to the team, confidence in the future. A lot of organizations score pretty well on that. Questions related to your direct manager or your personal impact, How much do you feel like you’re impacting the success of the organization? And then most organizations score pretty well in the categories of tools and resources. If you’re running a survey for the first time and you’re thinking, what am I gonna have to do as a result of the survey, here’s a pretty good summary of where to expect to struggle. Where we would say, even without doing a survey, I would say you’re going to have a hard time with cross team collaboration, organizational communication, employee growth. You might also have issues with workload and stress right now because it’s more of a topical thing. It’s been more over the last three years where we’ve seen that in the lowest scoring items. There are probably gonna be some issues around senior leadership, especially in larger organizations where people are pretty far removed from senior leadership. They’re gonna think senior leaders don’t know what’s going on in the organization. So you can expect to have something like that. And then if you choose to include questions about compensation and benefits, you are a glutton for punishment because you’re also gonna get very low scores on that. I can tell you that right now if you include it. And in fact, that’s why we have this section about what not to ask. Don’t ask anything that you don’t have any intention of acting upon. We talked about that earlier, but I’d say it again, if you don’t have any ability or intention to act upon something, don’t put it in your survey and communicate, “Hey, this is something that we want your opinion on so we can do something about it.”
[00:51:27] Questions related, compensation and benefits. We would encourage you, and I encourage every client I worked with not to include this, some of them still do but the reality with compensation and benefit questions is they almost always come back at some of your lowest scoring items and they’re gonna show up in every report that anybody ever sees is one of your lowest scoring items and people are gonna say, well, you didn’t do anything with the survey if you don’t adjust compensation. So I would say leave it out and see bullet point 1 for why you would leave it out.
[00:51:55] And then questions that ask employees to evaluate themselves. If you’re asking them, how hard do you work or how much effort do you put in? Or how much do you care about clients? Those are all gonna come back at about 99% favorable. So anytime when you’re asking employees to evaluate themselves, that’s probably something to stay away from.
[00:52:13] All right. Sorry we’ve gone through pretty quickly here. But we’re at the end. Quick summary. As you build your engagement survey, you really wanna look at those major constructs that we’ve talked about, anchor questions, drivers of engagement. We want to include enough satisfaction elements to understand satisfaction in the organization. And it’s also great once you’ve created that core survey to say, What are the other metrics that we want? Do we want NPS? Do we wanna understand attrition? Do we wanna understand diversity, equity, and inclusion to a greater degree? And then you can build your survey from there. Try not to go more than 40 or 50 questions on your survey. We’re always here, by the way, as DecisionWise, this is what we do. We’d love to work with anybody in creating the survey and actually running the survey and getting the most outta that survey. But this is the methodology we use and that we’ve really espoused over the years.
[00:53:01] Charles: Excellent. Thanks Dave and thanks everyone for joining again. This session has been recorded. We’ll have it on our website and our YouTube channel. Email us at info@decisionwise.com if you do have any questions. Thanks everyone. Thanks Dave.
[00:53:22] Dave: Thanks everybody. Bye.
