Date: Wednesday, August 17, 2022
Time: 1:00pm Eastern / 10:00 am Pacific
Presenters: Dave Long, Chief Operating Officer, DecisionWise; Charles Rogel, VP of Consulting Services, DecisionWise
Cost: Free
Understanding employee engagement is critical to any organization. Listening to your employees is key to this understanding, but knowing what you need to ask them can be a challenge. Join us for a webinar on August 17, as engagement experts, David Long and Charles Rogel, discuss what to ask in your next employee engagement survey.
Register and reserve your spot today!
This Webinar qualifies for SHRM and HRCI credit.
Transcript
Charles Rogel | 00:00
<silence> Well, hello and welcome to our presentation today on What to Ask on your Employee Engagement survey. My name is Charles Rogel, I’m the Vice President of Consulting Services here at Decision Wise and your host. Our presenter is Dave Long, chief Operating Officer here at Decision Wise.
David Long | 00:16
Hello, everybody.
Charles Rogel | 00:18
Couple things before we begin. Uh, this webinar does qualify for one hour of SHRM and HRCI credit hours. So following the session, we’ll send an email with the codes to all participants who meet the criteria. And if you’re watching this on demand, please email us at info@decisionwise.com, and we will send you those codes. This webinar, along with all of our webinars, can be viewed on demand on our website or our YouTube channel. We won’t be sharing the slides from today’s presentation, but you can access the content from the recording. We do wanna welcome your questions and your comments. Just use the chat feature to post those questions, and we will try to respond to those, um, during and also after the session. With that, uh, Dave, let’s get started.
David Long | 01:05
Okay. Thank you for joining everybody. Uh, really what we are here to talk about, and hopefully what you’ve here joined us for is to talk about and have a conversation about, um, what to ask in an employee engagement survey. And decision-Wise has quite a bit of experience with this. We have been, uh, around for 25 years. Um, started I think more in the 360 arena. Uh, do do a lot of 360 degree feedback. Even today, a lot of our business is one-on-one, 360 degree feedback, coaching, that sort of thing. But one of the things that we’ve been doing for the last 20 years or so is to perform employee engagement surveys. And we’ve had a lot of data over the years that we’ve used, um, and collected. Um, we’ve tried this many different ways and, and, uh, and through many different methods, uh, but also, uh, asked many questions over the years on engagement surveys and have kind of come to, through our research and through our analysis of the data have come to what we think is a pretty, uh, a, a, a very solid methodology for, uh, measuring employee engagement and everything that is around really the employee experience.
David Long | 02:13
And that’s what we’re here to talk about today. I’m glad Charles is here with us. Um, his, his, his team is very much on the front lines of designing surveys and, and designing survey items, um, and really helping organizations make the most out their out of their survey experience. So we’re gonna love his expertise on this call or this, uh, webinar today. Uh, the agenda for today is gonna be, uh, we’ll talk first and just kind of go high level how to approach your engagement survey. Many of you may have already run, uh, engagement or org surveys in your organizations. Uh, we would like to talk a little bit about how we think about it, the questions that we might ask as we’re moving into the survey. And these are really good questions to ask before you start saying, okay, let’s design a survey. We wanna ask questions, uh, that, that will help prompt how we’re gonna do that.
David Long | 03:07
So we’ll talk about that here in a second. Um, we’ll talk about key measures to include in engagement surveys, some of the major measures that you might include, but also, we’ll, we’ll talk about some of the, um, some of the ancillary, uh, metrics that we’ve been inclu including in engagement surveys over the last few years. In fact, part of that discussion will be what are some of the trends that we’ve seen in employee experience and in these surveys and as we work with clients, uh, to kind of tell you kinda what, what are, what are other organizations interested in, or what have they been interested over the last few years, especially as we hit the pandemic in 2020? A lot of that shifted considerably. And so today, um, some of the things that we do with our engagement surveys a little bit different than what we’ve done in the past.
David Long | 03:51
So we hope to be able to give you some of that information today. Uh, we’ll talk about what’s the difference between an engagement survey and a satisfaction survey. As I know those words are both still used, I think, on balance. Most people are now using the word engagement when referring to their organizational survey. But, uh, but, uh, satisfaction is still word that’s used commonly, and it’s still something that we believe in and something that we try to measure. Engagement, anchor questions, uh, we’ll talk about how to kind of the direct measure of engagement, how to do that, how to determine engagement drivers, um, and how to include items on your survey that would be typical drivers of engagement, so that you cover all of those bases. We’ll talk about a model for that. Uh, what do we look at when we look for satisfaction elements or, or measures of satisfaction?
David Long | 04:39
We’ll look at some other key metrics. And then we’re gonna look at, uh, common findings that we see in our engagement surveys, at least, uh, over the last few years, what we’ve been seeing in engagement surveys and what you could expect if you were to run a survey in your own organization. Again, my assumption is many of you have probably, um, at some point or another, been involved in an engagement survey. Uh, so we, we won’t do too much definition of what, uh, what is an engagement survey. We assume that most of you will know that information. If you don’t, you’ll hopefully pick that up as we go through this. Okay. Um, talking about beginning with the end in mind, what are some of the questions you might ask yourself out upfront before you even start to design a survey? What are the key questions that you might ask yourself is, what do you plan to do with the information?
David Long | 05:30
And what I really mean by that is, who, who do you plan to have, see the data? Uh, will this be something that the HR team is mostly using? Is this something that you want executives to see? Is this information that you wanna share organization wide? Or you just want managers working in this information? So it’s an important question to ask yourself before you choose the questions you’re going to ask. You wanna ask yourself, who’s gonna be seeing the data? Are there action planning expectations around the data? And who would be expected to do that? Um, are we expecting action planning to happen kind of at the HR structural level? Do we want executives to be involved in that? And do we want managers to be involved with that? What you ask on the survey is going to is, or who’s gonna be doing action planning on the survey, will largely determine what you ask on the survey.
David Long | 06:22
If you’re gonna be having managers a, uh, do significant action planning on the survey, we wanna make sure that we have questions on the survey that are directed toward managers, or that they’re questions at least that managers can do something about. So knowing your audience for the data is as important as anything as you’re kind of determining what are you gonna ask on the survey. Uh, are there things that you would like to, to measure that you, maybe you do not want to act upon the things that you, maybe you’re curious about but you don’t, but we’re not ready to do anything about. And, uh, if, if that’s the case, and we’ll talk more about what to do with those kinds of items that you, you think, well, I, I’m kind of curious about this, but I’m not sure if we can really actually do anything about it.
David Long | 07:08
We’ll talk more about that as we get into this presentation. Are there any key metrics, key measures that you’re wanting to obtain? And this goes beyond just, um, we wanna understand, uh, the level of engagement. Do we wanna understand, uh, organizational communication or likelihood of attrition, or do we wanna understand diversity, equity and inclusion to a greater degree in our organization? And all those things are possible with an engagement survey, but you kind of want to have an understanding of what are the things that you would like to know from the survey before you even start. And the last thing that you would wanna ask yourself is, what, what sort of benchmark comparisons am I wanting to see? Have I done a previous survey? Do I wanna have significant comparisons between this year’s survey and the previous survey? Uh, then that would determine in large part, what you’re gonna ask on this year’s survey.
David Long | 07:58
Or am I looking more for external benchmarks? Am I looking for comparisons within the industry or companies my size or, uh, or just want to see a, a global comparison between my organization and all the other organizations out there? Um, or do I, is it more important that I just get really specific with my own organization and just have a very customized instrument that I can’t really benchmark outside of my organization? But we’ll have internal comparisons, one department against another, or, uh, one team against another? So, uh, another question you wanna ask yourself before you start the process is, are benchmarks important to me, and especially are external benchmarks important to me? Because that will enlarge part determine how you design your survey.
Charles Rogel | 08:45
Let me qualify this too by saying what we’ve prepared today is, is geared towards this annual employee survey where we’re collecting kind of a larger, larger amount of data to work with. And so we have kind of the ability, or at least flexibility to ask on several different topics and to really analyze the results and determine themes. This will apply to doing, you know, pulse surveys, other types of surveys in your organization, uh, but for the most part, best practices to conduct a yearly, you know, what we’d call maybe an anchor survey to, uh, really give you a, a broad understanding of what’s going on in the organization. And then you can focus specific in, uh, initiatives around pulse surveys and other things, uh, ongoing. Yeah,
David Long | 09:28
And, and that’s a really good point, Charles. ’cause another question you might ask yourself is, how frequently do you want to ask these questions? ’cause if you’re wanting to do more of a, a, a quarterly pulse or semi-annual pulse or something like that, you might think about trimming the survey a little bit so it’s not as long of a survey and maybe just trying to kick collect that information over time as opposed to trying to ask it all in one survey. Most organizations that we’re working with are still kind of on more of an annual model, and well, they’ll do an annual anchor survey as Charles referred to it. And if they’re doing a pulse or they’re doing several pulses, it’ll be a shorter subset of questions, um, that they’re asking on a quarterly basis just to, and usually on things that they’re working on, plus a quick measure of employee morale or employee engagement within that pulse.
David Long | 10:17
Uh, so they get kind of an ongoing understanding of where, uh, people sit in the organization. Um, that actually, uh, leads right into the next slide. I have, or, or I’m sorry. It’ll be the one after this where, where it’ll lead into kind of what is contained in a typical anchor survey. But before we do that, I just wanted to, to talk about, you know, since we’re sort of in a different world now, um, and certainly decision-wise has experienced a lot of change over the last few years as we’ve worked with clients, um, that, uh, that, that, that the concerns have evolved over time. And in 2020, as you can imagine, we had quite a rude awakening in that every single organization that we wanted to work with, after about, I don’t know, March 14th, 2020, started to want to understand more about safety, um, in their organizations.
David Long | 11:11
Usually a topic that had been relegated mostly to manufacturing organizations or, or public utilities or, or things like that, became at the forefront of almost every organization’s mind. And we wanted, we started to need to have items that would address safety in a different way than we’ve addressed them in the past, which would be more of a, you know, a health, um, safety, my, my personal health is safe. Yeah. Sort of questions in, in the survey. So we started addressing more of that in 2020, and that became the forefront of everybody’s mind. Charles, did you have something to add to that?
Charles Rogel | 11:48
No, I, I think you’re right. It was funny, you know, maybe 10 to 20% of our clients were asking questions about safety, and all of a sudden it was like 90%
David Long | 11:55
Or more <laugh>. Yeah. Yeah. I mean, it went very quickly in the direction of safety. And the other thing that happened in the summer of 2020 is that the world became more, uh, if they weren’t already aware, they became far more aware and far more concerned about diversity, equity, and inclusion issues. And that became at the, for especially probably around June of 2020, uh, we had always, as, as an organization, uh, we’ve always had, uh, measures for diversity, equity, and inclusion, uh, that we have encouraged. Most of the organizations, uh, that, that have run surveys with us, we’ve always encouraged them to include those within their survey. And most organizations have, um, at least on an implicit basis, included diversity, equity, and inclusion questions in their surveys. What happened in 2020 is that organizations wanted to become more explicit about specifically calling out initiatives and or commitment to diversity, equity, and inclusion within their own organizations in their survey. And we’ve, we’ve gone, it’s kind of a similar thing, Charles, wouldn’t you say about 20% were doing it before and we’re probably 80 or 90% now of, uh, including diversity, equity, and inclusion in, in surveys, at least here in the states. Yep,
Charles Rogel | 13:11
That’s right.
David Long | 13:12
So, okay. Uh, c communication, uh, became a priority, uh, because, because people were working remotely, people were trying to understand the level of communication people were receiving. Interestingly enough, in 2020 scores around communication really spiked. Um, the, the deliberate efforts that were happening in 2020 to communicate with employees really paid off. And, uh, and most employees felt more communicated to in the middle of the pandemic than they ever had pre previously. But along with that, because people were working from home, they also wanted to know, you know, how do we make sure people are productive or feel productive or have the ability to be productive and also collaborative? As we kind of got into more of the, more of the pandemic and, and people started to get used to it, we started having more different concerns. Certainly people were still concerned with safety, they’re still concerned with DEI, that’s, that’s still at the forefront of everybody’s mind, I think.
David Long | 14:09
But we also had things that started to come up, like, work from home. What are we, you know, what are we doing about this long term? What are we, uh, is this something that we’re gonna, that’s gonna come to an end when the vaccine is out? Um, or is this something that we want to kind of make part of our process going forward? Uh, what do we do about hybrid work, uh, arrangements, et cetera. Workload became a huge issue. Uh, we kind of came out of this period of almost a, a mini recession. I mean, it was a huge, huge recession, but for a very short amount of time. Um, and then all of a sudden people became very busy again, and many organizations had layoffs, and so they didn’t have as many people to do the same amount of work On the other end, and we had a re number three on this list was Reten retention became a big issue because many people started to leave their organizations.
David Long | 14:59
We called it the Great Resignation. And that something that I, that in large part is still going on here in 2022. Um, this idea of how do we retain our employees, DEI still front of mind, and then we started to, to, to have more and more organizations interested in culture and in many ways, culture in the way that it connects to, uh, DEI. And so in 2020, we have DEI in culture, sort of in the same category. Um, and that’s kind of like many, I’ve heard many people start call referring this to, uh, diversity, equity, inclusion and belonging. And belonging is kind of the cultural element that we’ve, we, we’ve added to the DEI discussion, which I think is really useful. Uh, when we’re looking at our engagement survey and looking at what are the things that matter to us, the, the, the subject of belonging, or as we call it a decision-wise connection is a really important subject to, uh, to understand if you’re running an engagement survey, it’s almost like if I understand the degree to which somebody feels like they belong in their organization, I can understand a lot about their experience just by understanding that they belong or they don’t belong there.
David Long | 16:13
Flexibility a big issue. Um, again, it’s kind of the hybrid work arrangements are, are still something that people are, are wanting to ask about on the surveys. Again, trying to understand retention through the survey. Um, what are the, what are the key parts of our value proposition that can help attract new talent? We can understand part of that from the survey as well. And then still, workload is a concern here in 2022. We would even throw in there the word stress. It’s not just workload, it’s people feeling stressed, um, for other things and not just related to the amount of work that they’re expected to do.
David Long | 16:50
So just to kinda give you an overview, uh, we will talk about each one of these bullet points, uh, as we go through the presentation, except for the first one, I’m just gonna tell you right off the bat, 40 to 50 questions is a typical anchor survey. Uh, you, you can, and the questions that we often get about this is, well, can we ask fewer questions than that? The answer, of course, is yes. You can ask fewer questions than that. Some people will ask, can we ask more questions than that? And the answer is, of course, yes. Um, but, uh, but you wanna be careful with not creating survey fatigue by going to 60, 70, 80 questions. We start to get people get people going through that survey and wanting to sign out of it, and some of them do. They quit and they quit the survey before they even click submit.
David Long | 17:35
So we want to keep it 40 and 50. Between 40 and 50, we think is the sweet spot that allows people to feel like they can finish the survey within 10 minutes. And also, uh, where we can get enough information to not o only understand the level of engagement in the organization, but also all the factors around engagement that are causing it. And we’ll talk more about that because, uh, because most, most, uh, surveys will include not only a direct measure of engagement, but also engagement drivers and then other ancillary satisfaction elements that relate to the, uh, employee experience. We also design our surveys around a five point agreement scale. We’ll talk about that here in a second. We get a lot of people that say, well, why do we do a five point agreement scale? Uh, why do we, why, why do we use, uh, an agreement scale rather than varying the scales throughout?
David Long | 18:26
We’ll talk about that here in a minute. We’ll talk about frames of reference or dimensions that you can include in your survey, different aspects of the employee experience that we would ask about and kind of group our questions that way. Um, uh, the other thing that we’ll talk about toward the end of this conversation will be, what about if we want people to be able to kind of freeform some open-ended comments in the survey, how do we do that? And we’ll be able to, we’ll be able to address that as well as we go as we get into this presentation. First, just to address the scale, we use a five point agreement scale. Um, if, if you ask me kind of the main reason why we use a five point agreement scale, the main reason would be because most every other, uh, survey provider out there uses the same five point agreement scale.
David Long | 19:18
And the second reason would be we’ve been using the five point agreement scale for 20 years, and all of our benchmarks are based upon a five point agreement scale. And so we really do, uh, hold to this very strongly because we want the best comparisons we can, we can possibly get from the engagement survey. A really good way to get that is by using a five point agreement scale. So I can compare now against other organizations in my industry, other organizations of similar revenue, similar size, similar geography. Uh, we are all based on the same five point agreement scale, and that’s why we use it. We give people the option, uh, in our survey to say, don’t know, not applicable. That way. If we have newer employees that are taking the survey and they don’t feel like they have enough information to answer question, they can say, don’t know, not applicable.
David Long | 20:07
And that really keeps that neutral area clean. And what we’re looking for with the neutrals are people that have had a combination of positive and negative experiences, um, which is why we really like to have a neutral option in our survey. And we don’t go with a four or a six point agreement scale. We love to have the neutral because we want to see the neutral. We, we’ve found are the people that are most likely to be impressed one way or the other, either to fall into the more unfavorable category or to in the future, be more favorable. And so we think that neutral is a really important thing to, to see and understand from a survey. So that’s part of the reason why we include a a, an odd numbered scale as opposed to an even numbered scale scale where there is no, no neutral response.
David Long | 20:49
Um, you would keep a consistent scale throughout a a, a consistent scale. Also allows people to get through a lot of questions without having to read and understand the scale every time. Uh, that’s a really important point as well. If you’re gonna ask 40 or 50 questions on a survey, if you change the scale on each question, then people have to now stop after each question and not just react to it. Now they have to understand the scale and then choose something within the scale. That’s why you don’t do kind of that multiple choice idea of everything has a different sort of set of responses. Everything has the same responses. Then now I’m reading statements on a survey and I’m reacting them to them as quickly as I can or, or, or as quickly as I want to, I should say. And what ends up happening is people are able to get through a survey like that in 10 minutes and sometimes less than that.
David Long | 21:38
Um, and if you had only 20 questions, it would be, you know, seven or eight minutes that they would be able to get through it. Uh, varying the scale also hurts your ability to analyze the data. I want to be able to compare the responses to one question, uh, you know, about tools and resources. And to another question about training. And I can’t do that if those questions are asked on different scales. Um, I wanna be able to understand very clearly when I look at the data, these are the areas where I scored high versus these are the areas where I scored low. It would be impossible to do that if we vary the scale throughout, throughout the survey.
David Long | 22:18
Uh, someone asked, can you not include neutral if you do not have new employees? Um, we have a small issue with that. If you don’t have new, really, the the new employee thing was the number six here, don’t know, not applicable that we include on our scale neutral. I want to include no matter what, uh, I, it’s a big preference for me to be able to have neutral. And, and the main reason is I wanna understand how, how much am I gonna be able to change the thoughts and opinions around this subject. And a good way to understand that is to look at the size of the neutral. So I personally try to include, uh, neutral no matter what.
Charles Rogel | 22:55
<inaudible> just this question around the, you know, Microsoft approach to measure thriving versus engagement when we get to the engagement anchor questions.
David Long | 23:03
Okay, that works. Okay. Let’s talk about frames of reference. And now we’re talking about kind of as we build the survey, what are the kind of main major categories? And I’ve seen this done a few different ways, and we do it a few different ways here at decision wise. One is to kind of say, here are the subjects or the themes that I want to understand from my survey. Uh, I wanna understand about communication, I wanna understand about training, I wanna understand about career development. You can do that. And then I can say, okay, I can fill in questions beneath that. Another way to do it in a way that we really like to do it is by using dimensions, uh, or frames of reference. So my job, uh, the questions related directly to my job are the, the work that I do every day, questions related to the people that I work with most frequently, or my team questions related to my direct manager or supervisor, and then questions related to the organization.
David Long | 24:04
Now, we also include if you, if you had within your area a next level supervisor that you wanted to include questions about, we do that sort of thing. Or if you wanted a separate section about senior leaders, we can do that. But essentially what we’re doing is we’re saying, looking at somebody in their job and we’re saying, okay, what are the frames of reference? There’s one frame of reference that I just look at the, the things that I do every day, that’s my job. And then the people that I see very often, my team, and then the person that I have to interact with as, as a, as a superior, that’s my supervisor. And then how do I feel about the organization? And interestingly enough, it’s how I feel about my job and how I feel about the organization overall. That mostly impacts my ability to engage.
David Long | 24:47
I can like my supervisor and like my team and not be engaged, but that’s still that. But even as I say that, we still know that team and supervisor relationships are extremely important. And I would say if I don’t like my team, I don’t like my, my supervisor, I’m not gonna stay in that job for very long. But if I can like them very much and still hate my organization and might still be engaged, is sort of the point that I’m making there. So it’s important to gather information from all of these frames of reference. And we like these as our major categories, not only because we think that they’re important categories to understand, but also because when we do factor analysis on our survey items, this is how the survey items group, it’s by job factors, it’s by team factors, supervisor factors, and organization factors.
David Long | 25:34
So, uh, so we really like to organize it this way in the survey, uh, mostly ’cause our research has indicated that that’s the way that we should organize it. Um, examples of, of, and I’m gonna try to sprinkle in examples of survey items that we might use as we go through this, but job would be things like, I enjoy the day-to-day tasks. I perform a team question or, or survey item might be something like, the people on my team treat me with respect, uh, supervisor, I trust my supervisor organization, I understand the vision and goals of this organization. So those are just kind of a taste of what you might include in those sections.
David Long | 26:11
The other thing that you want to include is a measure, a direct measure of engagement versus, and, and, and especially as that relates to to satisfaction as well. In order for us to have this conversation, I just wanted to really quickly talk about the difference between, if we’re gonna say this is an engagement survey, not a satisfaction survey, what do we mean by that? Well, satisfaction elements and a satisfaction survey, um, would, would really focus on the contractual or transactional side of the employer employee relationship. So I take a job and I expect by virtue of the fact that I’m taking a job, that certain things are going to be in place for me. One of the, an example of that is I, if I take a job, I expect that I’m going to be provided with a reasonably safe work environment. I expect that I’m going to receive tools to do my job, that I’m gonna have basic training, uh, in order to do the, the function of my job.
David Long | 27:14
And all of these things are things that I would expect to be part of my job. And, and by the way, satisfaction, I expect that I’m gonna be paid at a certain level that’s commensurate with other people in the industry doing similar things that I do. So satisfaction items, kind of what sets them apart is I could be very happy with all of these satisfaction items and still not feel super motivated to do my job. If I show up every day and I need a, a computer to do my job, and I show up every day and I have a computer, I don’t think to myself, wow, what a great thing that I have this computer to do. My job isn’t this wonderful, and how motivated do I feel? In fact, if I don’t but it, but if I show up to that same job and I need a computer to do my job, and I don’t have a computer one day, it’s broken or it it is gone, someone stole it from my office, whatever.
David Long | 28:06
I don’t know what happens with your computers. I usually have a computer, but if I didn’t have one, I would be very frustrated that I couldn’t do my work and therefore it would lead to dissatisfaction. But even because it can lead to a great deal of dissatisfaction, it doesn’t necessarily mean that the reverse is true, that I would all of a sudden be engaged just because I have it. So what we’re looking for is going from a transactional survey, which is all those transactional issues are still foundational. It’s very hard to be dissatisfied and engaged at the same time, at least over a long period of time. So we want to keep satisfaction as sort of a foundation and move into more of a trans transformational space where we have people that are engaged and we have the words, heart, spirits, hands and minds written underneath. Engagement, really because we want people, their, their full commitment, um, to, to the work that they’re doing, to the organization, to the team that they’re working with. We’re looking for that higher level of commitment, discretionary effort, whatever you want to call it. That’s what we’re looking for. We’re looking to connect at people, with people on a deeper level, um, in order to bring, for them to bring more of themselves to their work on a daily basis.
David Long | 29:18
In order to do that, really what we look for are things beyond tools, resources, safety, et cetera, where we’re looking for things that might connect with them, um, deeper. We’ve done some research around this in order to understand that we’ve had many, many items, hundreds of items that we’ve asked over the years on engagement surveys, uh, 150 to 200 items that we have that are fully be marketable for most industries in our, in the decision wise benchmark. And so we had a lot of items and a lot of topics to choose from as we went through and analyze the data. And really what we were able to come down to is generally speaking, and we’re talking about across many organizations, we found that there were factors that lead people to be engaged. And what luck those factors spell out magic. And those of you who have, who have followed decision-wise for a long time, would understand that decision-wise is very interested in making, um, engagement accessible and understandable to everybody in the organization, not just to the HR team.
David Long | 30:25
So we use, we use a framework like magic to try to help us understand what are the key factors that lead people to, to be committed at a deeper level. And as we build our survey, we not only look to build a, a construct in the survey that would measure directly engagement, we also look what are the things that are causing engagement? And they’re these things meaning autonomy, growth, impact, and connection. And I’m gonna get to those items as we get more into how we’re building out the survey. Uh, but let’s first talk about, talk about how to measure directly engagement, and we do that with what we call engagement anchor questions.
Charles Rogel | 31:03
Dave, let me answer the one question. I know there’s different ways people kind of define engagement and, and some of these topics that, you know, Microsoft recently started using the term thrive instead of engagement or kind of going beyond engagement. And they did describe it basically as to be energized and empowered to do meaningful work. And so we would say, well, that kind of fits or is a different way of kind of describing what we’re showing here because the energized piece we say is kind of the engagement piece. When you’re energized to do your work, the, um, empowered piece would fall under this autonomy concept where we see autonomy as a driver of engagement. When you have, when you feel empowered, you have autonomy, you tend to be more engaged. And then meaningful work, you know, ties into the m here for meaning, where if you feel like your work is meaningful, you’ll tend to be more enthused about doing your work. Um, and another thing I’ll say is, you know, engagement really drives the performance of someone and satisfaction’s more about retention. Where if those things are working, you’ll tend to stay at an organization longer, but you might not be as motivated to, you know, do your best.
David Long | 32:08
Uh, IIII honestly think a lot of these things are different versions of the same book. Yeah. Um, you know, thrive, it’s Microsoft calls it thrive, great thrive. It’s talking about the same thing that we’re talking about when we talk about engagement,
Charles Rogel | 32:23
Right? Right.
David Long | 32:24
Uh, you know, I think a lot of that is, well, let’s, let’s pretend like there’s something out there that’s something that’s greater than engagement. And there may be, you know, some, some level of emotional commitment that people have that is beyond just feeling engaged and, and, and happy to do their work. Uh, but, uh, but, uh, you know, we we’re just really talking about different, different versions of the same book, I think when we’re talking about the difference between thrive and engage, at least for my part.
Charles Rogel | 32:51
Yeah. And let’s
David Long | 32:53
Talk about how to measure ahead. Engage, go ahead, Charles.
Charles Rogel | 32:55
I was gonna say employee experience is kind of the, the next level where the ex employee experience is kind of the entire experience that we’re measuring on the survey, right? But we’re trying to see with them what is, what is resulting, is the experience resulting in people being more engaged? So if your entire employee experience, is it a good experience that kind of motivates you to be more engaged or allows you to be more engaged? So we would connect it that way.
David Long | 33:19
Right. Uh, very good point. Uh, in fact, if you were really to classify our surveys accurately, you would really call them employee experience surveys as opposed to just engagement surveys. Yeah. Containing within the employee experience survey as more of an engagement, uh, survey. But within it are many factors that influence the employee experience, and that’s why we, we categorize between engagement and satisfaction. Satisfaction items are a huge part of the employee experience, and we don’t wanna demean them or say that they’re not important. They are important, but, uh, but what we’re trying to do is we’re trying to understand what is leading to that higher level of commitment within our employees that that causes them to bring more of themselves and to perform at higher levels. Okay. Um, so let’s talk about how do we measure that? Um, we do using engagement anchor questions, and most of our surveys include about five questions, uh, that are measuring engagement.
David Long | 34:21
And they’re designed to measure, and this is where I think people have a hard time as we’re working with them in building the survey, they’re designed to measure outcomes of engagement. And what I mean by that is they’re not designed to be directly actionable, meaning it’s when, when we’re asking these questions, we’re asking for a general feeling of, of where they are mentally, emotionally, as opposed to asking them specifics about their work. So people are like, well, those aren’t very actionable, and that’s true, but what we wanna do is we want to action on the rest of the survey in order to influence their level of engagement. So, um, and, and, and the, our engagement anchor questions are designed to address different aspects of engagement as determined by research. So there’s research out there that that tells us that different aspects of engagement are the level abs ab absorption that you feel into your work. How, how absorbed do I feel on a day-to-day basis? How dedicated am I to the tasks, uh, to, to making sure that I can get to the end and finish a task? How much vigor do I bring? I don’t love these words, and we don’t use these words anymore in our surveys, but we ask questions that kinda get at these topics.
Charles Rogel | 35:34
Yeah. We try to translate that into human speak
David Long | 35:37
On the survey. Exactly. Absorption, dedication, and vigor. Uh, we don’t really use those are, those are the words though that were determined in, uh, through the research in this that happened in the ni in the nineties around engagement. And we still use those same concepts today because we still think that they’re very accurate and very valid. And frankly, our, uh, our running surveys for the last 20 years, 20 years have done nothing but confirm that those are important factors of engagement. What we try to, uh, we try to understand those things by asking, kind of, as Charles put it, we put it in layman’s terms. Uh, like for example, this one, this is one of our major, uh, engagement anchor questions. Most days I look forward to coming to work. Uh, now actually that’s evolved over time because we, during the pandemic, and even still today, much of the time, we, we say most days I look forward to starting my work or beginning my work or something.
David Long | 36:28
Yeah, exactly. So, uh, used to be we’re all going to work, but, uh, but now we sometimes just stay at home and, and start our work. Uh, but there are other questions like, like, uh, overall I love my job or time passes quickly while I’m at work. Things that help us understand the level of engagement. And again, I’m not gonna be able to act on most days. I look forward to going to work. I can’t just go out and tell everybody, Hey, you should look forward to going to work more. Please do that now, please <laugh>. Um, it’s not gonna be very effective. Uh, and so what I have to do is I have to create an environment where people would, would now feel that I look forward to coming to work every day. Um, and, and so we’re acting on the rest of the survey and that’s why we’re, uh, we’re big on, uh, understanding what’s actually causing engagement because we can address those factors directly.
David Long | 37:21
The cool thing about using an engagement index is now we’re able to classify your organization. This is sort of the outcome of using an engagement, uh, engagement anchor questions. We can index employees in an organization on, uh, into four different profiles. We’ve got your fully engaged, uh, employees, and they’re everything that you think that they are. Uh, they’re very motivated, they’re very excited, uh, to be working there. They often come in with fresh ideas and innovation. Uh, they’re often trying to, they’re putting in more effort than they really have to to get their work done. They’re often working on proactive projects and things like that. We love having people that are fully engaged. In most organizations, that’s only about 30% of your, your population. Key contributors are, uh, as we kind of go down this index of, of employees, key contributors would be people that are kind of very satisfied, maybe sometimes engaged, but not, uh, not really that, uh, not really that fully engaged, uh, profile that we talked about just a second ago.
David Long | 38:25
We would put them more in a category of being kind of your strong and steady population. About half of most organizations are comprised of key contributors. That’s not a bad thing to have key contributors. Uh, it’s definitely a good thing on balance. Of course, we’re looking for more and more people in that dark green, fully engaged group, but it’s really good to have a foundation of, of, of strong key contributors in your organization. The reality is, if you have everybody that’s fully engaged in your organization, that can become exhausting to manage. Uh, just people, everybody’s excited, <laugh> excited to do their work. I know that sounds awful for me to say, but you do want some people that are, that are strong and steady in addition to having people that are fully engaged. And a good mix between those two is really what we look for.
David Long | 39:09
And on balance, of course, we want more people in the fully engaged category. Opportunity group is a very neutral group, and this is one of the reasons and the question that we was asked earlier, do we really wanna include neutral? Yes, we do. And one of the reasons why we want to include neutral is to understand how many people are in, are in an engagement group that we call Opportunity Group. This is a group that will move. So as we analyze our data, we see we have people in the opportunity group. This is a group that’s either gonna move up in their engagement, down in their engagement, or they’re gonna leave the organization between now and the next time that we survey. And largely what we do with the data for our, from our engagement survey and how we handle ourselves over the next year, and the kind of experience that we create for employees, we’ll determine if this group is moving up, moving down, or moving out.
David Long | 39:55
So it’s important to understand what is our opportunity group, that neutral group, and also what are the things that are kind of keeping them on the fence? Are there items in our survey that are leading this group to be more on the fence? And that’s again, what we’re looking for when we’re looking for drivers of engagement, fully disengaged. This is about 5% of most organizations. That’s, that’s the average that we have in our database. It it’s, it’s exactly what you think they are, which is they’re board frustrated. Uh, they’ve, we sometimes call this the quit and stay population in the organization. Um, and you’re unlikely be to be able to change their opinion. They’re not gonna wake up one day, no matter what you do. They’re not gonna wake up one day and say, you know, maybe I had it wrong. Uh, I really do love this organization. It’s, it’s just unlikely they’re ever gonna get to that point.
David Long | 40:43
So once we understand the engagement index, and once we understand where everybody in your organization sits, and again, we’re, we’re, we’re talking about confidential surveys. So even though we’re classifying employees as engaged or disengaged or opportunity or key contributor, we’re not identifying who is what. We’re just putting them into groups in order to understand further drivers of engagement, we’re trying to understand what is leading people to be fully engaged, what’s leading people to be on the fence, what’s leading people to be fully disengaged? And we can do that most effectively with engagement drivers. So because we’ve known this for a long time in the last decade, we’ve really tried to simplify this into a framework that helps us understand what do we need to include in our survey in order to understand what would be causing people to be engaged. And so when we’re doing survey design, we lean on this model a little bit to say, alright, we need to understand the level of meaning, autonomy, growth, impact and connection that people have in their jobs.
David Long | 41:44
So what do I mean by that? Let’s give you some examples. When we talk about meaning, what we’re saying is, your work has purpose beyond the day-to-day tasks you perform. And the way that we would test for that in the survey would be to say, this is one of the questions. My job provides me with a sense of meaning and purpose. Very simple. It’s a question we’ve been asking for years and years and years, and often shows up as one of the key drivers of engagement for most organizations. Second, autonomy. What do we mean by that? This is the power to shape your work environment in ways that allow you to perform at your best. A sample question would be, I simply, I have the freedom to decide how best to do my job. Um, again, looking for, what we’re looking for here is do people feel trusted to do their work?
David Long | 42:35
Do they feel the trust of their manager by the organization? Do they feel like they’re being micromanaged? Um, all those things are important so that we understand. Do people feel like they have enough autonomy to feel engaged as they feel more trust and more trust is placed in them, they’re going to, they’re going to react to that positively and become more engaged. Next one is, do they feel like they’re making professional progress, being challenged and stretched in ways that result in personal and professional progress? Is the definition that we have for growth, do they feel like they’re progressing? And this doesn’t just mean that they feel like they have, uh, promotion opportunities, although that can be a part of this. I’m not gonna downplay that. People definitely engage when they’re building toward an, uh, a promotion and right after they have a promotion. But what do we do in the other times to make sure that people feel like they’re getting enough challenge enough, uh, stretching in their work in order to feel like they’re making progress?
David Long | 43:28
Uh, are we helping them add to their skillset? Are we helping them add to their, uh, to their resume, really? I mean, uh, in, in, in broader terms, that’s what we’re looking for. But we’re really looking for people to feel like they’re, they’re stretched. And we, we look for that by looking for growth. And we simply would ask people on an engagement survey, my work gives me opportunities to learn and grow impact. Do people feel like on in the day-to-day work they’re performing, that they’re seeing positive outcomes, that they see that the work that they’re doing is, is effective, that there’s positive results from their work? And that’s really how we define impact. And we might ask an impact question, which is, most days, I feel like I’m making progress on important work projects or initiatives. And then connection, a very critical part. And often what is the biggest driver of engagement in organizations is connection.
David Long | 44:23
Um, and it’s simply stated, the sense of belonging to something beyond yourself. If you feel like you belong in your organization, you’re much more likely to be engaged. And a sample question for that is, is very simply, I feel like I belong here, shows up very often as one of our key drivers of engagement. When I say, and I’ve said that a number of times, what do I mean by it shows up often as a key driver of engagement? Well, we’ve done research on all of our database that helps us, that helped us come up with this magic acronym. This, these things that we know are very common drivers, enga age of engagement. But it turns out each organization individually has a life of its own, uh, has its own sort of, its own personality, its own culture. And so when we go from one organization to the next before we survey, it’s hard to predict what is, what is actually driving engagement in this organization.
David Long | 45:16
It’s usually some mix of these five items, but it may not be, it may be from outside of this, these five items. And there may be one of these items that is more of a, a driver of engagement than the others. So we do driver analysis with, with most of the data that we perform. And really what we’re doing is we’re taking the anchor questions as the, uh, de dependent variable and setting everything else as an independent variable and trying to understand, okay, all these things that we’re asking about the the employee experience, what are the things that are driving engagement in that organization? And it looks something, um, I’ll show you on the next slide what it looks like. So while magic elements provide a good framework, it’s not always, it’s not always, uh, gonna show up as, as the drivers of engagement in your organization.
David Long | 46:02
It’s important to include the magic elements because it’s a pretty good guess as to what is driving engagement in your organization, but it’s not always gonna be that way. Uh, so we conduct a broader survey, uh, with lots of different elements in the employee experience, and we put those all into this, uh, driver analysis. And it may include, uh, things from outside the magic framework. So let’s look at what that looks like. This is what a driver analysis looks like. Um, and this is not, uh, this is not what we, you obviously run this through, uh, a statistical program like our, this is not the input output, this is just the output that we create on our end when we’re, when we’re presenting this information. But we’re looking for what are the highest impact drivers of engagement. This is actually a real, um, engagement, uh, driver analysis that, that we ran on an organization where 83% of engagement in that organization was explainable by these four items.
David Long | 46:57
And so we can see, I’m proud to work at this organization, my pro, my my job provides me with a sense of meaning and purpose. That is both of those are meaning I feel like I belong here comes from connection. I feel challenged and stretched. That is a growth question. So it did turn out that we had three of our magic elements that were drivers of engagement in this organization. But again, we now can prioritize the magic framework and say, okay, we know of the magic acronym, M, G, and C are the most important things in our organization. And that’s where we can place our focus in, in order to, to drive engagement, uh, the most. So Charles, did you have anything to add?
Charles Rogel | 47:40
I think that’s spot on. Yeah. And we do notice that some of these, like you said, the magic drivers differ by organizations. So in general, magic, you know, shows up as some of the drivers, but it is unique to each, uh, company,
David Long | 47:54
Right? And that’s the important thing to remember when you’re designing your survey, even though you may think, oh, well we have the magic framework so we can make a shorter survey. That’s just an engagement measure plus meaning, autonomy, growth, impact, and connection. Yes, you can do that and you would get some pretty good, pretty, you’d be able to understand engagement to a large degree in your organization, but you also want to see, okay, is there anything else that’s out there that would also be driving engagement or even getting in the way of satisfaction? So we include satisfaction topics in surveys. So again, kind of building the survey we’ve talked about when we create frames of reference and we start filling our frames of reference, and we do that by adding first anchor questions and then we add probable drivers of engagement to that. And now we’re filling in the survey even further with satisfaction topics.
David Long | 48:44
So what are the topics that we would want to understand? Well, do people have the resources they need to do their job? Simple question. Like, I have the tools and resources I need to do my job well. Um, would, would be appropriate for that safety? Do people feel safe in their work environment? I feel safe in my work environment would be a very easy way to, to test that training. I have the training I need to do my job well. Very simple. We’re getting, uh, remember whenever, if you’re writing your own survey questions, you want to be very direct about the thing that you’re trying to measure and put it in the simplest terms possible. And we’re not trying to measure two things with one question. Like saying, for example, I have the tools and training I need to do my job well, would not be appropriate because, because people might put neutral and say, well, I have the tools, but I don’t have the training or I have the vice versa.
David Long | 49:34
So we wanna make sure that we’re separating each concept and everything that we’re trying to measure. And if we wanna understand training, don’t try to fool people into, uh, answering the training question. Just ask directly, do they have the training and they need to do their jobs Communication. This organization communicates well about what is going on, would be a question that we would ask very commonly about communication, collaboration. This case we’re talking about cross function collaboration. So we, we work effectively across departments and functions. Stress is a big one that we’ve been asked asking about a lot more lately. It’s always been kind of a part of our survey, but it’s been more of an emphasis over the last three years. The level of stress in my job is manageable. Same with workload. The amount of work I’m expected to do is reasonable. And then paying benefits, we’re gonna talk more about paying benefits here in a minute, but if you did wanna include a pay and benefit question that would fit into the, into the category of satisfaction, it is not Pay can motivate people over the, as we’re building up to a raise, and they’ll feel a certain level of motivation after the raise.
David Long | 50:39
But in the 12 months in between that or the 10 months in between that we need something else to kind of get them more engaged. They, they get, they reach a point of stasis with their pay very quickly after they receive a pay increase. And then you’ll get some, uh, boost in motivation directly following that. But then, uh, you know, you’re gonna need, uh, you’re gonna need something else to people keep them engaged the rest of the year.
David Long | 51:06
Another, so now we’ve, we’ve filled in the survey with engagement anchor questions, with, with drivers, with satisfaction elements. Another element that we include in our surveys would be this, a co open-ended comment question, meaning we’re asking an open-ended question to try to get people to react to, to it and, and really write as much as they can about their experience. So most engagement surveys, uh, provide this opportunity to their employees. We usually include one or two comment questions. If we include one comment question, it would be simply a neutrally worded item. Something that says, tell me more about your experience working at this organization. And we’re not assuming that it’s good or that it’s bad, and we’re just letting them determine how they feel about their own experience. It can be a good way to understand where the sentiment goes when then I’ll open-ended question and we certainly do measure sentiment in some of our analysis.
David Long | 52:05
Uh, what more commonly I would say we use two comment questions at the end of the survey. Um, and they would be one positively framed question and one more negatively framed question to understand what do they like about their job and what do they not like about working for that organization. Um, we can, the great thing about comments is that we have the ability to using, uh, algorithms and, and, and computers and systems. We are able to categorize and analyze the surveys. So the, the responses. So we know we had 600 responses that all mentioned communication and and 200 responses that men mentioned the manager and 150 that mentioned payer benefits. So we can categorize it that way, but the real value in comment questions is to be able to read them and understand them on a one by one basis. Sample question would be, what are the areas that need the most improvement in this organization?
David Long | 53:03
That’s a negatively framed comment question. And you let people react. That’s not specific. And some people, sometimes people want to get very specific with this comment, questions are the easiest way to create survey fatigue. So we usually don’t like to go beyond two. If we do, maybe we’ll go to three or four, but really not anywhere beyond that because people get tired of responding to open-ended questions very quickly. What we’re not getting specific with this, because we’re trying to understand generally where does their energy go when they want to talk about the things that aren’t going well. And likewise, we wanna understand where does their energy go when they’re talking about the things that are going well? Kind of, uh, and I know I’ve been talking very quickly, Charles, did you have anything to add to that?
Charles Rogel | 53:46
The comment questions also point to, you know, sometimes low scoring questions from the other questions on the survey, so they really kinda highlight or draw attention to some of the main topics you wanna focus on.
David Long | 53:58
Yeah, and I would say oftentimes if you read through the comments that are categorized, let’s say communication is an issue in your organization, they would, you know, you can go to all the comments that mention communication and see are there any ideas that are out there that that can prompt our action planning discussion. Mm-Hmm, <affirmative>. And it can be useful from that standpoint. So other key metrics that we might include, I’m gonna go through this very quickly. Um, employee net promoter score is something that we really love in pulses, uh, and for them to be valuable in pulses, we would include this question also in a a, uh, an a net promoter score question in a, an anchor survey as well. EMPS is a good general metric to measure employee morale or even a, it could serve as a proxy, a one question proxy for an engagement, uh, anchor question.
David Long | 54:47
In fact, you can use it as an engagement anchor question in your, in your survey. Um, and it can be if you wanted to, if you, if you have customer MPS your organization, you can place this on the same scale, a 10 point scale if you would like to. And then, uh, one nice thing about NPS is that leaders and executives are familiar with the format and, uh, the NPS question can be used, as I said, as an anchor question. So it’s very simple. I would recommend this organization as a great place to work. You can also put it in the framework of, uh, uh, of, uh, customer NPS and, and have it say something like, on a scale of one to 10, how, how, how much would you recommend this organization as a great place to work? Something to that effect. So we, so it’s, it’s a great, uh, it’s a great way to kinda get a one question measure.
David Long | 55:35
And again, this works really well in pulse surveys. We also wanna understand how likely are people to leave the organization. So we include questions regarding the intent for employees to say, uh, or, or to not say to stay. So we create an an index that looks a lot like the engagement index using questions that are related to attrition. And that would look like something like I would choose to remain here even if the job was similar, pay and benefits were available elsewhere. And we, we would ask other questions related to that item and then we would create an index just like we did with the engagement index into category of fully committed employees, content, undecided employees and employees that are likely to leave. So attrition is another thing that you can really measure with your engagement survey. We did a, a webinar last month. Uh, you go to our YouTube channel to look this one up.
David Long | 56:27
It’s, uh, it’s something that, uh, we, we get into great depth. How do we include diversity, equity, inclusion in our surveys? Um, and we went through all these constructs on that. But, but this is a one that’s a very common ad and a common analysis that we’ve been doing over the last three years. And so something that you might consider including in your survey. And what we do is we divide this into kind of their own dimensions. Diversity, equity, inclusion, important topics related to that would be the level of voice that I feel like I have, the amount of growth opportunities I feel like I have. Uh, how much do I feel like I belong here and how much is this organization committed to diversity, equity, and inclusion? And we look at these items and, and really the whole survey through demographic comparisons. So we can look at it by ethnicity or race and we can look at it by gender, we could look at it if you have the information, we can look at it by sexual orientation or many other different ways that you would wanna look at it to make sure that we’re being inclusive of everybody.
David Long | 57:23
And the factors that we’ve really found are important to that are voice growth, belonging, and organizational commitment. So diversity, equity and inclusion is something we definitely have been including. I’m gonna skip over this part ’cause we just have a few minutes left. Um, we’ll include this in a future webinar. Um, just really quickly, what to expect? Well, you can expect, um, that there’ll be some areas that you do well that are commonly where you score pretty well. Questions related to the team confidence in the future. A lot of organizations score pretty well on that. Questions related to your direct manager or your personal impact, how much you feel like you’re impacting the success of the organization. And then most organizations score pretty well in the categories of tools and resources. If you’re running a survey for the first time and you’re thinking, what am I gonna have to do as a result of the survey, here’s a pretty good sum summary of where to expect to struggle.
David Long | 58:14
Where you, where we would say, if I, even without doing a survey, I would say you’re going to have a hard time with cross team collaboration, organizational communication, employee growth. You might also have issues with workload and stress right now because it’s more of a topical thing. It’s more been over the last three years where we’ve seen that in the lowest scoring items. There’re probably gonna be some issues around senior leadership, especially in larger organizations where people are pretty far removed from senior leadership. They’re gonna think senior leaders don’t know what’s going on in the organization. So you can expect to have something like that, uh, come back. And then if you choose to include, uh, questions about compensation and benefits, you are a glutton for punishment because you’re also gonna get very low scores on that. I can tell you that right now if you include it.
David Long | 58:58
And in fact, that’s why we have this section about what not to ask. Um, don’t ask anything that you don’t have any intention of acting upon. Uh, we talked about that earlier, but I’d say it again, if you don’t have any ability or intention to act upon something, don’t put it in your survey and communicate, Hey, this is something that we want your opinion on so we can do something about it. Um, questions related to compensation and benefits. We would encourage you and I encourage every client I work with not to include this, some of them still do. Um, but the, the reality with compensation and benefit questions is they almost always come back at some of your lowest scoring items and they’re gonna show up in every report that anybody ever sees is one of your lowest scoring items. And people are gonna say, well, you didn’t do anything with the survey if you don’t adjust compensation.
David Long | 59:43
So I would say leave it out and, and see bullet 0.1 for why you would leave it out. And then questions that ask, uh, employees to evaluate themselves. If you’re asking them, you know, how, how hard do you work or, or how much effort do you put in? Or how much do you care about clients? Those are all gonna come back at about 99% favorable. So anytime when you’re asking employees to evaluate themselves, that’s probably something to stay away from. Alright. Sorry we’ve gone through pretty quickly here. Um, but uh, what we’re at the end quick summary. As you build your engagement survey, um, you know, you really wanna look at those major con constructs that we’ve talked about, anchor questions, drivers of engagement. We want to include enough satisfaction elements to understand satisfaction in the organization. And it’s also great once you’ve kind of created that core survey to say, what are the other metrics that we want?
David Long | 01:00:35
Do we want NPS? Do we wanna understand attrition? Do we wanna understand diversity, equity and inclusion to a greater degree? And then you can build your survey from there. Try not to go more than 40 or 50 questions on your survey. Um, but, but, uh, we we’re always here by the way, as decision-wise, this is what we do. We’d love to work with anybody in, in creating the survey and, and actually running the survey and getting the most outta that survey. But this is the methodology we use and that we, we’ve really espoused over the years.
Charles Rogel | 01:01:04
Excellent. Thanks Dave and thanks everyone for joining again. This, uh, this session has been recorded. We’ll have it on our website and our YouTube channel. You’ll be able to access it there and it’s also eligible for Sherman HRCI credit. So we will be sending out that to our attendees, uh, and email us at info@decisionwise.com if you do have any questions. Thanks everyone. Thanks Dave.
David Long | 01:01:25
Thanks everybody. Bye.
