Matthew Wride discusses 11 things to watch out for when looking to do 360-degree feedback surveys. He talks about how no process is perfect, and the 360-degree process can have some serious shortcomings when not handled effectively.
Multi-rater feedback (or 360-degree feedback) can be a powerful tool in your employee development arsenal. If not properly implemented, it can also have some serious drawbacks. However, when those shortcomings are studied and managed, the benefits and results of a 360-degree feedback program are heightened and improved.
At DecisionWise, we are strong advocates of a well-designed 360-degree feedback program. So, you might find it strange for me to be writing on the hazards associated with 360-degree feedback. No process is perfect, and the 360-degree process can have some serious shortcomings when not handled effectively. Multi-rater feedback (or 360-degree feedback) can be a powerful tool in your employee development arsenal. If not properly implemented, it can also have some serious drawbacks. However, when those shortcomings are studied and managed, the benefits and results of a 360-degree feedback program are heightened and improved.
To recap, a 360-degree solution solicits feedback and evaluative information from a variety of sources that surround the focal person (i.e., from a 360-degree perspective). Feedback is solicited from one’s supervisor, from his or her peers, from direct reports, and from a miscellaneous category that we ingeniously call “others,” which is then compiled together with the focal person’s own self assessment. Confidentiality is maintained for the larger groups in order to enhance feedback quality. Most people providing feedback in a 360-degree process (i.e., the “raters”) work from a standard set of questions or from a pre-defined instrument.
When rolled out properly, a 360-degree solution will provide some of the most meaningful information a participant might ever receive regarding his or her business skills, communication skills, interpersonal abilities, emotional intelligence, etc. In fact, many of those with whom we’ve worked have gone so far as to describe the process as “career-changing.” But much of the effectiveness of multi-rater feedback is dependent upon a successful process.
Organizations find that 360-degree feedback is far more valuable than most simple performance appraisals or a once-a-year personal development chat. A participant in a 360-degree process is able to see how he or she is perceived across the totality of his or her relationships and really uncover those stubborn blind spots. 360-degree solutions also uncover patterns and themes, and they help clarify behaviors. Even more valuable, in fact, is the notion that 360s raise the “undiscussibles”—those areas that are often off-limits during casual conversations or performance discussions. 360-degree feedback provides absolutely vital information and it is a powerful tool that guides and encourages professional development.
That said, there are some shortcomings associated with many 360-degree solutions. Here is a list of some of the more common problems:
360-degree feedback can be demoralizing when presented without context. We often have distorted pictures of our performance (ok, not you, but everyone else). Sometimes this is an exaggeration of our good qualities, or even an over-emphasis on our potential derailers. A 360 is more likely to bring these to the surface in a less biased fashion. Because of potential misinterpretation, we recommend that all 360-degree feedback be presented to a participant through a debrief that is conducted by a trained coach or manager.
We find that many people attempt to guess at “who said what.” While 360s are geared towards confidentiality, there are times when it may be possible to pick out particular raters, particularly when it comes to written comments. For example, we find that use of language or situations that are common for a particular rater might allow the participant to quickly guess at the identity of the rater. When this is the case, confidentiality cannot be maintained for all raters, and thus feedback may be less than candid.
There are many times that, particularly with off-the-shelf surveys, a 360-degree solution is too broad in what it measures. For example, the instrument talks about customers, and attempts to measure customer interaction, when the participant has little to no direct customer contact. For this reason, we encourage clients to take the time to customize their 360-degree questions before launching them.
Participants and coaches may focus too much on isolated feedback that distracts them from seeing larger themes or behavioral patterns, which are far more important to the participant’s long-term success and development. A participant may become so focused on one comment or piece of quantitative feedback that he or she misses the big picture.
Some organizations do not place enough emphasis on the quality of information that is obtained through 360-degree feedback, or they downplay the solution’s value. A participant will sense this lack of commitment and never really dig in and find those nuggets that might transform his or her career. Never drop off a report with a pithy comment such as, “It looks like you are doing great.” If that’s happening in your organization, then your missing the point and wasting your time and money.
360-degree feedback is a snapshot in time; it’s not a movie. Thus, feedback might be impacted (positively or negatively) by extraordinary factors that otherwise normally do not exist. In addition, because 360-degree feedback is static, it cannot show changes across time, nor can it account for variability based on longitudinal dynamics. One factor that we need to consider is what we refer to as “recency.” This is the tendency to provide feedback based solely on the most recent experience with the individual.
If a 360-degree solution is used for administrative action (i.e., promotion, succession planning, hiring, firing, or compensation), it could be gamed by the participants. You may also get some “you scratch my back, I’ll scratch yours” going on.
360-degree solutions require commitment from the organization and those participating, since it can take a lot of time to complete the surveys and to do so with care and consistency. As mentioned previously, when the support isn’t there, neither is the quality of the survey process.
Sometimes the rating scale is not universally applied. We are often asked for “norms” or “benchmarks”—comparisons against other individuals or organizations. However, benchmarks may be skewed or ineffective as it is comparing apples to oranges in many cases. For example, does it make sense to compare the CEO of a tech firm to a new teacher in an elementary/primary school? Probably not. Similarly, organizations often use different rating scales, or have different standards of performance. What may be “good” performance in one organization may be considered only “fair” performance in another.
There is no guarantee that the rating scale (i.e., “good” versus “fair”) is universally applied among the raters for a particular participant, which leads to skewed gaps. Hence, it is important to spend time explaining the 360-degree process before implementing it within your organization. We recommend that you take the time to “launch” the process by giving people an introduction and an explanation of what will happen and what is expected of them.
Without follow-up coaching, a 360-dgree solution does not generally, in and of itself, create accountability or proper action planning. This is why effective 360-degree solutions typically include a coaching component for a period of time following the feedback in order to create awareness, action, and accountability.
As can be seen from the list above, the deficiencies can be real, and do exist in many organizations. Yet, multi-rater feedback can be a powerful tool when used correctly. I would emphasize that none of these weaknesses are fatal, nor would they justify not using 360-degree feedback within your organization. The benefits of a well-run 360-degree program vastly outweigh any negatives. And, with some consideration and planning, most of the foregoing shortcomings can be turned into positives, such as using coaches to debrief reports and to guide your participants’ action plans.
So, go for it! Take advantage of an impactful instrument. But do it in the right way. Otherwise, you may be doing more harm than good.
Receiving your results from a 360-degree feedback survey can be a bit nerve-wracking. It can feel like being on the business end of a firing squad: You sit there with a blindfold on wondering what people will shoot at you.Will they aim for your head or your heart? How will you know who says what? And how can you defend yourself against this anonymous army of sharpshooters?
As a leadership coach I conduct debriefs with clients on their 360-degree feedback. I find that the 90 minutes I spend with them explaining how to make sense of their report really turns the lights on for them and they invariably find great value in the feedback, be it reinforcing or redirecting. There are very few cheap shots and never a need to defend themselves. Instead, they find ways to embrace the data and move forward with it.
In the debrief we try to answer two overarching questions:
What do the numbers and comments really mean? What do people hope you’ll hear through their scores and comments?
What does it mean to you, the recipient? That is, what should you do about it—if anything?
I’ll share my secrets with you: The four keys to unlocking the meaning—and value—of your 360-degree feedback report. Key 1: How well do the raters know you?
I begin by reviewing the demographic summary and asking my 360 friend to explain how well each of the raters knows his/her work. I ask, “How long have you worked with each, and how often and closely do you work with each?” This gives us some insight into how well-formed the opinions are and what perspective the raters are coming from. This is our first clue into what the ratings may mean.
When someone doesn’t know you as well in an area or on a specific item they’ll likely do one of two things: 1) answer “Don’t Know,” or 2) give you a safe score, typically 4 or five (on a seven-point scale). So, if you have a lot of 6s and 7s, a 5 looks like a bad score, but it may just be the result of low visibility into your performance on that particular item or area. The rater or group of raters didn’t have enough data points to give you a top-of-the-scale score and so chose to play it safe. This tends to happen most with peer groups because peers don’t usually work together regularly or they may even work at different locations. Key 2: The Tin Question and the Golden Question
One question I never ask when looking at 360-degree feedback reports is: Who’s right? That is, when the scores between rater groups differ (which is both natural and frequent), who’s score should we believe? For example, if I rate myself a 7 (top of the scale) on a behavior and my boss rates me a 5, am I just full of myself, or is my boss missing something? Is my boss’ score the one I should believe and not my own because, well … he’s the boss? What about if others rate me 7s as well—does majority rule?
These are natural questions to ask, but they aren’t even relevant because 360-degree feedback is subjective. It’s not about right and wrong, but perceptions of performance. That’s not to say the scores are given willy-nilly or pulled out of thin air; what it means is that people have different perspectives on your work—different expectations of you and differing datapoints of interactions with you. No one has an exclusive on what is true about you, just his or her opinion of your performance. And while that’s not absolute, it is important to understand. Others’ perceptions of your performance influence your credibility with them, their trust in you, and the strength of your working relationship.
Instead, the “golden” question—the one to always ask—is, “Why did you get that score from that person or group?” This question produces great food for thought. For instance:
Why does your manager think this is a strength, but others don’t (or vice versa)?
Why does this group rate you the highest of all rater groups?
What type of interactions and work do you do with that group that is different from the kind of work you do with others, hence the different scores?
Considering the data in this way yields insights into what people mean by their scores. Key 3: Pick the One or Two Most Important Action Items—Only
Once the meaning of the data is more or less clear, you can now answer the second question: What should you work on?
A 360-degree feedback survey contains much more information than you can practically address, so it’s best to select one or two high-leverage areas to focus on. In doing this there’s an almost universal assumption that needs to be examined, and often discarded: “I should work on my lowest scores, obviously.” Not so fast.
The practical value of a 360-degree survey is to give you input into how others perceive you are doing on a vast array of performance competencies. But you have to provide the key ingredient into making the data meaningful: importance. You ultimately decide what the numbers mean by determining the relative importance of each competency area to the work you’re doing.
For example, I often see people scoring in the 5.0 to 5.5 range (solid scores) in the category of Innovation and Creativity. When these scores are the lowest, they look like the obvious place to focus on improving. But when I ask them how important innovation and creativity are for their work, they often say they’re not important—and sometimes not even allowed! So why would you work on improving an area that isn’t important? Obviously, because you want to waste time and energy.
Sarcasm aside, this little exercise reveals that a score of 5.0 in an area of low importance probably makes that score perfectly acceptable. We then look at other scores in areas of importance that would be beneficial to raise. When you combine the importance factor with what others are saying, you have good input into what you’re doing well (so you don’t have to worry about improving those areas), and what would make the biggest difference if you did improve it.
Once you sift through the entire 360-degree report, you may be tempted to pick several items to improve. Don’t. Instead, prioritize your list of possibilities and focus your efforts on #1 and #2. The reality is that you’re going to have to find the time and energy to work on improving these areas in the midst of your already-busy life—so be practical. If you really do have three or four (or five) areas you want to improve, start by focusing on one or two at a time and make a plan to address the other priorities at a later date. Key 4: Give Yourself a Head Start on Improvement
Another common tendency is to basically ignore your best scores in favor of obsessing over your lowest scores. But while your highest scores typically indicate areas you don’t need to improve, they shouldn’t just be set aside. Instead, ask yourself how you can leverage those strengths to help you get better at the one or two things you’re going to work on. This gives you a head start on your improvement efforts and engages competencies you’ve already developed. I find that this is an approach most people hadn’t considered, but makes perfect sense once they think about it.
For example, if you have strengths in business acumen and are trying to develop your delegation skill, then use your knowledge about what matters to the business when you delegate tasks. There are several ways to do this.
Match skill sets to impact: Delegate high-impact tasks to a direct report who is highly competent. Give lower-impact tasks as development opportunities for less experienced employees.
Clearly explain the importance of the task so your direct report will give it proper attention. Your high business acumen gives you insights about the task’s importance that others may not see.
As you check in with your direct report in status meetings, you’ll be able to ask informed questions about how things are going, and provide expert coaching as needed.
Make the most of your results
On the face of them, 360-degree feedback scores can be at best confusing or mysterious, and at worst misleading or upsetting. That’s why it’s essential to have a way to think through the avalanche of numbers and shape them into input you can use. I’ve found these four keys to be both enlightening and empowering in explaining and taking action on 360-degree survey reports.