360° of Heaven or Hell?
In Q4 of 2011, I read an article in the CSCMP Supply Chain Quarterly by Kate Vitasek and Tracy Maylett, “A Hard Look at the Soft Side of Performance”. Kate is well known and respected for her work in warehouse metrics and measurements. This article stepped away from the standard discussion of productivity measures and focused on attempting to measure the softer side of management in order to improve operations performance. The premise is that with a focus on a basket of interpersonal skills such as leadership, developing and nurturing relationships, accomplishing team and individual goals, collaborating effectively, and communicating clearly, companies can improve bottom-line results.
It's easy to argue that interpersonal skills that promote and nurture strong relationships have a direct impact on operations performance. It is the interpersonal skills of leaders, managers, and employees that get the job done. One could argue that by measuring operational performance, you can measure the effectiveness of the interpersonal skills of everyone who works within the operation.
But what if you want to measure the interpersonal skills of a single individual? By measuring that person’s performance, you can improve the operations of the company. That is a noble goal, one worthy of study. A number of questions arise. What tools do you use to do that? How easy is it to measure these skills?
The article suggested a tool through a focused example of the use of the 360° feedback measurement tool. If you have worked in a major corporation in the past decade, you have experienced a 360° feedback program. When properly deployed, these assessments can be a great tool for improving an individual’s interpersonal performance. However, there is a dark side to the 360 assessment: as more companies jumped on the bandwagon, the new fad created just as many problems as it solved.
What Is the 360?
The German army in WWII was the first to use feedback from multiple sources to rate performance. It is also known as multi–rater feedback, multisource feedback, or multisource assessment. In 360, the subject of the feedback is figuratively and literally in the center of a circle, receiving constructive criticism and feedback from superiors, peers, and subordinates. There is also a self-assessment, a chance for the target of the feedback to rate their own interpersonal skills.
In a common deployment, a collection of anonymous supervisors, peers, and subordinates answer a series of survey questions, sometimes providing specific examples and written comments. In most cases, a 360 will include five to seven different raters in the analysis, while in some cases, the population of the raters can be quite large. Companies (usually through a consultant or an HR systems vendor) use a system to administer the program and collect the responses, generating a report that synthesizes the scores presented to the subject of the 360.
Lifting a line from Dr. Phil, "How does that work for you?" Well, let’s look at the spectrum of success.
Heaven
In a successful 360 deployment, the HR consultants or HR department personnel introduce the program to the participants, showing them how to complete the assessments. Often, trained facilitators engage in training and course-work to present the program and teach raters how to think about the task of evaluating someone else's interpersonal skills. The facilitators work with senior management to use the 360 data to coach subordinate managers to improve their interpersonal skills. Finally, successful deployments focus on creating positive outcomes, changes in behavior, and cultural changes within the organization.
When successful, 360-degree feedback processes provide feedback based on behavior that other employees can see. The feedback provides to the employees insight about the skills and behaviors that help them accomplish the mission, vision, and goals of the company, and live the values of the company culture.
When done right, 360-degree feedback helps each participant understand their strengths and weaknesses, providing insight into aspects of their work that need professional development.
Heavenly results are the product of a two-step delivery process. First there is a group feedback session for all participants, after which facilitators present the participants their written results and teach them how to understand those results. These facilitators are trained to ensure that the participants accurately understand the results on a personal level.
The best scenario is one in which people participate from an educated point of view, of their own free will. They learn best when they have volunteered. Voluntary participants will get more out of it than people who are forced to participate. The freedom to opt out helps build trust, and with trust, more people opt in. To add more incentive, even with the freedom to opt out of the developmental process, people still must be held accountable for developing their leadership skills.
Purgatory
When budgets are tight, management often encourages more “we will do more of the lifting” on projects like this. In many cases, the work of the facilitators focuses on training internal champions, and the delivery of the software used to manage the process. More of an IT solution than an HR solution, training focuses more on the tactical steps of managing the process, rather than on the strategic focus of working to a desired outcome. Training for evaluating others and management-coaching-skills training are sacrificed in an effort to control budgets. Because of the lighter investment, senior management places less importance on what becomes “just another HR project.”
In this Purgatory, the process still delivers some value, but nowhere near the kind of change needed to return on the investment in the systems and the effort. Participants’ ratings become more subjective, and potentially less focused. Sometimes there is a "Lake Wobegon" effect—nobody wants to give their boss or their peer a bad score. It is very hard to honestly rate your boss when your comments cannot be anonymous. Many programs fail to get enough subordinates involved in the process, making it easier for a supervisor to figure out who made which comments. In these cases, the comments sections remain empty, the exact areas where critical power can come from the process.
In Purgatory, supervising managers attempt to coach subordinate managers. Companies fail to foresee supervisor and participant anxiety as they examine the feedback, which prevents them from accurately understanding the information. Doubt about the results lowers trust in the intentions of the company. In some cases, because of the lack of senior leadership support, there is little or no real follow-up on the results.
Participants sometimes expect too much from this feedback system. When an organization fails to understand that 360-degree feedback is just one tool in leadership and interpersonal skills development, when they fail to integrate it into a comprehensive improvement process, the company fails to achieve the best returns on its investment of time and money. High hopes and expectations result in disappointment.
In Purgatory, the company, not the participants, own the information. The boss gets to see the feedback, along with HR managers. Successful programs rely on the participant to share or not share with his superior, or others. A lack of freedom not to share the results hardens resistance on the part of the participant, who sometimes refuses to accept the results. When the participant chooses to share the results with their boss, they openly accept the results and discuss the results to develop improvement plans.
Hell
Some companies use the 360-degree feedback as a mechanism to rate actual performance. When deployed in this manner, personal animosity or affection can weigh in heavily on the scores. When the company culture understands this mechanism, unpopular managers fall victim to negative comment. The most damage does not come from below, but from peers in other departments or areas of the company who do not appreciate, or have an animosity towards the manager.
Raters give positive feedback when they know that the feedback will affect someone’s job or pay. This is particularly true for upward rating; when the trust in the process is low and chances high the supervisor can tell who participated in the rating.
Now, ask yourself an important question. How willing are you to listen to constructive and challenging feedback when it could instantly affect your livelihood? Presented in the context of a performance review, constructive and challenging feedback becomes a series of negative and damaging assertions. When confronted with this information in this context, people may openly proclaim that they can and will change—out of fear of losing their job. Internal resistance will be high, so the outward acceptance hides the internal rejection.
This approach destroys trust in the process, and in the company. When used as part of the routine, the culture of the company becomes hostile. People will avoid giving honest feedback, and aggressive managers may go after the source of the perceived attack, the person who gave honest feedback. In fact, as mentioned above, peers can use the 360 process to attack and remove other peers with collective down ratings.
Misapplication or Misunderstanding?
The problem with measuring soft skills, which are measurable only through subjective observations, is that the measurements are soft. Measuring leadership ability by the opinions and feelings of others is a subjective art. In developing metrics, you want hard, number-based metrics. While clever people can devise measures for subjective opinion, the measures move with the movement of opinion.
Consider this: Is a score of 2 in communications good or bad? Go ahead, think about this. Don’t know? How about 9? Oh, you were thinking like a ball game, and the real metric in this program is like a golf score. Context is important, and context will change the meaning of a subjective, opinion-based score.
The base assumption—that a manager who develops great relationships with his staff and peers produces better results—is easy to rationalize and difficult to prove. This is generally accepted to be a true argument, but there is a lack of empirical published data to supports it. “There is no data showing that [360-degree feedback] actually improves productivity, increases retention, decreases grievances, or is superior to forced ranking and standard performance appraisal systems. It sounds good, but there is no proof it works." (Pfau & Kay, HRMagazine, 2002).
In fact, the co-author of the Supply Chain Quarterly article, Tracy Maylett, found no correlation between an employee's multi-rater assessment scores and their top-down performance appraisal scores. In a 2009 article in Compensation and Benefits Review, Maylett advises that care should be taken in the implementation of multi-rater feedback for appraisal. The research suggests that 360-degree feedback and performance appraisals get at different outcomes, and both 360-degree feedback and traditional performance appraisals should be integrated to evaluate overall individual performance.
The Supply Chain Quarterly article asserts that by measuring soft skills, company performance improves. The article cites reports from the consulting firm, Corporate Executive Board Company, University of Tennessee’s Vested Outsourcing research, and work by University of California-Berkley professor Oliver Williamson to suggest that a focus on softer skills improves supply chain performance.
Is the assertion true? Not really. In each of the examples, anecdotal evidence suggests that a focus on company-to-company relationships help improve bottom-line results. However, a parallel argument, made specifically with Dr. Williamson's material, suggests that using a focus on fair dealing, with honest and accepted metrics that are easy to measure and document and hard to dispute, coupled with a collaborative business-to-business approach, is the road to success.
The article fails to establish a strong connection between the real strategic power of using a development tool like 360 to improve bottom-line results. In fact, it suggests using the 360 process as a performance management tool, potentially managing the performance of individuals. That approach, as we illustrate above, is walking along roads that weak and ignorant companies can follow to drift into Purgatory and Hell.
The best approach to using 360-feedback is a voluntary integration of personal and interpersonal development that supports the mission and culture of an organization, fulfilling a strategic goal of creating a better company. Used strategically, with integrity that supports the culture of the company, multisource feedback is a powerful tactic for building leadership.
It is a poor measure of business performance.
Articles in This Series
Call Us! 877-674-7495 info@dksco1.com