I have a point of view about evaluation. I think it’s important for us, as coaches and consultants. It’s also important to our clients and client organizations. The reasons have to do with what are our respective concerns. My perspective is based on my learning experiences with ontological SMEs Fernando Flores, Julio Olalla and Bob Dunham, and evaluation SMEs Jack and Patty Phillips, Judith Hale and Ingrid Guerra-Lopez. In addition, it’s based on simply being human, with our predilections for making assessments all the time. This article will take a closer look at evaluation from four perspectives: evaluation and what it is and does; stakeholders and their relationship to evaluation efforts; measurement and how it can support evaluation efforts; and contracting and its role in evaluation efforts.
Evaluation
As human beings, we make assessments about things that are relevant to us, things that impact our concerns and affect our future. We have concerns about the direction and shape of our future. We will make decisions we believe will enhance that future. For example, when I select a conference to attend, I choose it for the sake of what matters to me and how I believe the conference will contribute (best) to me.
Other things being equal, my decisions will be more effective when I base them on the evidence from reliable, valid and relevant data. The quality of my evaluation will impact the quality of my decisions. This is the same for you, too. So two essential ingredients for making evaluations are some concern to take care of and some basis to make a choice among alternatives. As it is true for us, so it is true for the stakeholders regarding the coaching work we do.
Stakeholders
As human beings, we have multiple cares, concerns and commitments we take care of as we go through our daily activities. Some of those are personal. Others may be related to our professional roles in our organizations. When we coach someone, we call anyone impacted by the coaching results a stakeholder. In organizational settings, there are several stakeholders for our work: the coaching client (coachee), their manager, perhaps the manager’s manager, Human Resources and, of course, us. There might be other stakeholders as well, such as the client’s direct reports, fellow team members or external customers.
In that setting, what matters to each stakeholder is related to what’s important to them relative to the client’s changes and their impacts. For example, the client is probably interested in their experience during the coaching engagement, what they learned and how they can apply it on the job. The client’s manager might be primarily concerned with how their report is now performing on the job and the impact of those changes. And, as coaches, we may have cares about how effective we are, how well we managed the process, and what the client learned.
Each stakeholder has their own perspective and their eyes on what matters to them. What can we do to respond to them? In conversation with the organization, usually HR, we can make decisions about which stakeholders (or all stakeholders) we will account for with our evaluation efforts. Keep in mind that as the number of stakeholders increases and the number of evaluation questions to answer increases, the effort it will take to plan, design and do so (and the related cost) may also increase.
Measurement
The questions we want to answer through our evaluations will help each stakeholder answer the questions, ‘What happened?’ and ‘Were we successful?’ Any question will have a range of legitimate measures we could use. We must select among the alternatives for what we believe will address each stakeholder’s concerns, preferably given their input. For example, evaluation about the learning that occurred answers questions about ‘What do you know now?’, ‘What can you do now?’ or even ‘How confident are you that you will use what you learned?’ Evaluation about new or enhanced behaviors on the job might address such questions as: ‘How much better are you doing this now?’ or ‘What newly learned behavior are you now applying?’ If we are evaluating business impact such as quality, we might be asking about changes in rework, error rates or number of accidents. Finally, as a coach, we might be inquiring about our effectiveness with questions about ‘How soon were you able to apply this?’ or ‘What barriers did you experience to applying what you learned?’ Organizations may be imprudently frugal about whether or not to evaluate and indifferent about what to measure. I think it’s our job as coaches to raise the issue and have the conversation.
Contracting
That evaluation conversation can be held most effectively when doing initial contracting with the client organization. My experience is that when the evaluation conversation is ignored or held later in the process, it invariably produces upset and strained working relationships. When we talk about evaluation and what the organization wants to accomplish through our coaching, we set up a virtuous spiral of intention. Through our discussion, we get to elicit organizational support and alignment, not only for the proposed coaching but also for our evaluation efforts. We can agree on the stakeholders whose opinions we want to consider. We can talk through the logistics of data collection (who does what?) and data analysis (what methods will we use?). Either during contracting or within the first two months, we can discuss what measures to use and then select or design them. We can put those things in place that make it possible for us to tell the story of ‘What happened?’, ‘Was it successful?’, perhaps even ‘Was it worth the investment?’ if we do an ROI study. We tell that story in our report to stakeholders, sharing relevant data, interpretations and recommendations, while making conservative, credible claims about what we enabled.
Conclusion
Evaluation is an essential component of any change effort. People are curious and interested in what happened and what’s different, hopefully better, as a result. They are already making their own judgments, drawing their own conclusions, and making their own decisions accordingly. As the change agent on site accountable for both coaching delivery and project/process planning and management, we as coaches have the chance to shape the expectations, commitments and data access of the client organization.
Do we want evaluation to be a surprise and an afterthought? Do we want clients to ‘evaluate’ based on anecdote, irrelevant or invalid measures, or impressions? Do we want the organization to base its business decisions on a shaky foundation? I believe we must and can do better. Being a reflective practitioner, active planner and responsive partner about evaluation issues sets the table for mutual benefit. Let’s develop ourselves to be competent or masterful here, too. Grounded, defensible claims about results and trustworthy professional reputations, among other things, are at stake.
Resources
Anderson, D.L., & Anderson, M.C. (2005). Coaching that counts. Burlington, MA: Elsevier Butterworth-Heinemann. [One of the first books to look at evaluation, measurement, business results and the story to be told about the impact of coaching]
Guerra-Lopez, I.J. (2008). Performance evaluation. San Francisco: Jossey-Bass. [An excellent view of the role, issues, practices, variations and pros/cons of different evaluation methods]
Hale, J. (2002). Performance-based evaluation. San Francisco: Jossey-Bass Pfeiffer. [An excellent perspective on evaluation issues, considerations and procedures for how to conduct specific methods]
Lazar, J. (2006). Contracting as context for effective coaching. Presentation at ICF European Coaching Conference, Brussels, Belgium. [A perspective on the important role contracting plays in setting context for coaching and related support activities]
Lazar, J.B. (2015a). Measuring your coaching results: Did you satisfy your stakeholders? Invited presentation for ICF Phoenix Chapter monthly meeting, Phoenix, AZ. [A perspective on stakeholders, their perspectives and relevance to evaluation concerns]
Lazar, J.B. (2015b). Telling your coaching impact story: Targeting measurement and stakeholders’ concerns. Invited webinar for ICF Science of Coaching Community of Practice monthly meeting. [A perspective on the issues to anticipate and address so you can tell the relevant story of coaching impact]
Phillips, P.P., Phillips, J.J., & Edwards, L.A. (2012). Measuring the success of coaching. Alexandria, VA: ASTD Press/ROI Institute. [A recent publication about evaluation issues and the ROI Methodology as it applies to coaching opportunities]
Phillips, P.P., Phillips, J.J., Stone, R.D., & Burkett, H. (2007). The ROI field book. Burlington, MA: Elsevier Butterworth-Heinemann. [An excellent resource with guidance and tools for how to conduct an ROI study for any learning and development intervention]
Institute for Generative Leadership: www.generateleadership.com
Newfield Network: www.newfieldnetwork.com
ROI Institute: www.roiinstitute.net