In this article I begin with a fundamental premise: there are three primary goals regarding the future of coaching that are interdependent. One cannot happen without the other two: (1) the evolution of coaching as a field of professional practice, (2) the recognition of coaching as a cross-disciplinary field, and (3) the development of a research based common body of coaching knowledge. I wish to explore each of these three goals as they interweave and identify the challenges associated with their interdependence–primarily through presenting a case study of research recently conducted on professional coaching. I close with a set of recommendations regarding the future of coaching research.
Research and the Professions
Recognition as a professional practice requires that there be a common research-based body of knowledge; recognition as a cross-disciplinary field is grounded in that body of knowledge; a multi-faceted, multi-disciplinary coaching knowledge (on theory, practice and outcomes) requires a critical mass of academic researchers and coaching practitioners who are knowledgeable and skilled in the fundamentals of real-world research
Practitioners should have a sufficient understanding of critical perspectives and principles to be discerning consumers and users of research and be able to engage in skillful real world research by collecting and analyzing data from their experiences using the principles of post-modern qualitative and quantitative research methods (not necessarily using by-the-book methodological processes unless they intend to publish).
For coaches who want to be more discerning and informed consumers of coaching research I offer four areas for consideration.
1. Who. Who is the author? Is the author also the provider of the service or an outside reviewer? Does she/he have a vested interest in showing a positive outcome? Does the author have some experience and/or credentials that would support credibility?
2. What. What is the focus of this study? Are there clearly defined questions or directions for exploration or is this a fishing expedition? What is the purpose of the study? Is it defined in specific terms (e.g. to explore the impact of cognitive coaching on client’s workplace relationships) or broad terms (e.g. to prove the effectiveness of my patented method)? Setting out to understand a phenomenon or to document the impact of a coaching intervention is less prone to bias than setting out to prove that a particular method works.
3. How. How are the data collected and managed? Are there several sources of data (preferred) or just one (the author)? How does the author address data which are anomalous? How does the author address the issue of bias? How does the author ensure that data collection is inclusive and appropriate to the questions being explored?
4. Connections. Are the author’s conclusions supported by evidence? Is there a clear logic or reasoning? Does it affirm or challenge your experience? How can this be applied in your own work?
I often define research as a disciplined, well-informed curiosity. By disciplined, I mean that there is some system for formulating a question (or questions) to explore, for collecting the documentation/data, and for going about meaning-making. By “well-informed” I mean that the practitioner/researcher is conversant with the literature of coaching; that the focus and conclusions of a study are linked to both data from experience and from books and articles written by others. As a baseline, discerning coaches should be able to distinguish facts, inferences, judgments and assumptions in their documentation and engage in meaning-making with that clarity.
This broader framework of research for coaches can be applied for three different purposes: reflective learning from experience to improve professional practice; documenting impacts and outcomes; contributing to the development of the field through scholarly publication. In each case, the research process involves a systematic approach to collecting salient data and a meaning-making process using that data which recognizes distinctions between objective and subjective data, and draws conclusions based on the evidence in the documentation and relevant literature. Regardless of purpose, formal and informal research can be conducted by individuals or in a social contexts such as supervision relationships, action research groups or collaborative research partnerships.
Reflective Practice
All effective professionals–including coaches–are researchers reflecting on their work, building and testing assumptions about their practice. Reflective practitioners intentionally and systematically document and review their experiences in practice to refine and extend their knowledge and skills. The development of reflective practice has long been integral to the preparation of professionals in several fields which share with coaching a reliance on the integration of practice skills and theoretical foundations combined with some degree of artistry. The learning potential of reflective practice is especially salient for coaches, practitioners in a relatively new field with a nascent body of theoretical literature. Reflective practice invites the coaching practitioner to regard coaching engagements as opportunities for the development of judgment artistry.Coaching reflectively provides us with detailed information which enables us to: (1) better serve and assist our clients; (2) talk about what we do as coaches; and (3) document connections between coaching interventions and client change.
Reflective coaching practice is not formal research, which is guided by theoretical, methodological and ethical considerations. It is, nonetheless, a form of individual practice research which is consistent with a professional emphasis on continuous learning. The practice begins with sound documentation of all aspects of our coaching interactions. Many of us already collect such data in the form of coaching agreements, assessments, coaching plans. Most coaches also take notes or record individual sessions. The documentation for reflective coaching practice requires that we note both sides of the interaction–i.e. what we as coaches are thinking, saying and doing, as well as what the client is saying and doing. When we ask a powerful question, for example, what are we hoping to achieve with that intervention? When we notice ourselves feeling baffled or stymied, thinking “Hmmm, where can I take this?”, that internal dialogue needs to be recorded as well. Eventually, what we have is our own version of a 360, a global view of what happens in the coaching process: our intentions, the goals for the session, our actions and the client’s agenda, the responses to and the outcomes of those coach intentions and interventions. It’s important, and ethical, to be transparent with the client as we do this and invite co-participation in the reflection process.
The next step of the reflective process entails making meaning of the information we’ve collected. We may keep a coaching journal, for example, and look at the notes and reflect after each session, comparing the two sides of the conversation. What happened here? What worked? What was the turning point? What did I do with this client that made a significant difference? Reflection questions can take several directions. Here are four that I’ve found helpful. First, how did my actions as a coach match with my theory of coaching? Am I coaching in ways that are congruent with or contrary to the values and vision I present to clients? A second direction invites looking at the intentions, interventions and outcomes. This kind of reflection helps us make the best possible match between the specific coaching strategies and competencies we use and their effectiveness with individual clients. A third reflective direction encourages looking at where we, as coaches, are in “flow” and where we get stuck. A fourth direction for reflecting involves looking for themes or recurring patterns within or across client conversations. Each focus for reflection provides a framework for organizing and making sense of the information we’ve captured and suggests possible directions for action.
The plans we develop for ourselves in response to learning from reflection can move us professionally in several ways. Knowing what works best with each client enables us to engage in purposeful coaching, being more deliberate and strategic with each individual we assist. Knowing where we are stuck suggests areas for further professional development. Professional coaching organizations–through conferences, workshops, teleclasses and local chapter activities–offer on-going opportunities for learning across a broad spectrum of coaching skills. A professional development plan may include hiring a mentor coach. Self-development may also involve becoming familiar with the growing body of coaching research literature. If comparing our coaching actions with our espoused theories of coaching shows up a disconnect, we may want to revisit our mental models and do some work on realignment. Finally, noticing patterns of client challenges or coaching interactions may spark a curiosity which results in undertaking a formal coaching research project.
Documenting Impact and Outcomes
Much of the research in executive and organizational coaching initially focused on documenting impact and outcomes, often in the form of equating specific outcomes with dollar or bottom line value. Much of the data and documentation collected for purposes of reflective learning can be put to use to document coaching impacts. Formulating and documenting specific goals for the coaching engagement, for example, and providing benchmarks and timeframes for review, are a form of baseline data and provide the variables for follow up assessment. Many studies use pre-post assessments and simple, descriptive statistics can be helpful in showing impact. A coach who is consistent in using simple formal agreement and documentation procedures throughout a coaching engagement generates a rich source of data not only for single cases (think case studies) but also for meta-analysis across multiple cases.
Simply tracking results and client behaviors over a period post-coaching, as Lew Stern (2014) discussed in his LPC interview, can be valuable. In particular, post-coaching tracking can show, using objective evidence that the client’s changes in thinking and behavior have been internalized and are sustainable. It also provides persuasive evidence of the effectiveness of a particular coach’s model and practice as long as it is offered with a caveat about the limited nature of the data. Moving from data collection for individual learning to data collection and analysis for purposes of marketing does require a higher level of awareness regarding the potential for bias. If a coach sets out to “prove” his coaching works, the potential for limiting the data sources and overlooking contradictory information is high. Discussing and reviewing data with a colleague, mentor, or supervisor, or partnering with a coach who has formal research experience can be very beneficial in maximizing these research opportunities and producing results which are reasonably valid and credible.
Research for Publication
Formal studies published in peer-reviewed journals are essential if the field of coaching is to rest on a shared body of evidence-based knowledge and practice principles. While there are many qualified individuals who are engaged in this type of research as an individual pursuit, there are several benefits from partnerships and collaborations that bring together coaching practitioners and academics or formally trained researchers. Collaborative projects benefit from a mix of coaching practitioners and researchers who can contribute to meaningful dialogue on study design and development. Research partners and group collaborators also tend to have a wider range of familiarity with relevant literature.
In the case below, we had the benefit of an experienced researcher in the development of psychotherapists and a collective of academics and experienced coaching practitioners to provide input into the formulation of the survey items with language relevant to coaching and to offer different perspectives on methods of data analysis. With a collaborative or action learning group, a facilitator or coordinator is key in managing the process and in eliciting and integrating the input of the members. The case study offers just one way of proceeding with a collaborative study. While the initial group involved six individuals, the resulting study using data from the one qualitative question on the survey was conducted by only two of the members. The case provides some chronology, challenges and observations about the collaborative process.
Case Study: Producing and Publishing Coaching Research
The Development of Coaches project was launched through an informal conversation with psychotherapy researcher David Orlinsky and coach educator Linda Page based on David’s experience with the development of psychotherapists project. David generously shared the Development of Psychotherapists Survey (DPS) which had been used in an extensive, long-term study with that population. So we had the advantage of a well-designed instrument to adapt and did not have to start from the ground up. This offered (we thought) some credibility and validity to the study.
My particular interest, as an educator, was in uncovering some empirical data that would support the development and refinement of coach education and training programs and to also inform discussions on the distinctions between practitioners in the two fields of practice.
Setting Up the Project
We spent considerable time reviewing and adapting the original instrument to make it relevant to the coaching field. We also discussed the variables addressed in the sections of the survey and added one open-ended question to investigate how coaches perceive particular personal experiences influence their coaching.
After several rounds of refinement and revision, the first version of the survey was created on Survey Monkey. We drafted an invitation letter and the five members of our group who are coaches sent the invitation to coaches within his/her personal and professional network inviting folks to help us test the version and give us feedback. Our networks were broad and diverse, encompassing coaches at various stages of experience, expertise and specialization.
Feedback from the coaches who responded led us to make several revisions including shortening the survey (it was initially quite lengthy in order to be as comprehensive as possible and address as many factors as might be relevant in coach preparation). Many of the initial testers said that it took too long to complete (20-25 minutes). We made choices re: making some questions required and others optional (some respondents said that some of the questions were too personal–e.g. questions about their feelings while coaching or about family patterns); and allowed people to leave off taking the survey and return to it at a later time.
We gained in-kind support from ICF and the revised version was put on a Vovici platform through ICF. The open invitation to participate was issued through ICF newsletters and mailings, and through collaborative members again reaching out through their networks, including ACTO, GSAEC and LinkedIn. DCRC members also did presentations to various coaching groups and shared the link. The invitation invited any coach who received the invitation to pass it along to any coach of their acquaintance. There were no restrictions on who could take the survey.
Outcomes of the DCRC Project
Despite the broad and repeated outreach, the shortened questionnaire and the opportunity to stop/start, the results were a disappointing 185 out of coaches globally. One article did result from the study (Campone and Awal, 2012).The five life events which respondents most frequently identified as having significant impact were: Personal events (such as illness, divorce or family situations); Formal Coach Training; Work Experience; Practice Coaching; and Receiving Coaching/Mentoring. Different types of significant events affected different aspects of the coaching commitment and engagement. These impacts included:
• Insight about one’s self and/or about one’s world: including a restructuring of the interpretive framework, a realization arising from this, and a greater sense of empowerment over one’s life and the world around.
• Change in Knowledge. This category included changes in conceptual knowledge (e.g. a change in cognitive framework, theoretical understanding or knowledge base) and practical knowledge (e.g. learning how to do something new or differently).
• Change in Skills. Enacting different behaviors or the acquisition of specific skill sets relevant to coaching practice.
• Empathy. Empathetic understanding of or identification with the experiences and feelings of others.
• Self-Worth. Self-recognition or affirmation.
The results echo implications in the studies of differences in coaches from different professional backgrounds while surfacing additional factors which have an influence on coaches’ presence and practices. Different categories of events have different impacts, however, suggesting the need for a multifaceted strategy for the preparation of coaches which addresses cognitive, technical and affective aspects of the individual.
We found that both formal coach education in skills and theory have a significant impact, suggesting that both elements are important. Formal coach preparation which includes both theory and skills development can serve as the basis for informed decision-making by coaches and provides the ground for deepening professional reasoning and decision-making skills. Coaches learn from both positive and disappointing experiences. Adverse personal experiences lead to the development of empathy. Coaches with prior experience in counseling and psychotherapy may bring transferrable skills, specifically in the establishment of the working alliance, deep listening and the management of emotions. They may also bring a broader repertoire of intervention strategies to their coaching practice.
Given the diversity of prior professional training and experiences of coaches, results suggest that differentiating coach training and education may warrant closer consideration to address different practice contexts and introduce knowledge and skills as warranted by the practitioner’s prior experience and training.
While the potential for continuing this avenue of study remains with some members of the original collective having an interest, continuation would require a larger contingent of members of the collaborative and individuals with experience and expertise in undertaking studies of this nature. One original members, David Orlinsky, has asked for the Excel database with the 185 responses to undertake his own comparison with his development of psychotherapists work. A corollary consideration is what work needs to be undertaken to help coaches and coach educators and trainers to become aware of and apply these results.
Insights and Lessons from the Project
Collaborative Research
In undertaking a collaborative research project, it is helpful to spend some time at the outset getting a sense of participants’ interests, skills and how they frame their ability to contribute to the project. It’s also helpful to discuss the nature of the collaboration and work out details about shared responsibilities as well as ownership of the intellectual properties which result.
Collaborative projects would also significantly benefit from leadership by someone who is familiar with research and–more importantly–project management. Scheduling meetings, agenda preparation, follow through on discussions and agreements is very time consuming and requires as much (if not more) effort than the work of conducting the research.
A Research Culture
The Development of Psychotherapists study garnered many thousands of responses globally; the Development of Coaches study response was quite small. Why is this? I suggest there are some significant differences between the two groups. Presently, the field of coaching does not have a research culture. Except for some subgroups of coaches (e.g. academics, those trained in psychology or a similar field, organizational development specialists who are accustomed to outcome metrics and measures), research is not a part of the working vocabulary nor is it regarded as relevant to coaching skills and practice.
The professional education of therapists provides at least a familiarity with the forms and standards of social science research; coach training does not generally address this. Therefore, many coaching practitioners are not familiar with (or may be put off by) the level of detail required for good data collection nor do they have a basis for being discerning consumers of coaching research. Thus, the survey may have seemed overly long and personal. It’s interesting to note that many of the therapists who took David’s original survey found that the experience provided them with a useful (and for some enjoyable) opportunity to reflect on themselves, their practices and their professional journey. Coaches, by contrast, suggested that the level of detail was burdensome.
Next Steps
Several years ago, I conducted a series of interviews with coaching researchers to find out why they did what they did. The findings offered some useful insights in support of wider engagement by coaches in research. Respondents’ beliefs, value frameworks and thinking were challenged and stretched as a result of engaging in coaching research. As one stated “what you experiment with you learn from and research is a well-organized learning experiment.” This learning spiral depicts an upwardly emerging interplay between the processes of differentiation and inclusion.
Coaching research begins by taking a step back from the practice with an objective eye; becoming intimate with the observation experience; stepping back to look at outcomes; incorporating outcomes into practice; stepping back once again to assess and moving to higher levels of complexity at each step. Professional identity becomes rooted in the social network of the coaching research community. Coaching behaviors draw on and are polished by reflection on objective evidence. Coaching researchers acquire language and conceptual frameworks of increasing complexity to describe and enact coaching.
An inclusive coaching research model needs to include three elements. First, the model would use a “new sciences” orientation to the practices and processes of research. We must acknowledge that the quantitative, positivist orientation of the scientist-practitioner model has changed since its beginning. The scientist-practitioner model requires a minimum level of competence in statistical methods; recent views of the model have expanded to encompass evaluation, policy research and case studies as research contributions. Since its inception, the emphasis on scientific research has waxed and waned and while the ideal continues to be supported, effecting the ideal has not happened in large part because of the lack of research methodologies suitable to practice. In other words, “It is not that the idea was wrong but rather than inability to develop the tools to implement the idea.” (Barlow, Hayes & Nelson, 1984, p 23) The Heisenberg principle in coaching research underscores the need for the coaching community to consider collectively what and how we view data. A new sciences perspective can inform a view of coaching research in which data encompasses both lasting and ephemeral objects. We can borrow a model for analysis of such data from quantum physics where systems emerge from relationships between and among the parts and to acknowledge that such research design is organic and interactive.
The next element of a coaching research model encompasses defining fields of inquiry. In this model, coach researchers undertake systematic exploration of coaching research questions. These might include formal and informal coaching research:
• to develop a better understanding of the human change process
• to improve the accuracy and reliability of coaching assessment and outcome measures
• to develop more grounded methods of coaching interventions and
• to develop more grounded models for promoting client change and growth.
The research questions may be driven by the coach researcher’s individual interests or by the interests of some external organization. Alternatively, research questions may emerge as a result of constructed, collective thinking within the coach researcher community.
The final element of the coaching research model is an integrated support structure which comprises skills development, access to information and technical and collegial support. The education of coach researchers should provide practitioners with a solid grounding in the ethics and protocols of data collection and treatment to ensure the integrity and validity of coaching research. At a minimum, practitioners would have the skills to read and use research literature and to set up and use reflective coaching research practices.
Coach researchers also need access to a collegial community of experienced colleagues who can provide mentoring and technical support, enabling less experienced practitioners to work through challenges of research design or execution, someone who could accompany the coach researcher in her own development by being a helpful part of the process. A recognized coaching research community would support, promote and encourage both reflective practice and contribution to the field of coaching studies, and would reinforce these elements as an integral part of a professional coaching identity.
_______________
Reference
Barlow, Hayes & Nelson (1984 ) The scientist-practitioner: Research and accountability in clinical and educational settings. Allyn & Bacon: Needham Heights, MA.
Campone, Francine and Awal, Deepa. Life’s thumbprint: the impact of significant life events on coaches and their coaching. Coaching: An international journal of theory, research and practice. vol 5 (1) March 2012
Corrie, S. & Callahan, M..M. (2000). A review of the scientist-practitioner model: Reflections on its potential contribution to counseling psychology within the context of current health care trends. British Journal of Medical Psychology. September, 73, 413-427
Stern, Lew (2014) Interview: Research on Professional Coaching. The Future of Coaching @ The Library of Professional Coaching. www.libraryofprofessionalcoaching.com.