Wither Comes the Information?
If there is clarity regarding the criteria to be used in the gathering of information, then the next central question concerns the people from whom the information is collected. This question, in turn, breaks down into two parts. First, how many people will be studied? Second, who specifically will participate in this study?
The issue of quantity is very important, for someone gathering information and someone serving as an expert can’t have it both ways. If the study is to be quantitative in nature, then the sample size must be large; if the study is to be qualitative in nature, then the sample size can be much smaller, but the information gathering itself must be intensive and in- depth regarding each person being studied. All too often, the sample size is small even though quantitative measures are being used. We see many coaching studies that yield conclusions based on much too small a sample size (under 50), even though the measures being taken are quantitative, superficial and often one-dimensional (for example, based only on self- ratings of satisfaction with the coaching process or supervisor’s ratings of the coaching client’s improvement in performance).
Later in this essay I will identify multiple methods of data collection and propose that three or more different methods should be used in any broad-based study of coaching practices. At the very least, larger sample sizes should be required – pointing to the value of collaborative information gathering strategies involving multiple coaches, coaching firms and organizations that use coaching services (I will have more to say about this in my Issue Three essay).
The small-scale quantitative information gathering project will rarely yield credible data. Without major funding, isolated projects are usually a waste of time. On the other hand, the small and highly focused qualitative study is feasible—even without major financial support. The project, often framed as a case study, can be quite valuable, though it is important (and should be obvious) that definitive conclusions regarding evidence of coaching effectiveness can’t be generated from these studies. The focus of qualitative studies should be placed on trying to understand the nature of specific coaching practices, rather than on trying to demonstrate that specific coaching practices are effective.
Both information gathering goals are very important. It is not enough to know that coaching does work. It is also important to understand why certain kinds of coaching work when addressing specific kinds of coaching issues. No one qualitative study will successfully address the second of these two questions, but each study helps—particularly if framed by a shared coaching taxonomy (or at least shared language regarding coaching strategies and practices).
The other big challenge is to identify participants in the information gathering project. Do we study both the coaches and the clients—as well as others impacted by the coaching process? Many years ago, the famous psychotherapy, Irvin Yalom, conducted a study in which both he, as psychotherapist, and his patient wrote in their journal after each session regarding their shared psychotherapy experiences. Yalom (1991) discovered that the accounts written down by himself, as therapist, and his client were quite different. I suspect that the same holds true for the coach and client. They are likely to identify quite different points in a human-based process or event as being important and may even convey quite different stories about what happened at a specific moment. Perhaps most importantly, they are likely to use quite different criteria in determining the level of success and the outcomes of any one step in the process or any one component of the event.
Download Article 1K Club
