If one is at all interested in a broader assessment of the impact which a specific process or event has had, then the data sources must be expanded to include those who indirectly benefit from this process or event. I return to the concept of the complex environment in which most human based processes and events takes place. The environment is complex (and not just complicated) because everything is connected to everything else. Thus, information gathering must eventually address these broader, systemic issues. Evidence of impact must extend beyond the boundaries of the immediate process of event being studied. The behavioral economists push even deeper into the issue of information sources (e.g. Kahneman, 2011). They suggest that we often change the question we are posing when we either don’t like the answer to our original question or can’t find an adequate answer. We narrow our assessment because we don’t want to know about what is really happening—especially when what is really happening is outside of our control.
Gathering the Information: Numbers and Words
What measurement tools do we use? Can we also listen to what people say, observe what people do, read what people have written, and review other human accomplishments and artifacts? Must source of information come only from the realm of science? Must they be based in research-based measurement and calculations? Might they also come from the humanities with narratives, history and scholarship being engaged? This search for diversity of information sources is the next major challenge which we face in assessing a process or event—especially one that is human-based.
Admittedly, the tools are often predetermined. The information gathering question is often framed or reframed in a manner that presupposes the use of specific tools. We may even change the information gathering question on occasion to conform to the restraints of a specific measurement tool. At other times, a specific tool is quickly selected, setting aside the question of which tools might be most appropriate and even more importantly what occurs when a single tool is employed. I am not alone in suggesting that effective information gathering dealing with a complex phenomenon such as professional coaching should deploy more than one measurement tool—preferably at least three tools. This three-fold approach—often called triangulation—is a classic in the annuals of information gathering methodology. (e.g. Merriam, 2009) In fact, this three-fold approach is often identified not just with the use of three or more measurement tools, but also with the use of three or more sources of information.
This multi-source/multi-method approach is clearly quite demanding regarding both resource requirements (time, money, etc.) and the need for careful planning. This demanding approach, however, is worth the effort given the valuable outcomes that can be obtained. When only one measurement tool is used, the method of information collection itself can influence the system being studied. If two sources are used, those gathering the information risks obtaining contradictory information based in part on differing methodological biases. There is no clear-cut way to resolve these differences. Three or more sources of information allow for constructive resolution of these discrepancies. Typically, at least two of the three (or more) sources will yield similar information, or, at least, common themes. If all three information sources yield discrepant data, it is evident that the system being studied is complex, contradictory and in need of broader investigation.
Download Article 1K Club
