The Cosmopolitan Expert: Dancing with Numbers and Narratives

Unobtrusive Measures

Other types of information found in activity records (for example, schedule of appointments, budgets, evaluation forms and related archival sources) are of value, even if they do not have to do directly with the convening problem—for archival sources of information tend to be nonreactive or unobtrusive. The collection of this information will not disrupt or influence the continuing process or event. Contracts, minutes from important planning meetings, and even the informal stories (and jokes) about the process or event are of comparable value. These unobtrusive measures tend to be descriptively rich and persuasive. They reveal something about the “real life” of the process or event being studied

Obtrusive Measures

The reactions to certain data collection procedures are also of some benefit, for these reactions tell much about not only the process or event, but also broader environmental dynamics (the nature of the rugged or dancing landscape in which the process or event is taking place). For instance, the way in which an information gatherer is introduced and provided with an orientation and appropriate support services may be indicative of the level of support for a specific process or event in the organization. Similarly, to the extent that collection of information disrupts the flow of work when observing it, one can infer (with confirming evidence from other sources) that there is no precedent for peer observation and probably strong attitudes supporting autonomy. Such information has value when interpreting the apparent success or failure of a program to influence behavior. The response of people to the current initiative or to previous information gathering initiatives is indicative of attitudes, goals and receptivity to change. The obtrusive event serves as litmus paper. It helps in preparing a map of the landscape.

Performance Reviews

Various psychometric devices should also be available to someone studying specific processes or events. Performance reviews by supervisors, peers or others in the organization (often in the form of 360-degree feedback processes) can be used to determine relative levels of achievement in a specific area. Similarly, the performance of those engaging a process or conducting an event can be rated by the users of the process or those attending the event as well as other key stakeholders. It is often particularly valuable for this rating to be done by accessing multiple sources: the clients, the coaches and the stakeholders.

Questionnaires

A second psychometric device, the questionnaire, is used almost as frequently as interviews. Sometimes those gathering information will design and distribute a questionnaire that specifically focuses on the process or event being used in a specific setting. At other times, a standard questionnaire is used to cut down on design time or to compare one institution or program with others. A questionnaire can take many different forms: multiple choice, checklist, true-false, matching, scalar, short answer, or open ended.