Experts in both the sciences and humanities work in a rugged landscape: it is not clear when one is at the highest peak or whether one is moving in the correct direction toward some goal (given the many hills and valleys that must be traversed). In this type of environment, there are no simple solutions, and it is not even clear when one has been successful, given that there are multiple goals (peaks) and many ways to get from the current position to one of the desired goals (peaks). Those who work in messy and wicked world suggest that it is very hard to assess progress in such a world. I think Scott Page would agree.
Not only are the people with whom an expert is working likely to be operating in this type of messy, wicked and rugged environment, they are likely during their coaching sessions to ask their coaches to focus in particular on the challenges of complexity inherent in this type of environment. Coaches aren’t brought in to help a client solve simple puzzles that have clear goals achieved through the application of existing skills (though both the client and coach might hope this is the case). Rather experts are often brought in to help recipients of their expertise prioritize multiple (and often conflicting) goals, navigate through rugged terrains and acquire new skills needed to meet the often-shifting challenges (Page refers to landscapes that are not only rugged but also dancing!).
Given the prevalence of these messy environments, how does one assess the extent to which a specific expert has offered credible perspectives or advice? If everything is linked with everything else in a complex environment, how does one determine whether or not action taken on specific expert advice has made a difference? Many other factors may have contributed to the success (or failure) either independently or (more often) in connection with the expert-based intervention. What constitutes “evidence” in a messy environment? How does one sort through relevant data from the “noise” of a rugged and dancing landscape? In this essay I will address the issue of nested problems and messy environments by asking: (1) who is sitting at the table, (2) what is the nature and size of the sample being studied, and (3) what is the nature of the information gathering methodologies being used. These fundamental issues must be addressed prior to any considerations regarding use of the evidence—for the evidence will only be influential if it is credible and the resulting expertise is likely to be either questioned or accepted in a naïve and potentially damaging manner.
Who is Sitting at the Table?
There is a revolution (or at least a readjustment) going on in the field of applied economics, especially as it begins to interact with the fields of cognitive psychology and neurobiology. This revolution often goes by the name, “behavioral economics,” and it is based in part on recognition that traditional economic theory, with an emphasis on rational decision-making and self- correcting economic dynamics, is to be challenged. (Ariely, 2008; Kahneman, 2011) One of the key points made by the behavioral scientists is that the criteria for assessing outcomes may be more important than the actual assessment that is being done.
Download Article 1K Club
