These significant results can alternatively be interpreted (because of the wording of specific items) as the respondent’s rating of the extent to which they make use of these outside support services. The fourth item that yielded significant difference between the ICF certified and renegade coach responses concerned the overall influence of institutional conditions in which they have done coaching during their career. The ICF certified respondents are significantly more likely than the renegade respondents (.05 level of confidence) to indicate that they have been influenced by the institutional conditions.
What have we found? It seems that there are quite a few significant differences regarding responses to the Development of Coaching questions as a function of whether or not the respondent completed ICF certification. While most of the differences in mean scores as a result of gender and age were minimal, we find not only many differences as a function of certification that are significant at the .05 and .01 level, but also many that come close to significance (hovering at the .10 level of significance).
Unlike in our demographic analyses concerning gender and age, we seem to have discovered at least one of the sources of variance in the responses of coaches to the two surveys—though we should be reminded of Rey Carr’s cautionary note regarding Survey Monkey results. Furthermore, we need to be reminded that when many statistical calculations are being performed, the use of .05 and .01 confidence levels become suspect. Put simply, if one hundred calculations are performed, then five of them will be significant by chance. Technically, the levels of confidence should be adjusted—and the “bar” of significance raised when multiple t-test (or analyses of variance) are performed.
Given these cautions, it is important to note that the differences to be found among respondents who are ICF certified and those who are not certified (the “renegades”) are quite striking and do not resemble in any way the minimal differences to be found as a function of age or gender. Furthermore, there are some specific differences that reached significance: our respondents seemed to be discerning in their rating of specific items. There is not some generalized “social desirability” or “acquiescence” biases that impacted on one of our two groups.
In sum, we do seem to have hit “the mother lode” regarding identification of at least one of the major factors contributing to variance in mean scores–and, this is with the division of respondents into two very rough categories (especially those in the “renegade” category). With finer differentiations in future studies, even greater differences might be found. Keeping these caveats and considerations in mind, we turn specifically to the significant differences we did discover and speculate on what these differences might mean.
