Cheri Tree ∙ Assessment tools need to have shown the reliability of inferences from their scores by assessing Cronbach’s alpha and split-half testing. For new tools > .70 is the threshold. More established tools, especially with over 20 items should be > .80. Sources of validity should correlate appropriately > .70 positively for convergent validity and negatively for divergent validity. Effect sizes and confidence intervals should always be reported. Many assessments do not have predictive validity and are, therefore, inappropriate for selection decisions, such as hiring, firing, promotion, etc., but are useful in other ways.
Q: How do instruments deal with bias such as academic discrepancies, cultural differences generational differences?
Tricia Nadoff ∙ Biases are universally present in all human endeavors and assessments are no different. However, there are things that assessment creators can do to both minimize the possibility for biased responses in the structure of the questionnaires and in the structure of the feedback. Further, assessment providers can also attend to insights about biases through their research. To deal with academic discrepancies in participants, most assessment providers will standardize on a reading level of questionnaire vocabulary at the eighth or ninth grade level (depending on the intended audience for the questionnaire, this may be standardized at a lower grade level).
Cultural differences are much more complex to attend to. First and foremost, language translation is critical since most assessment takers will be more accurate in their responses in their native tongue. To do an accurate translation the process requires several steps. First, the original questionnaire is translated into the desired language. Second, the translated questionnaire is back translated into the original language. Third, the original English text and the translated text are compared and discrepancies are worked through with the assessment creator and the translator in order to get a more accurate translation.
Once this is done and in person native speaking practitioner reviews the translated text and makes recommendations for cultural nuances that may not have been picked up in the back-translation reconciliation. Finally, you can expect a new translation to go through three or four minor iterations over time until you have an accurate translation both in language and cultural nuance. The degree to which the content of an assessment needs to be modified culturally is based on the type of assessment.
In the case of the LEA360, because we are not promoting an idealized model of leadership but rather a descriptive model of behavior, content adjustment for culture has not been needed because cultural variation is expressed in the pattern of usage across the 22 behaviors. MRG has been doing cultural research globally for the last 30 years and has a number of studies documenting the unique patterns of leadership by country.
Download Article 1K Club
