In my first installment of this three-part article, I discussed how the training and early job experiences of Warner Burke and myself influenced our thinking and therefore how we approached writing this book.
Warner was trained as a Lewinian psychologist, and my early work experience with David McClelland around motives and competencies played an important role in each of our philosophies.
This article is a continuation of some of our thinking. Warner and I were both impressed with the work of Mike Lombardo and Bob Eichinger, part of which was done through the Center for Creative Leadership and another portion, under their company, Lominger Ltd. Inc.
The real contribution the pair made in the area of leader development was focusing development efforts to more clearly match development needs of the person with development opportunities of jobs and assignments. They whetted people’s appetites that we were finally going to solve some of the thorny issues that prevented organizations from developing or weeding out candidates before they got bigger jobs.
Learning agility was the one big obstacle. Bob and Mike danced around the issue and described it as “learning on the fly”. I don’t know about you, but I still haven’t figured how to convey to a manager how one does that.
A test of sorts was created, but it described several different forms of learning agility. Based on what he saw – and didn’t see – in that “test”, Warner struck out on his own to create a measure for learning agility.
In Warner’s mind, learning agility is a singular concept, not something that has different forms. He and his doctoral students went to the literature on learning agility; that work became Chapter 2 in the book. Warner and his team of doctoral students began creating different test items and performed various research activities to show, statistically, that what he was measuring was, in fact, learning agility.
Four years later, Warner ended up with 38 questions that measure learning agility. While every human resource person is not expected to have a graduate-level capability in statistics, you do need a colleague who can help you ask the right questions or understand from a technical report that the test you are looking at meets professional standards.
All tests are not the same; they do not all meet the same professional standards.
Talent management is attracting more sophisticated practitioners with backgrounds in industrial/organizational psychology. When they look at a new test, their first question is, “Can I see the Technical Report?” Anyone can ask that question, but the question should be asked of the test developer.
Was a content validity study done? This will confirm for you that the test is measuring the concept (learning agility) you want to have measured.
There should be a test of reliability performed on the questionnaire. This will tell you at the test-item level that each question is contributing to (in this case) the measurement of learning agility. This test of reliability will also reveal internal consistency among the test questions.
The third assessment a reputable test should pass is one of construct validity. In the case of the Burke Learning Agility Inventory™ (Burke LAI™), there are measurements of nine different dimensions. Checking for construct validity will demonstrate that the test is measuring nine related yet independent factors.
If a test has successfully demonstrated these three measures of validity and reliability, then it is a solidly constructed and researched test.
A still higher standard for a test is a criterion validity test. In this type of assessment, you are independently measuring scores on the test and performance on the job. If a test is a good predictor of performance, then there should be a strong correlation between a higher score on the test and a person receiving a rating as a better performer. The Burke LAI passed this test, as well.
I know that, for non-researchers, this is likely a dry and boring topic. However, it is also critical to you being able to look senior management in the eye and tell them unequivocally that the test being used is a good, solid test and a strong predictor of learning agility.
This becomes especially important in instances when a senior manager that many other managers think “walks on water” receives a low score on the Burke LAI. There are some organizations that just throw out the test, assuming the results are obviously wrong. But if you have a well-researched test, you can push back and suggest the test may actually be accurate, and a deeper look may be needed… or perhaps this senior manager could be placed in an assignment or role to demonstrate whatever area in which the test indicated he is not strong.
Some learning will likely come from that, since it would allow a “test” of this person to see if he/she needs strengthening, and it can be done before being placed in the big job.
If you get nothing else from this article, please remember that all tests are not created equal. Warner Burke and I wrote a book about learning agility. He is the researcher and I am the practitioner.
As a practitioner, I need to know with a high degree of confidence that the test I am using is a good test. I can work my way through a technical report, but I also reach out to the expert researchers to confirm what I think I see.
Warner and I felt that any book on learning agility needed to stand on a strong base of good research.
David Hoff is the chief operation officer and executive vice president for leadership development at EASI•Consult®. EASI•Consult works with Fortune 500 companies, government agencies, and mid-sized corporations to provide customized Talent Management solutions. EASI•Consult specialties include individual assessment, online employment testing, survey research, competency modeling, leadership development, executive coaching, 360-degree feedback, online structured interviews, and EEO hiring compliance. The company is a leader in the field of providing accurate information about people through professional assessment. To learn more about EASI•Consult, visit www.easiconsult.com, email ContactUs@easiconsult.com or call 800.922.EASI.