The Future of Standardized Testing?
What can we lose through too much automation?
Posted Sep 22, 2015
In science fiction novel Ender’s Game, the talent and personality of each battle school student is adaptively assessed by the Mind Game, tailored to their interests and individuality. The game was used, in part, to select Ender Wiggin as the young commander who would save the world. And although computer adaptive testing may not have yet reached the stage of the borderline-sentient program envisioned by author Orson Scott Card, these ideas have been with us for some time.
The educational psychologist Richard Snow envisioned an automated testing program he named Quintilian—after the famous Roman educator—that assessed a student’s strengths and weaknesses and then matched a curriculum to their individual profile to help them learn and grow at their own pace. Such a program today may not be too far from reality. Sal Khan, founder of the Khan Academy, told me: “Yeah I think we are doing it in a very primitive way right now…What you’re going to see on Khan Academy…is much more rigorous deep assessment diagnostics on the site. This will really help people fine tune the activities that are going to be the most appropriate for them at any given time.” And testing isn’t just being used in education or college admissions. As Shaila Dewan discussed in the New York Times, businesses are using intelligence and personality testing as mechanisms to select candidates. There is a great deal of evidence supporting the predictive validity of such tests, but also because it is a cost effective and efficient way to filter through applications.
As our world increasingly moves toward automation, where human abilities are augmented by the power of machines—it’s worth considering what we might lose by relying too heavily on efficient methods of selecting people for positions.
William Deresiewicz makes a compelling case in Excellent Sheep that the educational system as it exists today encourages students to only value a certain kind of success: “Growing up elite means learning to value yourself in terms of the measures of success that mark your progress into and through the elite: the grades, the scores, the trophies.” And Adam Grant argued that we should overhaul our college admissions system, because “colleges learn a great deal about [student] competence from grades and test scores, but remain in the dark about their creativity and character.” Because standardized tests may not measure some intangible qualities such as aspects of creativity or character, or more generally the holistic qualities and nuanced talents of a person, there will always be some talented people who might miss being identified. For example, Scott Barry Kaufman in Ungifted tells his personal story of not performing well on standardized tests, being placed in special education, and through the grace of a teacher finally being identified as talented.
With the rise of computer adaptive testing programs and online learning, testing is becoming easier than ever to implement and could become an even more highly desirable filtering mechanism: objective, fair, efficient, cost-effective, and unbiased by the frailty of human judgment. On the other hand, given the strong resistance many have to the idea of too much testing, perhaps we will move toward a future where a more nuanced human approach is valued, where qualities other than those assessed by traditional tests are provided a fair and complete hearing.
And yet, no matter what tools we wish to use, whether standardized tests or individual human judgments, there will likely always be some kind of system—for better or worse—that sorts people. Even if we remove one system, another sorting system—perhaps even less fair or just—will likely take its place. As psychologist Leona Tyler wrote many years ago:
In our haste to abolish the unjust and the obsolete, we cannot afford to ignore the psychological realities that generated such systems in the first place. There are highly significant psychological differences among individuals, and the soundness of our social institutions depends upon how successfully we take them into account…A complex society cannot regard its members as identical interchangeable parts of a social machine. Its complex functioning depends upon the contributions of individuals specializing along different lines, equipped for carrying out different specialized tasks. For this reason we must not be content with any system of universal education that provides identical treatment for all pupils. We must look for ways of diversifying education to make it fit the diverse individuals whose talents should be developed and utilized.
The future may hold some intriguing and perhaps scary possibilities. Maybe a battery of personality and intelligence tests could be used in a computer adaptive setting like that envisioned by Card in Ender’s Game. And this information might be paired with data from an individual’s social media use that indirectly reveals intelligence, personality, and even employability. Maybe brain scans will be used along with standardized tests, or even in place of them.
Or maybe we will move away from automated testing and increasingly use the more holistic (yet less efficient) method of getting to know the nuanced talents of each individual through more personal evaluations such as interviews and portfolios. As our measurement technologies advance, they will likely be able to capture in greater detail the diverse patterns of talents that exist in society. But we should remember that they will always be imperfect tools, and some talents may not be captured. It is how these tools are utilized in combination with human judgment that will influence the future structure and values of our society. Our technology, after all, is usually only as good as how we choose to use it.
© 2015 by Jonathan Wai