Donald Campbell was one of the leading psychologists of the second half of the twentieth century. His was a time of optimism for planners—there was a belief that the power of technology could be brought to bear on many of the world’s ills. And indeed they were, often with positive effects. As a result of central planning, more people receive water, more places are electrified, more children educated, and more diseases eradicated. All good goals with which Campbell would not quibble.
But Campbell noticed something else to, the emergence of “corruption pressures,” based on the general principle that is now known as “Campbell’s Law.”
The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.
In different ways, both Michael Scroggins and Max Holland have pointed at this basic problem in their recent blogs critiquing intelligence tests ranging from the standard tests, to the SAT and college entrance exams. Such intelligence tests are indeed used to divide up the spoils of placement spots at elite schools, and not surprisingly, ambitious parents seek to corrupt it by means both fair or not. But for use in general analysis like that done by the evolutionary psychologists, the consequences are that their data source over time is corrupted. The cheating scandals associated the No Child Left Behind Act are a byproduct of Campbell’s Law. So is the fact that the SAT exam was recently cancelled in South Korea due to widespread cheating. However most of the corruption does not come from cheating. It also comes from the fact that such standardized tests are routinely gamed by testing companies which guarantee 100 extra points on the SAT through $1000 prep courses (I used one of these classes for my daughter—it worked!).
For what it is worth, tests like the internationally administered National Association of Educational Progress (NAEP) which do not have consequences attached to them are much less likely to be gamed. They do not have consequences for funding, admissions, etc., attached to them because they sample across broad areas, and report results on large geographical areas rather than individuals.
Which brings up the BGI Cognitive Genetics Gene Trait Association Study of Intelligence that Michael Scroggins wrote about, and which Dr. Steve Hsu is promoting as a member of the Core Team of BGI. The Chinese company is seeking people with “high cognitive abilities,” as defined by high scores on the SAT and other standardized tests, or PhDs in a limited number of fields (e.g. physics, computer science, electrical engineering) from “top” US universities.
The implicit assumption is that these people must have DNA which makes them higher functioning than the rest of us. There are a number of flaws with this approach, starting with those pointed to by Donald Campbell—particularly the fact that the measures they are using long ago lost the validity and reliability due to corruption pressures. There is also the problem that Michael, Max, and I have been hammering home here at Ethnography.com, which is that “intelligence” is always culturally defined, typically by those who have the power to define people like themselves as, well, “intelligent.” (Perhaps this is why BGI does not want people with PhD degrees in fields they have not studied, or from universities outside the US–this is who they are).
There are of course other reasons why BGI are off on a fool’s errand, some of which is described in Chapter 5 of my recent (2012) book Schooling, Childhood, and Bureaucracy: Bureaucratizing the Child. Chapter 5 is called “The Sorting Function of Schools: Institutionalized Privilege and Why Harvard is a Social Problem for Both the Middle Class and Public School 65 in the Bronx.” For that matter Chapter 8 “Seeing Like a State: Efficiency, Calculability, Predictivity, Control Testing Regimes, and School Administration” is also relevant. (Sorry the book is still only out in hardcover at $90, and Kindle for $72—check your library for a copy, or wait for the paperback version). To summarize the findings in my book: Success on tests are inevitably associated with reproducing the status quo, whatever status quo the elites of the day might be promoting.
As for Campbell’s Law, I hope that the people organizing such projects as the Gene Trait Association Study of Intelligence read Donald Campbell’s article carefully, even if he is not an electrical engineer or physicist with a PhD from a top US University, or an 800 on the math portion of the SAT.
Tony Waters is czar and editor of Ethnography.com. He came to us from the Sociology department at California State University at Chico where he has been a professor since 1996. In 2016 though he suddenly found himself with a new gig at Payap University in northern Thailand where he is on the faculty of the Peace Studies Department. He has also been a guest professor in Germany, and Tanzania. In the past, his main interests have been international development and refugees in Thailand, Tanzania, and California. This reflects a former career in the Peace Corps (Thailand), and refugee camps (Thailand and Tanzania). His books include: Crime and Immigrant Youth (1999), Bureaucratizing the Good Samaritan (2001), The Persistence of Subsistence Agriculture: Life Beneath of the Marketplace (2007), When Killing is a Crime (2007), and Schooling, Bureaucracy, and Childhood: Bureaucratizing the Child (2012). His hobby is trying to learn strange languages–and the mistakes that that implies. Tony is a prolific academic, you can read more of his work at academia.edu.or purchase one (or more!) of his books from Amazon.com.