On Controlling for Family Influence on Achievement

As I review Berends’ and colleagues 2008 volume Charter School Outcomes (Lawrence Erlbaum), a key assumption of Anglo-American political theory, namely that just inequality is the result of “natural distinction” (as opposed to social distinction), undergirds the authors’ efforts to improve research methods for evaluating school choice policies.

Before addressing the political basis of this methodological project, it is important to note that the authors make the mistkae that Robert Yin suggests is all too common: research on school performance confounds schools as the proper unit of analysis with individuals; this is especially common with those obsessively turning to randomized field trials. (See Yin, R. K. (2009). Case study research: design and methods (4th ed.). Thousand Oaks, Calif.: Sage Publications.)

I think exposing their position as emanating from political theory — rather than an thoughtless imports from the natural and sciences — might prove helpful in both evaluating the book and articulating the political significance of school choice policy more generally.

Random Trials as Opportunity Science

Of note is the book’s adoption of U.S. Department of Education, and in particular the Institute of Educational Sciences, insistence on the “gold standard” of experimental design: the “random assignment of units to experimental and control or contrast conditions (2).”

“Randomized field trials” are thus adopted as the key method for studying school choice. By studying the measurable outcomes of applicants who were lotteried into an oversubscribed charter school or voucher program to those who were lotteried out and attended a traditional public school, the influence of family background can be separated out from that of the school itself (but again, this promotes confusion regarding the unit of analysis).

According to the authors, the strength of this method and other efforts such as over time measures of “value added,” is that they help “take into account the powerful influence of families” and help “establish the separate and distinct contribution of the school to a student’s achievement.”

The postulate that experimental design is equally the gold standard for the social sciences as it is for the natural sciences is taken for granted. It presents itself as a solution to a perennial problem in school evaluation research predating even the “Coleman Report”: controlling for the influence of family characteristics on school outcomes. It seems as a rational way out of that conundrum — but only if certain things are ignored or forgotten.

What is the assumption behind the presupposition that students must be separated from their historical position, their social circumstance, in order to assess the quality of their school and the degree to which they have learned what is required of them? How does this premise inform the cultural meaning of “achievement” as distinct from student learning?

What is the political significance of the fact that this kind of “controlling” for social circumstance was largely impossible under a traditional public school model where place of residence determined school assignment for all but a tiny minority of public school students?

Irrespective of the logic justifying the “controlling” for social circumstance, is not such a project irrational? Can one “control” for social circumstances? Such efforts reveal a profound distortion and patently unscientific view of social reality. Does not the entire project of “controlling” for social circumstance — which includes everything from assumptions about “ethnicity” and parental “SES” to larger understandings of religion, culture and sub-cultures of neighborhoods — itself constitute a social circumstance and a patently normative project which serves the interests of some over others?

Political Logic of Random Selection

In beginning to answer my own question posed above (“What is the political significance…”) I am not arguing with the general logic of the controlled experiment, or the statistical reality of randomization and its utility for understanding cause and effect. What I am arguing is that this fetish of random trials pushed by the IES is derived from the following notion: that schools are successful to the degree they produce students who successfully compete in the academic marketplace (the exchange of grades and test scores for places of opportunity, praise and so on). Closing the achievement gap is an official effort to contend with the overgrowth of social inequality while simultaneously violently blocking any real effort or even discussion of reducing (let alone eliminating) social inequality. This pathology stems from the long-standing assumption of American political theory that replaces class struggle with the struggle for education. (Refer to classic quotes from Horace Mann for an elaboration, or even better, see Rush Welter’s (1962) Popular education and Democratic Thought in America.)

The underlying logic of this strand of charter school research (Berends et al.) is that charters should be promoted, not because they are necessarily proven to be better, but because they create competition — not only among schools, but among teachers, as researchers document a lower average salary, yet a larger spread in annual earnings for charter as compared to traditional public school teachers. Randomization is opportunity science speak for fair competition (e.g., no “selection bias”).

This competition is key because it allows for arrangements heretofore difficult to make, like linking student achievement with teacher pay, something the authors deem of obvious value and unproblematic. Teachers are evaluated on the degree to which they help students compete (e.g., note the language and real meaning of “high flying schools”), irrespective of the background and ability of the students. Good teachers are those whose students successfully compete in academic competitions (i.e., high stakes tests). Charter schools eschew the working class politics of union and solidarity and stand as institutions more firmly on the grounds of individual merit and competition. That is to say, good teachers are those that help liberate students from their social place through academic competition (again, there are other notions of “good teacher”) just as black and poor kids are supposed to “achieve” because in this “meritocracy,” race and class aren’t factors in determining ones place in the social order — charters are to replace public schools as the means for this liberation. Those that have a different view are deemed to have a bad attitude (“low expectations”).

The logic goes like this: pointing to realities of structural inequality and the impact “going without” has on child development (and thus “achievement”) introduces bias, just as introducing lotteries eliminates it. One could of course point out that it is quite biased to set up a social system which forces some more than others to be in positions where they need to “choose a good school.” This question has of course been forced off the agenda by advocates of “change” and “innovation”.

Randomization helps create, then, as a standardized norm-reference test does, a “fair playing field” — a free market, unencumbered by the realities of ones historical location, only “merit” rules. Like academic tests, charters, the logic goes, are the engine of a meritocracy for educational institutions, and the “best and brightest” will rise to the top, but could fall any moment, like a dot.com, if they don’t continually “strive” and “achieve”.

In this way, the research continues not because it is helping to answer questions of policy makers or the public (an admission that openly appears in the book) but because it is a mechanism for instituting more forcefully that arrangement of which charters are a part. The idea that one “lotteries” into a school not only suggests an open disregard for planning for the future of youth, a willingness to gamble on their future, but also a particular notion of fair play — rich and poor are equally selectable by the dice.

This entire view is antithetical to education as a right and signals an outright rejection of the notion that society has any responsibility to its members. Yet, successful schools are those that are not able to coach kids to the top of the heap, but prepare them for full participation in social life, in solving problems, etc.

Leave A Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.