In the 1932 U.S. Supreme Court case New State Ice Co. v. Liebmann, Justice Luis Brandeis made the point that a “state may, if its citizens choose, serve as a laboratory; and try novel social and economic experiments without risk to the rest of the country”. Thanks to the federal structure of the United States, all fifty subdivisions of the country have considerable leeway in how they manage all sorts of economic, political, and social policies and institutions (though the extent of this power is a matter of perennial debate and jurisprudence.)
A recent report by the Economic Policy Institute (EPI) has validated this idea, arguing that the best way to improve America’s educational outcomes is to look not abroad, as is so often done, but within, at the many individual states, counties, and cities that have managed to attain high results.
It is a long read and dense read, I unfortunately have not the time to reproduce its most salient points with my commentary. Suffice it to say, it is well worth giving a look, especially as it raises many questions whether the international rankings that are relied upon by performers are truly as accurate, and thus informative, as many believe.
We have demonstrated that such comparisons are too simplistic and why they need to be used and interpreted with great care. This aligns with the findings of Carnoy and Rothstein (2013). A closer look at the data reveals that disadvantaged FAR students in the United States continue to make very large mathematics and reading gains compared with disadvantaged FAR students in most other countries (Germany and Poland are the exceptions), and that the opposite is true for U.S. advantaged FAR students (Table 2C). Although not quite as extreme, the gains on the TIMSS over 1999–2011 for the United States as a whole follow the same pattern—much greater gains for low FAR groups than for high FAR groups.
We have also shown that international test score interpretations based on U.S. average scores are too simplistic for another reason: The U.S. education system is actually 51 different education systems, as each of the states plus the District of Columbia constitutes its own system. (In fact, there are many more systems when we consider the thousands of school districts, many of them urban districts the size of systems in small countries, that operate quite independently even of their state education administrations.) We show that students in the states vary widely in their performance on the PISA and TIMSS and in their performance gains on the TIMSS. In some states, such as Massachusetts, Minnesota, and North Carolina, students have made large gains in mathematics performance in the 2000s, and these gains have been large for both low and high FAR groups. The gains have also been higher than in Finland, England, and Korea (Tables 5A and 5B).
The wide variation in student test performance in U.S. states (as well as among FAR groups and between subjects described earlier) suggests that U.S. policymakers should reconsider looking to East Asia and some of the European countries for lessons on improving education in lower-performing states. U.S. states that have made large gains and have achieved high performance among all FAR groups are much more likely to provide relevant policy lessons and more pertinent guidance for improving education in states that have made less progress. The contexts for education systems differ among states, but these differences are much smaller than those among lower-performing states on the one hand and, on the other hand, countries with different social, cultural, and educational histories. For example, it makes much more sense for Alabama to look to North Carolina for lessons than to Finland, Poland, or Korea. These reasons, in addition to the others explained in this paper, all illustrate why it is difficult to learn about improving U.S. schools from international test comparisons.
Indeed, there is as much variation within the U.S. as among different countries, providing us with a wealth of case studies and examples to compare, contrast, and learn from.
As a suggestive strategy for further (qualitative) policy research, we paired off neighboring and/or demographically similar states with different patterns of gains in average student performance in 8th grade math. We showed that students in Massachusetts made much larger gains after 2003 than students in neighboring Connecticut; that students in New Jersey made larger gains than students in New York after 2003; that students in Texas already started out scoring higher than students in California in 8th grade math in 1992, but still made larger gains over 1992–2013, especially after 2003; that students in North Carolina made much larger gains over 1992–2003 than students in neighboring Kentucky and Tennessee, but that students first in Kentucky and then in Tennessee caught up somewhat after 2003; and that students in Minnesota made larger gains than students in Iowa almost throughout the entire 20-year period. We argued that each of these comparison groups could provide important insights into the kinds of policies that enabled students in some states to make much larger adjusted gains in math scores than students in neighboring and/or demographically similar states.
Of course, none of this is to say that international studies and comparisons do not matter, nor that we Americans cannot learn a thing or two from other societies. But there seems to be far more attention to foreign metrics and systems than to existing and equally successful examples here in the U.S. We need all the help and ideas we can get to improve education, so why not include what has already been shown to work in our society?
What are your thoughts?