Measuring the Many Dimensions of Disagreement on the Supreme Court

By Benjamin E. Lauderdale and Tom S. Clark

Our article, “Scaling Politically Meaningful Dimensions Using Texts and Votes”, is both about the U.S. Supreme Court and about the methodology of measurement.  One of the many goals of political science as a field is improving measurement and description of political phenomena.  Over the past decades, we have developed methods for quantitatively summarizing the large data sets that are created by politics.  One of the earliest areas of development in this area was the analysis of roll-call votes in the U.S. Congress.  In a series of papers, and later books, Keith Poole and Howard Rosenthal developed methods for summarizing all the votes that all the legislators in Congress record (e.g. Poole & Rosenthal 1985, Poole & Rosenthal 1997).  These summaries, usually referred to as “ideal points” in the political science literature, characterize the voting tendencies of legislators in terms of relative positions in one or two political dimensions, one of which can usually be described as liberal-conservative or left-right.  Being able to summarize how members of Congress vote in one or two numbers has enabled an enormous amount of subsequent research, answering questions like what makes members of Congress vote the ways they do, whether there is a close representative link between constituent public opinion and individual representatives’ voting behavior, and many others.

More recently, there has been an explosion of research applying similar principles to measure a variety of concepts in a variety of political institutions around the world.  Just looking at the U.S. Congress, there are now estimates of the relative ideology/positions of representatives based on who gives them money (Bonica 2014), on who follows them on Twitter (Barbera 2014), and on what they say on the floor of the chamber (Lauderdale & Herzog 2015).  This research is increasingly showing up in newspaper articles and blog posts that help put the positions of elected representatives in clearer context, enabling readers to better understand what elected officials and other political actors are doing and who supports them.  Following early work by Andrew Martin and Kevin Quinn (2002), there is also a growing literature on measuring the general positions of judges, particularly on the U.S. Supreme Court, which we contribute to in our article.

One of the limitations of much of this research is that it describes the behavior of judges or legislators using a single liberal-conservative dimension, with some studies also estimating a second dimension.  This is a big improvement on no measurement at all, but we do know that judges and legislators vary in their voting behavior across different policy domains for various reasons.  If we want to understand their choices better, it would be helpful to measure these differences in a systematic way.  Unfortunately, the methods that have been typically used do not work well at recovering estimates across many dimensions, for reasons we describe in our paper.  Our contribution is to demonstrate one way to combine multiple sources of information in order to make highly multidimensional measurement possible.  Our model combines this kind of “ideal point estimation” from political science with methods for “topic modeling” developed originally in computer science.  This is an “unsupervised” measurement model: all that one needs to provide are the votes recorded by a set of political actors on a series of decisions plus a text describing each decision using a relatively uniform style.

In our application to the U.S. Supreme Court, we combine the judicial votes that each justice makes in each case with the texts of the opinions from that case.  One of the major arguments we make in the paper is that each of these types of information provide distinct and complementary information.  The pattern of votes provide a signal of the relative positions of the justices whereas the relative usage of different terminology in the opinions provide a signal of which mix of legal issues were raised by that case.  For a single case, all of this could be determined by simply reading the decisions, but with many thousands of decisions over many decades, quantitative measurement allows us to summarize broad patterns in the history of the U.S. Supreme Court.  We are able to construct estimates of the positions of justices that vary across the major issues repeatedly considered by the Court.  These issues are automatically labelled by the method, using the three terms that provide the most distinctive signal that a given issue is being discussed.  So to give four examples, we find issues labelled by the following sets of terms: “prison, inmates, parole”, “search, fourth, warrant”, “political, election, party”, and “title, vii, employment”.  These each correspond to major issues that the Court has returned to many times, and in which justices might vary in the extent to which they are “liberal” or “conservative”.

Our research shows the relative liberal-conservative positions of the justices serving between 1946 and 2005, across 24 areas of law that we identify from the opinion texts.  We show how some of the issues have become more or less common in the Court’s docket over time.  We are also able to say something about how the broad history of the Court’s liberal-conservative balance varied by issue.  Both previous quantitative estimates of judicial preferences and qualitative historical accounts have argued that the Court became increasingly liberal during the Warren Court before turning in a conservative direction since, primarily due to justice replacement.  This general pattern is reflected in our estimates, but that overall trend has interesting exceptions where we see little or even conservative change during the same period. These exceptions are associated with economic regulation: issues characterized by the frequent mention of terms such as “antitrust, price, securities”, “union, labor, board” and “commission, rates, gas.”  We argue that this arises because the justices appointed by President Franklin Roosevelt, who served until the 1950s, were particularly liberal on topics of economic regulation (they were chosen for that very reason) but were conservative on the issues that are central to the liberal reputation of the Warren Court.  As a consequence, their replacements during the Warren Court shifted the Court to the right in economic domains, even as it shifted the court left on issues relating to civil rights and civil liberties.

About the Authors: Benjamin E. Lauderdale is an Associate Professor in the Department of Methodology at the London School of Economics and Political Science and Tom S. Clark is the Asa Griggs Candler Professor of Political Science at Emory University. Their article, “Scaling Politically Meaningful Dimensions Using Texts and Votes” appeared in the July 2014 issue of the American Journal of Political Science.

 

References:

  • Bonica, Adam. 2014. “Mapping the Ideological Marketplace” American Journal of Political Science 58:367-386.
  • Barbera, Pablo. 2015. “Birds of the Same Feather Tweet Together. Bayesian Ideal Point Estimation Using Twitter Data.” Political Analysis 23:76-91.
  • Lauderdale, Benjamin E. and Alexander Herzog. 2015. “Measuring Political Positions from Legislative Speech” Working Paper.
  • Martin, Andrew D. and Kevin M. Quinn. 2002. “Dynamic Ideal Point Estimation via Markov Chain Monte Carlo for the U.S. Supreme Court, 1953-1999.” Political Analysis 10:134–153.
  • Poole, Keith T. and Howard Rosenthal. 1985. “A Spatial Model for Legislative Roll Call Analysis.” American Journal of Political Science 29(2):357–384.
  • Poole, Keith T. and Howard Rosenthal. 1997. Congress: A Political-Economic History of Roll Call Voting. Oxford University Press.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

The Editor of the AJPS is at Michigan State University and the Editorial Office is supported by
the Michigan State University Department of Political Science and the School of Social Sciences.

  Michigan State University 
%d bloggers like this: