The forthcoming article “Adjusting for Confounding with Text Matching” by Margaret E. Roberts, Brandon M. Stewart and Richard A. Nielsen,  is summarized by the author(s) below. 

We propose a family of methods for conditioning on text when it confounds the relationship between treatment and an outcome.   

Does being censored once make Chinese citizens more likely to be censored online in the future?  Is there gender bias in the reception of academic scholarship?  Do jihadist preachers get more popular when they are killed in counterterrorism operations?  These are questions we have tackled in our research because we believe the answers are important to society.   

However, they are also questions that are difficult to answer.  Gold-standard experimental evidence isn’t possible on these questions because experimentation with censorship, gender bias, and terrorism is usually unethical and infeasible.  Instead, we have to use non-experimental data, but then face the challenge of confounding factors.   

To get the intuition for why non-experimental data is problematic in these settings, consider our investigation of the effects of gender on citation counts in International Relations, a subfield of Political Science.  We want to know whether the author’s perceived gender influences the number of citations the paper ultimately receives.  Understanding this would allow us to better understand the obstacles certain groups face in academia.  If we could run an experiment, we could randomly assign names to papers and then see how the effect of having a female name on a paper influenced downstream citation counts.  However, we can’t do this for ethical and logistical reasons.  What we can do is collect academic papers in International Relations and compare the citation counts of those written by men and those written by women.  But this approach is also flawed.  Women tend to write about different subject matter than men, and we wouldn’t be able to tell whether or not the differences in citation count are due to these different topics, or due to their gender, a classic case of the old adage “correlation does not equal causation.” 

Modern statistical methods offer principled ways for trying to move from correlation to causation when we believe that experimental intervention is as good as randomly assigned within units with similar confounders.  However, we couldn’t use existing techniques because in each of these examples, we were working with confounders that were measured with text data. To make progress, we developed a new family of methods that we call “text matching.” 

To detect whether there is a gender citation gap, we start with the articles written by women and then use text matching to find a comparison set of articles written by men that are very similar in terms of subject matter and approach.  Comparing the downstream citation counts of these matched comparison sets gives us a plausible estimate of the causal effect of perceived gender on citations.  Consistent with previous studies, we find large negative effects of female authorship on citation count. 

Text matching is a family of statistical tools, not a single technique.  This paper describes our favorite approach, which we have given the unwieldy name of Topical Inverse Regression Matching, or TIRM for short.  However, there are now other options to choose from.  Since we first proposed text matching in a 2015 working paper,1 other research groups have suggested alternative techniques that might be better for some research questions.2  We urge readers of this paper to also learn about the exciting developments in the growing literature. 

About the Author(s): Margaret E. Roberts is an Associate Professor, Department of Political Science at University of California, San Diego, Brandon M. Stewart is an Assistant Professor and Arthur H. Scribner Bicentennial Preceptor, Department of Sociology at Princeton University and Richard A. Nielsen is Associate Professor, Department of Political Science at Massachusetts Institute for Technology. Their research “Adjusting for Confounding with Text Matching” is now available in Early View and will appear in a forthcoming issue of the American Journal of Political Science. 

The forthcoming article “Judicial Reshuffles and Women Justices in Latin America” by Ignacio Arana Araya, Melanie M. Hughes and Aníbal Pérez‐Liñán is summarized by the author(s) below. 

Can weak institutions facilitate access of excluded groups into positions of power? Weak institutions arguably have negative consequences for developing countries, but a research stream has linked institutional disruptions to women’s advancement to positions of power. We inquire into the relationship between de-institutionalization and gender equality by analyzing political reshuffles of the judiciary in Latin America.  

Judicial reshuffles are episodes in which a majority of justices in the Supreme Court or Constitutional Tribunal are forced to leave office. Latin America presents wide variation regarding judicial purges, and the number of women justices has increased dramatically, going up from 3% of all justices in high courts in 1980 to 19% in 2010. 

Gender equality can improve the quality of legal decisions by increasing the diversity of ideas, values, and legal styles. Women jurists are more likely to make decisions that promote gender equality, and may decide differently across cases. Their presence also brings institutional legitimacy to courts. At the same time, women justices can only exercise power if courts are independent. 

We argue that institutional disruptions facilitate the appointment of women justices, but only when left parties control the nomination process. Leftist governments have stronger motivations to diversify the courts than other administrations at least for two reasons. First, judicial reshuffles give leftist parties an opportunity to prove their ideological commitment to gender equality. Second, left governments may appoint women justices for strategic reasons. The appointment of women allows leftist governments to bring legitimacy to a power grab, enables them to control the narrative, and diverts attention from their efforts to limit the independence of the judiciary.  

We test this argument using difference-in-differences and dynamic panel models for Supreme Courts and Constitutional Tribunals in 18 Latin American countries between 1961 and 2014. The analysis offers support for our hypothesis, but gains in gender diversification are modest in size and hard to sustain over time. Political reshuffles may produce short-term advances for women in the judiciary, but do not represent a path to substantive progress in gender equality. 

Our findings connect to four bodies of literature. First, by showing that institutional disruptions may benefit women jurists, we have extended prior arguments about the gendered effects of institutional disruption. Second, our study connects to research that has assessed the role of leftist political parties in promoting women. Third, we contribute to the scholarship that has unveiled how anti-democratic forces nominally advance women’s rights and representation to distract from democratic failures. Finally, our study relates to the nascent literature on weak institutions showing that, under certain conditions, they facilitate the entrance of new players into power. However, institutional weakness may also undermine these gains over the long run. In the absence of deeper social transformations, women’s gains in the higher courts may be limited and hard to sustain. Therefore, there is still work to do to identify the circumstances in which the advancement of women justices reflects a progression that is likely to hold over time. 

About the Author(s): Ignacio Arana Araya is an Assistant Teaching Professor in the Institute for Politics and Strategy at Carnegie Mellon University, Melanie M. Hughes is a Professor, Department of Sociology, University of Pittsburgh and Aníbal Pérez‐Liñán is a Professor of Political Science and Global Affairs, University of Notre Dame. Their research “Judicial Reshuffles and Women Justices in Latin America” is now available in Early View and will appear in a forthcoming issue of the American Journal of Political Science. 

The forthcoming article “Media Influence on Vote Choices: Unemployment News and Incumbents’ Electoral Prospects” by Marcel Garz and Gregory J. Martin is summarized by the author(s) below. 

Our paper investigates a subtle but important form of media influence on electoral politics. We ask not if media coverage influences what people think, but if it influences what they think about. There are simply too many possible stories happening at any given time for news sources to cover them all, and editors must therefore make choices about which to emphasize and which to de-emphasize in their coverage. Political scientists have long recognized the possibility that this “agenda setting” function of media might have important electoral consequences. A corruption scandal, for example, can’t hurt an incumbent if media outlets opt not to cover it. 

The difficulty in testing for this form of influence is that editorial choices respond to and anticipate reader interest. Imagine that media coverage in some campaign focuses on candidates’ personality traits and ignores their policy proposals. Are news outlets shaping the public’s choices, inducing citizens to focus on personality differences over policy? Or are they simply anticipating that voters care more about the personality dimension, and providing voters with the type of information for which they have more interest? 

Our solution to this challenge focuses on a category of news for which new information arrives at regular intervals: the US government’s monthly unemployment reports. We show that media attention to these reports is higher when unemployment numbers cross a round-number threshold, or “milestone.” Milestone events are an example of a phenomenon that psychologists call left-digit bias. A report that the unemployment rate has just hit 6 percent tends to get much more media coverage than a report showing the rate rose to 5.9 percent, even though the underlying conditions in the economy are nearly the same in both cases. 

 This difference in coverage allows us to measure the effect of media attention to unemployment on voting, holding constant the actual economic conditions on the ground. We look at gubernatorial elections, and compare the election results of incumbent governors in states with very similar economic conditions at election time, but where one state has just experienced an unemployment milestone and one has not. We show that good milestones – where unemployment dips below a round number threshold – help incumbents, and bad milestones – where unemployment rises above a round number – hurt them. The effects are large in both cases, but they are strongly asymmetric: bad milestones hurt about twice as much as good milestones help. 

Our results demonstrate that media outlets collectively have substantial influence over election outcomes, deriving from their ability to emphasize or de-emphasize economic performance in news coverage. Citizens can only hold incumbents accountable for their economic performance if they are aware of it. Milestone events dramatically increase voters’ sensitivity to incumbent governors’ economic records, and thereby reveal an otherwise hidden form of media influence. 

 About the Author(s): Marcel Garz is Assistant Professor of Economics, Jönköping International Business School and Gregory J. Martin is Assistant Professor of Political Economy, Stanford Graduate School of Business. Their research “Media Influence on Vote Choices: Unemployment News and Incumbents’ Electoral Prospects” is now available in Early View and will appear in a forthcoming issue of the American Journal of Political Science.