The forthcoming article “Can Close Election Regression Discontinuity Designs Identify Effects of Winning Politician Characteristics?” by John Marshall is summarized by the author below.
More than one hundred published articles now use what I call politician characteristic regression discontinuity (PCRD) designs. These designs compare outcomes of interest across districts that narrowly elected politicians who differ in an observable characteristic X, such as gender, incumbency, party affiliation, or partisan alignment with other levels of government, from the candidate they defeated. PCRD designs are appealing and important because of their potential to investigate whether the characteristics of politicians matter for government responsiveness, electoral success, or citizen participation and welfare.
This is a non-standard application of the regression discontinuity (RD) design. RD designs typically compare units above and below a threshold; indeed, close elections can identify an average individual-level effect of getting elected by comparing politicians who narrowly win to those who narrowly lose. In contrast, PCRD designs compare districts narrowly won by one type of politician to districts narrowly won by a different type. By connecting the standard RD to this non-standard application, this article seeks to clarify what effects PCRD designs can and cannot identify.
A key conceptual decision for researchers using PCRD designs is to define what is and is not part of their treatment of interest, and thus their target estimand. I consider two leading estimands, which are both local to districts with close elections between candidates with and without the characteristic of interest. Akin to conjoint experiments, the first is the average effect of electing a politician with (binary or binarized) characteristic X, holding all their other politician characteristics Z constant (usually averaging over the distribution of Z in close elections between candidates with and without characteristic X). Rather than isolate the effect of X, the second estimand is the average effect of the bundle of characteristics that come with possessing X relative to not possessing X. Applied researchers consider both estimands, although many papers should state their estimand more explicitly.
I focus mostly on the challenges of identifying the effect of a particular X, using university education as another example in this blog. While separating the effect of a leader’s education from other characteristics may be theoretically appealing, PCRD designs yield biased estimates under the relatively weak standard RD assumptions for two reasons. The obvious reason is that politicians’ characteristics are usually correlated. Consequently, university educated politicians naturally differ from less educated politicians in other ways, like their policy preferences. This article highlights a subtler reason: PCRD designs introduce a form of post-treatment bias, even if university education were randomly assigned, by conditioning on a close race. For example, if electorates prefer more educated politicians, then the average university educated politician who narrowly wins must be less desirable to voters in other ways – what I call compensating differentials, such as advocating unpopular policies. The direction of bias will depend on the application, although PCRD designs may understate the effect of X when the candidate characteristics that appeal to voters affect district outcomes similarly.
My analysis has several implications for using PCRDs to isolate the effect of a single politician characteristic. First, the bias due to compensating differentials only disappears where the potentially large set of such compensating differentials do not influence a candidate’s vote share or do not affect the outcome of interest. Such assumptions, which are far stronger than the usual RD continuity assumption, are rarely empirically plausible. Averting the natural correlations between characteristics requires further assumptions, which may rely on manipulations that are not possible in isolation.
Second, if neither assumption holds, the need for compensating differentials implies that we should expect to find imbalances across other politician characteristics. While close elections still guarantee continuity in district-level covariates, balance tests for politician-level characteristics now serve to illuminate the nature of the compound treatment. Instead of implying an PCRD design is valid, failing to detect differences between observable politician characteristics suggests that there are differences in unobserved characteristics or between combinations of characteristics.
The second estimand, which is carefully articulated and exemplified by Hall (2015), embraces a bundled treatment. Rather than attempt to distinguish the effects of X from Z, this approach obviates the problem of politician-level confounding by subsuming all (observed and unobserved) politician characteristics into a broad conception of treatment. My identification result demonstrates that, under the standard RD assumptions, PCRD designs capture the effect of an average bundle of characteristics for politicians defined by X (weighted by the correlation between X and each Z) relative to an analogous average bundle for politicians who do not possess X, among the set of politicians in close races.
Beyond the appeal of requiring weaker identifying assumptions, the value of an estimand encompassing many characteristics depends on the application. The broad treatment is likely to be most useful when the analyst cares about bundles of characteristics that approximate available policy choices, such as whether political parties should recruit university educated candidates. Because some characteristics in the bundle are unobserved and each characteristic’s contribution to the overall effect is not identified, the bundled approach is less useful for testing theories which seek to estimate effects of a particular politician characteristic holding other characteristics equal, such as whether rising education has improved governance.
Ultimately, PCRD designs work differently from standard RD designs. To identify effects of specific well-defined treatment in the way that standard RD designs usually do, PCRD must impose strong additional assumptions. To maintain the relatively weak standard RD assumptions, researchers must embrace a bundled treatment that may be policy-relevant but less useful for accumulating evidence about theoretical mechanisms and informing high-dimensional policy decisions.
About the Author: John Marshall is an Associate Professor of Political Science at Columbia University. Their research “Can Close Election Regression Discontinuity Designs Identify Effects of Winning Politician Characteristics?” is now available in Early View and will appear in a forthcoming issue of the American Journal of Political Science.
Speak Your Mind