Joshua B. Fischman (University of Virginia School of Law) & David S. Law (Washington University, St. Louis – School of Law; Washington University, St. Louis – Department of Political Science) have posted What Is Judicial Ideology, and How Should We Measure it? on SSRN. Here is the abstract:
Scholars have for decades sought to measure judicial ideology and its impact on judicial behavior. However, they have not always taken care to identify the phenomenon that they are measuring in clear and convincing terms. Nor have they made a habit of evaluating different measurement approaches for their suitability to the task at hand. In this Article, we attempt to remedy the recurring weaknesses of the existing empirical literature in three ways. First, we identify the major conceptual and methodological obstacles to the empirical measurement of judicial ideology. Second, we compare the performance of several popular approaches to the measurement of judicial ideology. Third, we propose measurement approaches of our own that, for certain applications, offer significant improvements over some of the most popular alternatives.
At a conceptual level, empirical scholars confront the difficulty of defining ideology and the related challenge of disentangling ideological and nonideological preferences. As a methodological matter, scholars face the challenge that ideology is an intangible phenomenon that cannot be directly observed. From this fact stems the further problem of observational equivalence: the observable behavior of judges is open to multiple interpretations and may be attributable to both ideological and nonideological motivations. A further complication is the possibility that judicial ideology may be multidimensional across different areas of law.
It is crucial that scholars give explicit attention to the strengths and weaknesses of different measurement approaches, and their suitability to different types of research questions. To that end, we identify and contrast three broad families of approaches – namely, those that rely upon some easily observable proxy for a judge’s ideology, those that require assessment of a judge’s actual behavior in a particular context, and those that involve transplanting ideology estimates from one context to another. We also discuss issues surrounding the coding of cases for empirical analysis, with particular attention to the choice between unidimensional, multidimensional, and agnostic coding schemes.
Finally, we perform head-to-head evaluations of several popular measures of judicial ideology to see how well they predict voting behavior on the federal courts of appeals and the Supreme Court. For the courts of appeals, we test two widely used proxy measures – namely, party of appointing president and the judicial common space scores – against a novel approach that entails analyzing past voting behavior to estimate not only each judge’s ideology, but also the impact of collegiality concerns and dissent costs on judicial voting. As between the two proxy measures, we find that the common space scores perform only slightly better than party of appointing president at predicting actual votes. Moreover, both proxy measures yield almost identical estimates of the impact of ideology on actual judicial voting. By contrast, the novel approach that we employ boasts greater predictive power than either of the proxy measures, and it further reveals that ideology has a much greater impact on judicial voting than could be discovered using either of the proxy measures.
For the Supreme Court, we test party of appointing president, the Martin-Quinn scores, the Segal-Cover scores, and our own measure of judicial ideology computed from the voting behavior of the justices. Although the Martin-Quinn scores and their ilk yield the occasional counterintuitive prediction, their performance at predicting judicial votes is comparable on the whole to that of our customized measure and noticeably superior to that of the Segal-Cover scores, which in turn outperform party of appointing president by a substantial margin.
And from the paper:
Even in the relatively specific
context of empirical research on judicial behavior, there is an array of different phenomena that
people may have in mind when using the term. It might be considered “ideological,” for
example, for judges to seek to advance a particular policy outcome–a world characterized by less
environmental degradation, or of less regulation, or of greater or lesser levels of immigration.
Alternatively, the term “ideological” could describe a tendency to favor or disfavor certain types
of parties–criminal defendants, police officers, corporations, members of ethnic or religious
minorities, the disabled, and so forth. Indeed, the breadth of the concept of “ideology” even
makes it possible to speak of the existence of both political ideology and legal ideology. To say
that a certain type of judicial behavior is “ideological” need not mean that it is ideological in a
political sense: one might, for example, characterize adjudication that relies heavily upon logical
deduction from formal rules as narrowly “legal,” whereas adjudication driven by ideas about the
role of law and the responsibilities of judges might by contrast be characterized as both “legal”
and “ideological” in character.The existing literature, however, tends to equate judicial ideology with political orientation,
and to distinguish sharply between political and ideological motivations, on the one hand, and
legal motivations, on the other. The idea that “politics” and “ideology” are synonymous, and
that “law” and “ideology” are opposites, forms the basis of the theoretical framework articulated
by Professors Segal and Spaeth, who assign explanations of judicial decisionmaking to one of
three models: the “legal model,” the “attitudinal model,” and the “strategic” or “rational choice
model”.15 The “attitudinal model,” of which they are the chief proponents,16 equates “the
ideological attitudes and values of the justices” with their political leanings.17 As they use the
term, ideology refers simply to the political leanings of the justices.18 What they term the “legal
model,” by contrast, depicts judicial decisionmaking as the product of the interplay of law and
fact.19This distinction between the “legal” and the “political” or “ideological” is deeply
problematic. To take a concrete example, Justice Scalia has articulated at length a set of reasons
for favoring adherence to the original meaning of the text as a method of statutory and
constitutional interpretation.20 Most legal and political observers alike would conclude that it is
appropriate, if not desirable, for him to adopt an interpretive method, or “judicial philosophy,” in
light of his responsibilities as a judge. Yet that is not to say that his choice of originalism is
neither “ideological” nor “political” in character. First, it is possible that he has given an
incomplete account of his reasons for choosing a particular judicial philosophy. Might it be that
his choice is instrumental and motivated in part by a belief that originalism tends to yield
conservative results?21 Second, even if we assume that he has given a complete account of his
reasons for employing originalism, the fact that his choice of originalism is principled does not
necessarily remove it from the realm of ideology: there is nothing contradictory about the notion
of a principled ideologue. Third, the principles underlying his commitment to originalism might
themselves be considered both “political” and “ideological.” Why is Justice Scalia’s conception
of the proper role of judges in a democracy, and of the demands of the rule of law, not a
“political” view? If he decides cases on the basis of his commitment to originalism, are we to
say that his behavior is legally motivated, ideologically motivated, or both?
I found this paper enormously helpful.
Highly recommended. Download it while its hot!