Saying versus doing: a new donation method for measuring ideal points

Abstract

Scaling methods pioneered by Poole and Rosenthal (Am J Polit Sci 29(2):357–384, 1985) redefined how scholars think about and estimate the ideologies of representatives seated in the US Congress. Those methods also have been used to estimate citizens’ ideologies. Whereas studies evaluating Congress typically use a behavioral measure, roll call votes, to estimate where representatives stand on the left-right ideological spectrum, those of the public most often have relied on survey data of stated, rather than revealed, preferences. However, measures of individuals’ preferences and, accordingly, estimates of their ideal points, may differ in important ways based on how preferences are elicited. In this paper, we elicit the same individuals’ preferences on the same 10 issues using two different methods: standard survey responses measured on a Likert scale and a donation exercise wherein individuals are forced to divide $1.50 between interest groups with diametrically opposed policy preferences. Importantly, expressing extreme views is costless under the former, but not the latter, method. We find that the type of elicitation method used is a significant predictor of individuals’ ideal points, and that the elicitation effect is driven primarily by Democratic respondents. Under the donation method, the ideal points of Democrats in the aggregate shift left, particularly for those Democrats who are politically engaged. In contrast, wealthy Democrats’ ideal points shift to the right. We also document effects for Republicans and Independents and find that overall polarization is similar under both elicitation methods. We conclude with a discussion of our results, and the consequences and tradeoffs of each elicitation method.

Appendices

Appendix 1: additional figures

Fig. 8

Issue importance, Likert elicitation. Notes. Options: not important at all (1), not too important (2), somewhat important (3), very important (4), or extremely important (5)

Fig. 9

Prior knowledge of interest groups, donation elicitation. Notes. Question: Did you know of this group prior to participating in this study? Options: Yes or No

Fig. 10

Direction of ideal point change under donation elicitation (as opposed to Likert)

Fig. 11

Democrats only: predictors of individuals’ ideal point estimates

Fig. 12

Republicans only: predictors of individuals’ ideal point estimates

Fig. 13

Independents only: predictors of individuals’ ideal point estimates

Fig. 14

Predictors of absolute value differences in estimates, Likert versus donation elicitation

Fig. 15

Independents only: predictors of raw differences in ideal points

Fig. 16

Independents only: predictors of absolute value differences in estimates, Likert versus donation elicitation

Appendix 2: Estimation strategy

Bayesian multinomial IRT as presented in Hill and Tausanovitch (2015).

The authors assume a one-dimensional issue space wherein the probability that person i chooses response k to question j is given by the following:

$$\begin{aligned} Pr(y_{ij} = k) = \frac{exp (\beta _{jk} x_{i} - \alpha _{jk})}{\sum \limits _{l=1}^{K_{j}} exp(\beta _{jl} x_{i} - \alpha _{jl})} \end{aligned}$$

where $x_{i}$ represents person $i's$ ideology, $y_{ij}$ their response to question j, $y_{ij} = k$ indicates response k to question j, and $K_j$ indicates response options for question j, which may vary across questions.

The authors present the complete likelihood, which is the product of all individual observed likelihoods for each response, as below, where I is the set of all people, J is the set of all items, and $I{(y_ij} = k)$ takes the value of 1 if respondent gave response k to question j, and 0 otherwise.

$$\begin{aligned} \prod \limits _{i\in I} \prod \limits _{j\in J} \prod \limits _{k\in K_j} \left( \frac{exp (\beta _{jk} x_{i} - \alpha _{jk})}{\sum \limits _{l=1}^{K_{j}} exp(\beta _{jl} x_{i} - \alpha _{jl})}\right) ^{I{(y_ij = k)}} \end{aligned}$$