Topline Results: July 2023 Times/Siena Poll of Registered Voters

Methodology

The New York Times/Siena College poll of 1,329 registered voters nationwide, including an oversample of 818 registered Republican voters, was conducted in English and Spanish on cellular and landline telephones from July 23-27, 2023. The margin of sampling error is plus or minus 3.67 percentage points for all registered voters and plus or minus 3.96 percentage points for the likely Republican primary electorate.

Sample

The survey is a response-rate-adjusted stratified sample of registered voters on the L2 voter file. The sample was selected by The New York Times in multiple steps to account for the oversample of registered Republican voters, differential telephone coverage, nonresponse and significant variation in the productivity of telephone numbers by state.

First, the voter file was split by whether the registrant was classified as Republican or not a Republican; sample selection for the Republican and non-Republican samples was performed separately. The definition of a Republican registrant varied by state, depending on the availability of data on partisanship on the voter file. Party registration was used in states with party registration. In states without party registration, participation in recent partisan primaries was used to classify voters. In states without party registration or partisan primary vote history, party classification was based on a model of vote choice in prior Times/Siena polls.

Second, records were selected by state. To adjust for noncoverage bias, the L2 voter file was stratified by statehouse district, party, race, gender, marital status, household size, turnout history, age and home ownership. The proportion of registrants with a telephone number and the mean expected response rate, based on prior Times/Siena polls, were calculated for each stratum. The initial selection weight was equal to the reciprocal of a stratum’s mean telephone coverage and modeled response rate. For respondents with multiple telephone numbers on the L2 file, the number with the highest modeled response rate was selected.

Third, state records were selected for the national sample. The number of records selected by state was equal to the reciprocal of the mean expected response rate of the state’s records, multiplied by the state’s share of registered Republican or non-Republican voters nationwide, divided by the national sum of the weights.

Fielding

The Republican and non-Republican samples were stratified by party, race and region and fielded by the Siena College Research Institute, with additional field work by ReconMR, the Public Opinion Research Laboratory at the University of North Florida and the Institute of Policy and Opinion Research at Roanoke College. Interviewers asked for the person named on the voter file and ended the interview if the intended respondent was not available. Overall, 77 percent of respondents were reached on a cellular telephone.

The instrument was translated into Spanish by ReconMR, and Spanish-speaking interviewers were assigned to the modeled Hispanic sample. Bilingual interviewers began the interview in English and were instructed to follow the lead of the respondent in determining whether to conduct the survey in English or Spanish. Monolingual Spanish-speaking respondents, in either the Hispanic or non-Hispanic samples, who were initially contacted by English-speaking interviewers were recontacted by Spanish-speaking interviewers. Overall, 14 percent of interviews among self-reported Hispanics were conducted in Spanish, including 9 percent of the registered Republican sample and 19 percent of non-Republicans.

Weighting — registered voters

The survey was weighted by The Times using the R survey package in multiple steps to account for the oversample of Republican voters.

First, the Republican and non-Republican samples were adjusted for unequal probability of selection by stratum.

Second, the Republican and non-Republican samples were weighted separately to match voter file-based parameters for the characteristics of Republican and non-Republican voters. The Republican and non-Republican samples were also weighted to match targets for the self-reported educational attainment of Republican and non-Republican voters, based on previous Times/Siena polls.

The following targets were used to weight the Republican and non-Republican samples:

• Party, among non-Republican voters (aforementioned partisan classifications)

• Age (Self-reported age, or voter file age if the respondent refuses)

• Gender (L2 data)

• Race or ethnicity (L2 model)

• Education (four categories of self-reported education, weighted to match NYT-based targets derived from Times/Siena polls, census data and the L2 voter file)

• Marital status (L2 model)

• Home ownership (L2 model)

• Metropolitan area (2013 National Center for Health Statistics Urban-Rural Classification Scheme for Counties)

• Region (Census Bureau definition, except Maryland, Delaware and Washington, D.C., reclassified as Northeast)

• Turnout history (NYT classifications based on L2 data)

• Vote method in the 2020 elections (NYT classifications based on L2 data)

• Census block group density (A.C.S. 5-Year Census Block Group data)

• Home value (L2 data), among Republican registrants.

Finally, the two samples were combined and adjusted to account for the oversample of Republicans.

Weighting — likely Republican primary electorate

The survey was separately weighted in multiple steps to match targets for the composition of the likely Republican primary electorate and to account for the self-reported turnout intention of respondents.

First, the Republican and non-Republican samples were adjusted for unequal probability of selection by stratum.

Second, the first-stage weight was adjusted to account for the probability that a registrant would participate in the 2024 Republican primary, based on a model of turnout in contested Republican primaries. Separate models were fit for closed, semi-closed and open primaries. The probability that ineligible registrants — like a registered Democrat in a closed primary state — would participate in the 2024 primary was based on an analysis of Times/Siena polling from October 2019 and L2 voter file data, which found that 2 percent of ineligible registrants ultimately participated in the 2020 Democratic primary, including 10 percent of Republican registrants who identified as a Democrat at the time of the poll.

Third, the Republican and non-Republican samples were weighted separately to match targets for the composition of the likely Republican electorate. The targets for the composition of the likely electorate were derived by aggregating the individual-level turnout estimates described in the previous step for registrants on the L2 voter file. The categories used in weighting were the same as those previously mentioned for registered voters.

Fourth, potential Republican primary voters were assigned a probability of voting based on their self-reported intention to vote. Voters were considered potential Republican primary voters if one of three conditions were met:

• They identified as Republican or leaned Republican on two questions about party identification.

• They were registered as a Republican in a closed or semi-closed primary state where they were only eligible to participate in the Republican primary.

• They said they would “probably” vote in the Republican primary on a question asking if they were likely to vote in the Republican primary or the Democratic primary, or probably wouldn’t vote in a primary.

If a voter did not meet any of these criteria, they were assigned zero probability of voting in the Republican primary. If they met at least one of these criteria, they received a probability of voting based on the relationship between self-reported turnout intention and validated vote in a post-election analysis of Times/Siena data from the 2018 and 2022 midterm elections. Potential Republican primary voters who said they were “unlikely” to vote or likelier to vote in the Democratic primary were assigned a probability of voting based on the Times/Siena polling conducted in October 2019.

Fifth, the initial likely voter weight was adjusted to incorporate self-reported vote intention. The final probability that a registrant would participate in the Republican primary was two-thirds based on their ex ante modeled turnout score and one-third based on their self-reported intention, based on prior Times/Siena polls, including a penalty to account for the tendency of survey respondents to turn out at higher rates than non-respondents. The final likely electorate weight was equal to the modeled electorate rake weight, multiplied by the final turnout probability and divided by the ex ante modeled turnout probability.

Finally, the two samples were combined and adjusted to account for the oversample of Republicans.

The margin of error accounts for the survey’s design effect, a measure of the loss of statistical power due to survey design and weighting. The design effect for the full sample of registered voters is 1.86, with design effects of 1.26 and 1.28 for the two separately weighted samples of Republicans and non-Republicans, respectively. The design effect for the likely Republican electorate is 1.52, which includes the added variance due to incorporating the probability that a respondent will participate in the Republican primary.