Deep Data — and Big Money — Are Driving a New Era in Political Advertising

TV ads shaped by social science-based testing can have a powerful impact on voter attitudes, new research finds. But that comes with challenges for U.S. democracy.TV ads shaped by social science-based testing can have a powerful impact on voter attitudes, new research finds. But that comes with challenges for U.S. democracy.

In the rarefied realm of political campaigns, every election brings complex challenges to the experts who shape a candidate’s advertising strategy. Their standard practice has been to show prospective ads to a small focus group, get feedback, and then follow their instincts to choose the best ads.

But according to a new study co-authored at UC Berkeley, even experienced consultants don’t reliably know what works, and the impact of their television ads is unpredictable. As a result, the researchers say, campaigns today are investing millions of dollars in rigorous, science-based experiments on how potential voters’ attitudes and behavior change in response to specific ads.

The study “gives us a unique window into what's actually going on in political advertisements,” said Berkeley political scientist David Broockman, lead co-author of the new study. “Do these assessments really help? Are they over-hyped? Our answer is: These tools can help a lot. They can give campaigns a real edge.”

Broockman, in an interview, said ads developed with these new content-shaping systems can “easily double the effectiveness of their advertising spending” — and be enough to swing the outcome of a close election.

At the same time, the dramatic evolution in such persuasion tools has “important implications for American elections and democracy,” the authors write. “Because experimentation allows campaigns to increase the effectiveness of their advertising spending, experimentation increases the influence of money in elections, with benefits that redound principally to the best-funded campaigns.”

The new research — “How experiments help campaigns persuade voters: evidence from a large archive of campaigns’ own experiments” — was published online in the American Political Science Review on Feb. 9, 2024. The study follows another article co-authored by Broockman, released last fall, which found that campaign experts generally “have poor intuitions about how to persuade.”

Old-school political ads: art, science — and guesswork?

In the interview, Broockman described political communication practices that have been standard in recent decades. Experts would produce an advertisement, using their experience to guide key decisions. For example: Focus on policy, or the candidate’s personality? Use a celebrity messenger, or the voices of common folk? Use positive messaging, or go negative?

They would then assemble a focus group of 10 or 15 people who might represent that target audience — union members, perhaps, or suburban soccer moms or young voters. They would show that ad to the focus group, get reactions, then tweak the ad based on the response.

The process relied heavily on the experts’ gut instincts.

For their study, Broockman and his co-authors reached an agreement with Swayable, one of the firms that offers “creative testing” of political content before it is released to the public.

The company provided access to data on 617 advertisements prepared by 51 political campaigns in 2018 and 2020, including the 2020 presidential campaign. The researchers knew that all of the ads were for Democratic candidates, but the data otherwise did not reveal the candidates’ identities. In all, more than a half-million potential voters reviewed the ads for Swayable.

What Broockman and his colleagues found is that most of the old assumptions about effective campaign communication did not hold up to scrutiny.

The problem, he said, is that in any election year, the context of any individual race is constantly changing. The issues change, or the mood of the voters changes. Candidates’ attacks and defenses change based on the prominent issues, the climate or on sudden, one-time developments.

“Is it better to focus on the candidate’s policies or personality?” Broockman asked. “Is it better to have a testimonial, or not? What we’re finding — from the big picture to the stuff in the weeds — is that there is no answer. Because the answer basically changes all the time.”

The authors used Swayable data to develop a complex, full-page table showing how dozens of different factors registered on voters’ attitudes and voting decisions. The results were striking.

For example, in the 2018 races for U.S. House and Senate seats, ads focusing on issues were highly effective. Two years later, though, as President Donald Trump was challenged by Democrat Joe Biden, issue-based ads didn’t have the same effect on voters’ decisions in the presidential race or in congressional races.

Having a woman as the “messenger” in ads had a significant impact on voter attitudes and voting decisions in the 2020 Trump-Biden match, but wasn’t important in down-ballot contests. Ads that were “pushy” were effective in the presidential vote, but again, not in the down-ballot races.

A better ad can win a close race

The old approach seems almost quaint compared to the high-speed, data-driven experimentation now emerging.

Data-driven analysis done by Swayable and other companies is valuable because it can tease out even subtle impacts from advertising and do it quickly enough to have an almost immediate impact on the race.

Broockman offered an illustration: A campaign team develops a batch of ads, then takes them to specialty firms, such as Swayable, for evaluation. The firms assemble thousands of viewers to watch each iteration, and often within 24 hours, these social science experiments can discern which of the ads has the most power to persuade.

If factors in the 2018 and 2020 elections were shifting and inconsistent, then social science experimentation to sharpen persuasion campaigns has enormous value. “Choosing an above-average ad over a below-average ad could still easily determine the outcome of a close election,” the authors wrote.

That’s why they emphasized that the emerging tools amplify the effect of money on the political system. “Campaigns may increasingly have success at finding ads that persuade,” they argued, “but the benefits of this technology accrue principally to the campaigns with the financial resources to deploy the ads they select at the greatest scale.”