Takeaways from Designing and Evaluating the Philly Vax Sweepstakes

Takeaways from Designing and Evaluating
the Philly Vax Sweepstakes

Behavior Change for Good Newsletter | November 2022

In this newsletter, we’re focused on sharing key takeaways from our recently-published research on the effectiveness of Philadelphia’s unique vaccine lottery, which our team designed and implemented. Enjoy!

Katy Milkman and Angela Duckworth
BCFG Co-Directors

Do Geo-Targeted Vaccine Lotteries Increase COVID-19 Vaccination?

Our new paper in Nature Human Behavior features an analysis of the results of the Philly Vax Sweepstakes, a science-based lottery to encourage COVID-19 vaccination that we designed and executed last summer in partnership with CHIBE, the City of Philadelphia, and the Black Doctors COVID-19 Consortium.

What was the Philly Vax Sweepstakes and why did we design it?

The Philly Vax Sweepstakes was a “regret lottery,” meaning that Philadelphians drawn randomly as winners could only accept their check if they had previously received a vaccine. (Imagine the regret people felt when they couldn’t accept a prize!)

The lottery was designed to test the effects of geographically-targeting certain zip codes with high odds of a win (though we were also curious about the overall impact of the lottery on Philadelphia compared to surrounding counties). We conducted three drawings, in which we randomly selected one zip code from the Philadelphia zip codes with the lowest per capita vaccination rates to receive half of the lottery prizes available. This increased residents of the selected zip code’s chances of winning the Sweepstakes by 50-100x compared to other residents.

Did geo-targeting certain zip codes with higher odds of a lottery win work?

No. We found that massively increasing certain residents’ chances of winning up to $50,000 for being vaccinated did not have any estimated benefit, and we can confidently rule out that it produced even one extra vaccination per $4,690 spent. Sometimes we learn a lot from null results, and this is one such case.

What about the overall regret lottery in the city of Philadelphia? Did it add value?

Maybe. The jury is still out on whether regret-style vaccine lotteries can be an effective tool. Results from our attempt to assess the overall impact of our lottery on Philadelphians were inconclusive. Our best estimate is that the lottery generated about 4,000 extra vaccinations in Philadelphia per week during its six week run at an average cost of $16 in prize money per extra vaccine generated. However, Pittsburgh (which had no vaccine lottery) saw a similar bump in vaccinations during those six weeks. These findings indicate more testing would be valuable.

Where can you learn more about our findings?

Our takeaways from this work were featured in a Science Magazine mid-October roundup of the most important research published in other journals, an op-ed in The Philadelphia Inquirer, an article in Nature Outlook, an interview on the Freakonomics, M.D. podcast, and a panel for the National Academies of Sciences, Engineering, and Medicine.

What partnerships made it possible to design and implement a city-wide vaccine lottery?

We’re grateful to the amazing team of BCFG scientists and staff who collaborated on this project (Linnea Gandhi, Sean Ellis, Heather Graci, Dena Gromet, Rayyan Mobarak, Alison Buttenheim, Devin Pope, Ala Stanford, Richard Thaler, and Kevin Volpp), our funders, and the City of Philadelphia for partnering with us on such an ambitious undertaking.

Don’t Miss the Last Fall Event from the Behavioral Science Authors Series

Harvard Professor Max Bazerman will discuss his new book, Complicit, with Dave Nussbaum of Psychgeist Media on Friday, November 11th at 12 PM ET. Complicit is a call to action that we can all help prevent wrongdoing. We hope you’ll join us for this important conversation!

In case you missed any of our earlier fall events, the recordings from conversations with UCLA Professor Cassie Holmes (author of Happier Hour) and NYU Professor Dolly Chugh (author of A More Just Future) are now available on the BCFG YouTube Channel.

Selected Recent Popular Press Coverage of Work by BCFG Team Scientists

Introducing the Future Perfect 50: The Scientists, Thinkers, Scholars, Writers, and Activists Building a More Perfect Future, featuring Ahmed Mushfiq Mobarak, Vox
How to Spend Time on What You Value, featuring Ashley Whillans, The Atlantic
Ensuring Your Products Aren’t Used for Discrimination, by Michael Luca et al., Harvard Business Review
Understanding and Overcoming Belonging Uncertainty, featuring Greg Walton, Behavioral Scientist

Hear From BCFG Team Scientists on These Podcasts

Passion Struck: Max Bazerman and Don Moore on How to Empower Others to Make Better Choices

Max Bazerman and Don Moore discuss findings and case studies from their book Decision Leadership.

Choiceology: So Much in Common: With Guests Samantha Futerman, Anaïs Bordier & Kareem Haggag

Kareem Haggag talks with Katy Milkman about how susceptible we are to changing our preferences.

Dare to Lead with Brené Brown: Brené with Adam Grant and Simon Sinek on What’s Happening at Work
Adam Grant joins Brené Brown and Simon Sinek to discuss the current state of the workplace, including some disconnects between scientific-backed suggestions and managerial practice.

Follow @BehaviorChange to stay up to date on the latest from BCFG.

About BCFG

The Behavior Change for Good Initiative at the University of Pennsylvania is co-led by Psychology and Wharton Professor Angela Duckworth and Wharton Professor Katy Milkman. BCFG unites a world-class, interdisciplinary team of academic experts with leading organizational partners to advance the science and practice of behavior change.

We’re always interested in hearing about new areas to explore and potential collaborations. Contact us anytime.

Post

Forward