Philomathia Seed Fund Award 2011/12
RECONCILABLE DIFFERENCE? EXAMINING THE GAP BETWEEN ENGINEERING PROJECTIONS AND EX-POST RELIZED GAINS FROM ENERGY EFFIENCY INVESTMENTS
- Professor Meredith Fowlie (Agricultural and Resource Economics)
- Professor Catherine Wolfram (Business)
Professors Fowlie and Wolfram are conducting a state-of-the-art evaluation of the nation’s largest residential energy-efficiency program: the Weatherization Assistance Program (WAP). The American Recovery and Reinvestment Act (ARRA) committed $5 billion over two years to this important initiative. Federal funds are being used to improve the energy performance of dwellings of low-income families. The primary objective of this research project is to estimate household-level impacts of the energy‐efficiency retrofits administered through WAP and similar utility‐funded programs. The researchers will measure the direct causal impacts of weatherization on energy consumption and expenditures. The researchers will also examine the persistent gap between engineering projections and ex post realized savings from investments in residential energy-efficiency improvements. In particular, they will investigate behavioral responses to household efficiency improvements which can lead to increased demand for energy services.
This work will facilitate a comparison of WAP's cost effectiveness relative to other programs that aim to reduce the negative externalities associated with energy consumption. Importantly, this analysis relies on a randomized control trial so the results will be highly credible. The project thus demonstrates proof of an important concept. There is broad consensus in the social-science research community that, under certain conditions, well-designed and implemented randomized control trials provide the most valid estimate of an intervention’s impact on outcomes of interest. There is no precedent for incorporating random assignment into energy-efficiency policy evaluation. The research will demonstrate how random assignment can be used to generate estimates of energy-efficiency policy impacts that are transparent, simple to explain, and free of selection bias.
Spring 2015 Update: Putting Energy Savings Programs to the Test
By: Wallace Ravven
As Midwest temperatures dip in October, ash and maple trees flaunt bronze and flaming red leaves, and college football fans launch into super-zeal mode. It’s a great time to be outdoors. Not necessarily so in winter. In Chicago or Des Moines, the warmest part of the day is often below freezing. Nights drop down another 20 or 30 degrees.
For too many in the Midwest and other U.S. cold spots, a failing furnace or poor insulation forces families to choose between near-misery and unaffordable utility bills. The national economic stimulus package that became law after the 2008 global meltdown offered a way out: It added generous new funds to a government program designed to weatherize the homes of low-income families.
The $5-billion Weatherization Assistance Program aims to increase winter comfort and decrease energy use. A potential win-win. The 2009 funding provided qualified households an average of about $5,000 in improvements.
It also offered Berkeley economists Meredith Fowlie and Catherine Wolfram, together with Michael Greenstone, a professor at the University of Chicago, an excellent opportunity to study the effectiveness of such energy efficiency efforts. Most models developed by engineers and policy experts project impressive energy savings
from such weatherization programs. But few studies have systematically compared projected and realized savings.
With seed funds from the Philomathia Foundation, Wolfram and Fowlie carried out a multi-year study of more than 30,000 low-income households that were eligible for the program in southern Michigan. The researchers tracked monthly natural gas and electricity consumption before and after weatherizing improvements were made. They monitored homes that accepted the improvements as well as those that did not.
They also sought solid data on what percentage of households would take advantage of the apparent win-win weatherization opportunity. “We wanted to provide some real ground-truthing,” Fowlie says. They designed a “randomized controlled” experiment — the gold standard in scientific research, but one not commonly carried out in economics studies.
For the experimental part of the research, they divided the sample of some 34,000 households into an experimental group and a control group. The experimental group received a flurry of attention, including robo calls, house visits, door hangers and post cards to raise awareness and encourage participation in the weatherization program. The research team also helped interested families with the federal paperwork needed to apply to the program. The control group received no special outreach.
Somewhat surprisingly, only 15 percent of the “encouraged” households applied for the weatherization, and ultimately, only six percent of them actually followed through and received the weatherizing retrofits. In the control group, two percent applied and one percent ended up participating.
“This was disappointing,” Wolfram concedes. “The good news is that the encouraged households were six times more likely to participate than the control group, but a six percent response is quite low.”
Even among those who did participate, the results were sobering. Households that accepted the new weatherization saved 20 percent on their annual energy bills — less than half of what engineering models had predicted on average. Worse still, the savings in dollars was about half of the cost of the weatherizing retrofits.
“If you’re a low-income family, 20 percent is a huge savings, and it’s also a big savings in energy,” Fowlie says. “But the amount saved falls well below what was projected by the weatherization program. This means that, by most measures, savings did not exceed the cost of the improvements.”
One possible explanation for the discrepancy between projected and realized savings could be that households used more heat since efficiency improvements effectively reduce the cost of heating. However, results from a follow up survey provides little support for this explanation, Fowlie says.
This leads the team to conclude that the measurement gap is due to overly optimistic projections of how these efficiency measures actually perform in real-world settings.
Many carbon mitigation plans rely heavily on energy efficiency, so it’s crucial to identify which programs are working well, but also to identify which programs aren’t delivering the promised savings, Fowlie and Wolfram stress.
The two economists also study strategies to boost the use of affordable electric power in the developing world, where most energy consumption will take place in the future. In India, for example, they are analyzing electricity consumption of people who only recently gained access and were previously lighting with kerosene and walking to a village to charge their cell phones.
“The decisions these people make about energy efficiency may be very different from those people make in the U.S.,” Wolfram says. The rigorous Michigan research, like some smaller studies, highlights the need for more accurate energy savings projections. At the same time, it underscores the fact that people perceive benefits differently, Fowlie and Wolfram say.
When researchers punch the numbers and project a win-win, they run the risk of ignoring factors that can complicate real-world implementation, such as “hassle costs” households may face applying for the programs or anticipating the disruption that installation entails — factors that can throw a well-meaning project’s energy-savings predictions way off course.
“We hope that our ongoing research can improve the success of energy efficiency initiatives, both here and in the developing world,” Fowlie says.
The Philomathia Center plays a key role in advancing UC Berkeley’s innovative research programs in energy and climate research. Established in 2012, the Philomathia Center represents the enduring partnership between UC Berkeley and the Philomathia Foundation in support of a common vision of improving humankind through the creative application of technology, independent thinking, and a commitment to education.