August 2, 2016
Ever wonder how other program administrators are assessed and how that compares to the demands on you?
CEE members can now find out how their peers address questions about evaluation planning. Review the Evaluation Planning Criteria report, now completed and posted to the CEE Forum, to see the types of criteria used to plan and implement program and portfolio evaluations.
In order to help members better understand how their own evaluation planning criteria and coverage compare to those of other jurisdictions, members developed a project scope, survey questions, and collected information on current evaluation planning criteria and coverage. A key objective is that sharing particular and aggregated results with the committee gives members the opportunity to justify their existing criteria or decide on and advocate for changes. An additional benefit is that this information serve as a resource for members seeking to develop evaluation planning criteria for the first time.
Key findings from the report include:
- To help determine which programs in their portfolio should be evaluated and when, 74 percent of respondents out of 34 organizations surveyed currently employ evaluation planning criteria.
- 84 percent employ evaluation planning criteria for both impact and process evaluations.
- While a moderate percentage of respondents indicated their planning criteria include minimum frequencies, very few respondents reported that their planning criteria include impact size thresholds.
- When asked to describe their evaluation planning criteria, many program administrators mentioned that timing of evaluations, savings estimates, regulatory requirements, and uncertainty influenced whether or not evaluations occur.
- Eighty percent of respondents said they were either "somewhat" or "very" satisfied with their current evaluation planning criteria. Reasons cited for this satisfaction included confidence in evaluation results, cost, flexibility to make decisions, and ability to act on results.
Program administrators work together at CEE to share information like this in structured, supported, meaingful ways. While states and provinces most likely will always have large differences in their goals for and approaches to energy efficiency, members have repeatedly found that working together at CEE expands their perspective on their own programs.