Hints for Program Evaluation: A Funder’s Advice

Publication Date: November 1, 2011

Helen Davis Picher leads a planning exercise.

Michelle Eisenberg of consulting firm Schultz and Williams sat down with Helen Davis Picher, Director of Evaluation & Planning, to learn about WPF’s approach to evaluation.  She described how funders’ interest in evaluation (on the part of grantees, as well as larger self-evaluation) has remained strong over the past two decades, as many foundations are trying to be more strategic in their grantmaking. “The resources we have are scarce relative to the problems we are trying to solve,” she said.  While Ms. Picher’s comments represent the perspective of one funder, she offered some lessons and insights that are universally applicable:

What can grantees expect to gain by conducting evaluation of their projects and programs?

Assuming they are looking to find the best solutions to problems, it is helpful for them to understand whether something is or isn’t working – and why.  Are there barriers to implementation?  What should be done differently in the future?  Grantees should look for the most appropriate way to measure individual grant results – simple data collection or observing may sometimes be enough.  Sometimes one type of data can be a proxy for another.  For example, attendance at a museum exhibit can be an indicator of how effective the exhibit is at communicating its desired messages.

How would you recommend an organization begin implementing a comprehensive evaluation function?

The evaluation effort should stem from the organization’s strategic plan and what they want to accomplish over time.  The organization should  then look at the kind of data it currently collects, available research on outcomes in its field, and the progression of steps that lead to its likely or desired outcomes.  It is often not feasible or even necessary for an organization to conduct longitudinal studies of constituents.  Rather, if an organization can understand what is considered to be an indicator of success that corresponds to its step along a larger path, assessing that interim indicator may be sufficient.  For instance, an early childhood organization whose ultimate goal is to help children be ready for school should evaluate whether its program has the qualities known  to provide a good link between early childhood education and  school readiness.

Evaluation findings can be complex. Do you have any advice for presenting results?

Organizations should find a balance between the kind of detailed information that is critical for staff to have, and the “translated” information that lets a Board, for example, know whether the organization is making progress toward its strategic goals.  Part of this comes from not over-collecting information, adapting the appropriate level of inquiry for what the organization is attempting to learn. Organizations should also look for hidden lessons – sometimes data reveals more than you originally expected.



This interview was originally published on the Schultz & Williams website. Used with permission.