Thoughts on the Usefulness of Evaluation

Thoughts on the usefulness of program evaluation

Evidence as to why Managers don’t do Evaluations or don’t use the Results
In our work with clients we strive to find ways for the results of our evaluation activities to be used or at least provide opportunities for learning. According to feedback we receive from our clients, more often than not we are successful. However, too often than I would like to admit that is not the case and we are not alone…there is empirical evidence hiding out here and there as to why managers either don’t do evaluations or don’t use the results when they are done. One recent study is the 2013 federal US government GAO report on Agency Use of Evaluation, see Program Evaluation. This report found that most of the randomly selected 4,000+ federal civilian managers and supervisors surveyed reported they lacked recent evaluation of their programs. Of those respondents, 37% reported an evaluation done in the past five years, and the majority (80%) of these folks reported that “…the evaluation contributed to a moderate or greater extent to improving program management or performance and to assessing program effectiveness or value”.

Pay Attention to Program Stakeholders
For those of us interested in getting an evaluation done and or getting the results of an evaluation actually applied, one critical suggestion is to pay attention to the program stakeholders. See Michael Patton (2000) Utilization Focused Evaluation who writes about utilization focused evaluation, stressing the importance of engaging people who will actually care about the results of the evaluation in the evaluation process from the very beginning. In the 2013 GAO report, they stress the importance of involving stakeholders to facilitate evaluation influence, “…develop relationships to gain their input to planning and buy-in; providing assistance, training, and incentives; and disseminating usable messages”. Patton (2000) indicates that by “…involving specific people who can and will use information enables them to establish direction for, commitment to, and ownership of the evaluation every step along the way”. Program staff seeking to have an evaluation done of their program that can lead to a change or improvement need to be ready to take the time necessary to help us evaluators fully understand what they do. I think effectively involving stakeholders means that evaluators have to listen first to understand before we think too much or act to soon. Or as Yogi Berra put it, “You can observe a lot just by watching”.



About the Author:
Michel Lahti, Ph.D., is the CEO of LeCroy & Milligan Associates. He has an extensive background in program evaluation working in both the public and nonprofit sectors. In addition, he provides consultation to agencies implementing performance measurement systems.

Categories

Thoughts on the Usefulness of Evaluation

Thoughts on the usefulness of program evaluation

Evidence as to why Managers don’t do Evaluations or don’t use the Results
In our work with clients we strive to find ways for the results of our evaluation activities to be used or at least provide opportunities for learning. According to feedback we receive from our clients, more often than not we are successful. However, too often than I would like to admit that is not the case and we are not alone…there is empirical evidence hiding out here and there as to why managers either don’t do evaluations or don’t use the results when they are done. One recent study is the 2013 federal US government GAO report on Agency Use of Evaluation, see Program Evaluation. This report found that most of the randomly selected 4,000+ federal civilian managers and supervisors surveyed reported they lacked recent evaluation of their programs. Of those respondents, 37% reported an evaluation done in the past five years, and the majority (80%) of these folks reported that “…the evaluation contributed to a moderate or greater extent to improving program management or performance and to assessing program effectiveness or value”.

Pay Attention to Program Stakeholders
For those of us interested in getting an evaluation done and or getting the results of an evaluation actually applied, one critical suggestion is to pay attention to the program stakeholders. See Michael Patton (2000) Utilization Focused Evaluation who writes about utilization focused evaluation, stressing the importance of engaging people who will actually care about the results of the evaluation in the evaluation process from the very beginning. In the 2013 GAO report, they stress the importance of involving stakeholders to facilitate evaluation influence, “…develop relationships to gain their input to planning and buy-in; providing assistance, training, and incentives; and disseminating usable messages”. Patton (2000) indicates that by “…involving specific people who can and will use information enables them to establish direction for, commitment to, and ownership of the evaluation every step along the way”. Program staff seeking to have an evaluation done of their program that can lead to a change or improvement need to be ready to take the time necessary to help us evaluators fully understand what they do. I think effectively involving stakeholders means that evaluators have to listen first to understand before we think too much or act to soon. Or as Yogi Berra put it, “You can observe a lot just by watching”.



About the Author:
Michel Lahti, Ph.D., is the CEO of LeCroy & Milligan Associates. He has an extensive background in program evaluation working in both the public and nonprofit sectors. In addition, he provides consultation to agencies implementing performance measurement systems.

Categories