What United Way Can Teach Every Ministry (Yes, even yours) About the Need for Measurable Outcomes

Margaret (Meg) Plantz has a response to any nonprofit leader who may be wondering about the importance of outcome measurement to organizational effectiveness.

Actually, she would just as soon let a few satisfied customers do the talking. Measuring Program Outcomes: A Practical Approach, a groundbreaking workbook she helped write for United Way of America, quotes Peggy Ann Dorrian, Talking, Listening, Caring Program Supervisor, Catholic Family and Community Services in Schenectady, New York:

"Outcome information is a real morale builder for the staff. They take it hard when teens get pregnant again, but I was able to show them that only seven percent get pregnant while in the program, compared to 52 percent in the county and 44 percent in the state. It gives them a sense of pride to understand that these girls would be part of the 52 percent if we didn't work with them."

Or consider Kay Coffin, Executive Director, Big Brothers Big Sisters of Fresno County, Fresno, California, who says: "Our outcome measurement system has affected how we are viewed in the community. We can talk to Mr. and Mrs. John Doe in terms of cost-effectiveness for their dollars. It has given staff a sense of pride and accomplishment. We no longer have the turnover we had seven years ago. It has also affected our volunteerism. We had a 313 percent increase in one year."

Just what does a United Way publication like Measuring Program Outcomes have to say to ministry leaders wondering how to define, much less achieve, measurable results?

Please turn to page four and read along:

Why Measure Outcomes?

In growing numbers, service providers, governments, other funders, and the public, are calling for clearer evidence that the resources they expend actually produce benefits for people. Consumers of services and volunteers who provide services want to know that programs to which they devote their time really make a difference. That is, they want better accountability for the use of resources. One clear and compelling answer to the question of "Why measure outcomes?" is: To see if programs really make a difference in the lives of people.

"The most important reason for implementing outcome measurement is that it helps programs improve services," says Plantz, who today serves as a director of Impact Design and Learning for United Way of America. "Outcome measurement helps an organization clarify the purpose of its program and gets all staff focused on same the same goals. Through outcome measurement you can identify effective practices, improve the delivery of your services, enhance your record keeping, communicate your results to stakeholders and compete for resources."

The importance of financial transparency, integrity in fund-raising, and proper use of charity resources—all enhanced by the measurement of outcomes—was brought to light in 1979 with the founding of Evangelical Council for Financial Accountability. This was some 15 years after the National Health Council and the National Social Welfare Assembly, a secular nonprofit, became the first group in the U.S. to focus on nonprofit performance measurement documenting the expenditure of funds.

The demand for outcome measurement in the Christian nonprofit world has emerged, in part, because of a broader cultural groundswell. "In the 1980s and 1990s, several trends came together," says Plantz. "The first was the evolution of measurement. Since the 1970s, the nonprofit sector had been measuring certain aspects of performance from the money they spent to how many clients they served and their adherence to service delivery standards. They then began to track demographic data on clients and client satisfaction. The next logical step was to measure the results of those activities in terms of their benefits for clients.

A second trend was the increased focus by both the public and private sectors on quality improvement. Third, a number of local United Ways started asking the programs they funded to measure their outcomes, because they wanted to make sure they were investing the public's financial resources in programs that were effective and to assure donors that their money was being used in a productive way. Each of those local United Ways developed their own approach, materials and methodology. They began to say to United Way of America, "Rather than us doing this ourselves, you should produce resources we can all use." One result was Measuring Program Outcomes: A Practical Approach, which has sold 160,000 copies since its release in 1996.

In 1997, Frank Lofaro discovered Plantz while working for Prison Fellowship. In 2003 while working in President Bush's Faith-Based Initiative flagship program, the Compassion Capital Fund, he invited Plantz to make a presentation to the entire grantee leadership at the U.S. Department of Health and Human Services. The presentation was ground breaking. Looking back, Lofaro, now President/CEO of Christian Management Association, says, "The necessity for outcome measurement in nonprofits that United Way of America was researching and validating back in the '90s is the same message every ministry needs to hear today."

"By outcomes," says Plantz, "we're talking about the benefits or changes for the people that a program serves, as opposed to how many classes the program runs, or many hours of service it delivers or how many people it serves. With outcome measurement we're asking: 'After all that work you've done, can you show how many of them are better off?'

"Outcomes usually are benefits or changes in participants' knowledge, attitudes, values, skills, behavior, condition or status," notes Plantz. "While increased knowledge and changed attitudes often are key, ultimately you're usually looking for changed behavior." For example, Big Brothers Big Sisters of America evaluated an elementary school-based intergenerational linkages program. The study found an increase in students' grades across all school subjects and increases in social and emotional growth noted by the parent or guardian, the volunteer and the teacher.

Testimonials just a starting place

How does a ministry prone to communicate its effectiveness through life-changing stories, develop an approach to outcome measurement with quantified results?

"Testimonial stories are a starting place," says Plantz. "When an organization is trying to figure out what their outcomes are to know what they ought to be measuring, they can look at these stories and ask, 'What was the benefit of our program for these individuals? What is the change that our services and activities helped them achieve? What has been our value for them?'

"Inspiring stories can help a program understand what it is good at doing. Yet, such anecdotal evidence does not provide data that demonstrates whether one individual's particular outcome is a common result or achievement."

Three meaningful lessons

Perseverance, decisiveness and milestones are Plantz' watchwords to every leader ready to commit to outcome measurement. She offers three lessons:

"First, realize that measuring outcomes takes time. It's not going to happen overnight. You've got to be committed to this over the long haul or it will fall apart. Other things will happen. Emergencies will come up. Distractions will come into play, and your evaluation process could be dropped as the new thing that yet hasn't proven itself. If you're really serious about measuring results, you have to be committed to walk your talk and stay at it.

"Second, take some time to think thoughtfully with your staff, your volunteers, current and former clients, and others who know the program, about what really are the outcomes that your activities produce. What are the changes and benefits your users/clients experience, the things that can reasonably be expected to occur given the services you provide?

"You're likely to find that your staff has widely varying views on the program's intended benefits. The conversation, alone, will get your staff on the same page to say, 'Here's exactly what we're trying to accomplish—not the long-term impact on the rest of someone's life, but change that comes as a direct result of our activities and our services.' Lots of teams are pretty surprised to discover that what they first identify as their outcomes are not necessarily what their activities produce. The organization's activities often produce results that staff may not think about, things their clients may help the group recognize.

"Third, you need to figure out exactly what you're going to measure to show you how successful you've been in achieving those outcomes. Some outcomes are abstract or are open to different interpretations. For example, what are 'good parenting skills' or 'healthy choices'? You'll need to agree on the most important elements of such outcomes and on the specific behaviors or characteristics you want to track.

"Finally, as you make progress on implementing outcome measurement, recognize and celebrate meaningful milestones. When you establish a work group to start planning for outcome measurement, it's a milestone. Celebrate it. When your work group has gotten input from a lot of people, proposed a list of outcomes, and gotten agreement from relevant stakeholders, it's a milestone. Celebrate. When your work group decides on the appropriate indicators, when you collect your initial data, when you take steps to improve your services based on what you learn from the data, you need to celebrate.

"Remember," says Plantz, "the reason you're doing outcome measurement is because it helps you increase the effectiveness of your services and communicate the value of your program to your stakeholders. It's an ongoing process because you never stop trying to improve your program or let others know about the difference it makes in people's lives."