Yet, the information is crucial not only because it can strengthen the overall performance of the health system by enabling data-driven decision making by our partners on the ground, it is necessary in order to measure progress and impact. And as I’ve written previously, VillageReach is committed to monitoring and evaluating the progress and impact of the Dedicated Logistics System in Mozambique.

The key questions to ask when attempting to evaluate impact are whether or not the Dedicated Logistics System has met its objectives to improve health system performance and increase immunization coverage. In the baseline evaluation, we attempt to answer questions such as: how many children are immunized and how often do health centers run out of vaccines. Unfortunately, this type of information is just not available. There are no immunization registries to search. Health centers don’t keep patient records. Even if they did, we still wouldn’t be able to know how many children are not immunized because there are no vital record systems either. These things just aren’t tracked.

In order to get the information we believe is valuable to answer these questions, we went out and collected it. In order to estimate the immunization coverage, we conducted more than 800 household surveys in randomly selected villages across the two provinces of Cabo Delgado and Niassa. The sample size gives us enough statistical power to make an estimate of the true immunization coverage rate with 95% confidence and because the villages were randomly selected, the sample is representative of the population. In each household, we essentially asked whether or not the children living there had received certain vaccines. In addition, we conducted surveys in more than 60 health centers in those villages to get an idea of how the health centers were performing. We worked with a group in Mozambique who hired and trained local staff to conduct the surveys and complete the data entry. The process took about nine months from the time we first sought Ministry of Health approval for the study until we started seeing the data.

Despite a few challenges along the way including delays in schedules with field teams, traveling time to remote villages, and correcting for concerns with the data such as missing entries and incorrectly completed surveys completed, we have real numbers. We have real data from the very last mile. Finally, we can use this data to answer our questions regarding how many kids aren’t being reached by vaccination services and what we can do to improve this. Using this information, we can tailor our program activities in ways that can make the biggest impact and that is what we intend to do.

Before we can share the results publically, we need approval from the Ministry of Health. We’ve started the process and have already shared the results with provincial leaders in Cabo Delgado and Niassa. We will be presenting the results to the Ministry later this month or early next year. We are looking forward to sharing our results with you as soon as possible. Stay tuned.

Jessica Crawford
Program Associate
VillageReach

Did it work? This question is integral to our lives. Did my Toyota come to a stop when I pressed the brakes? Did my headache go away after I took Tylenol? We don’t always realize it but we are constantly conducting evaluations. These questions and answers contribute to our understanding of what works and what doesn’t work so that we avoid repeating the failures.  The concept is pretty simple, so shouldn’t it also apply to the health programs we support?

It appears that the importance of evaluation in health programs is gaining attention. Perhaps this is in response to Moyo’s argument that $1 trillion in international aid has actually increased poverty and the shocking lack of evidence to the contrary. Last fall, I attended a lecture by Richard Horton, Editor of the Lancet, at which he urged the global health community of Seattle to use their expertise to support critical evaluation of global health data. President Obama’s recently announced Global Health Initiative highlights the need for robust monitoring and evaluation efforts in order to accelerate best practices and impact. Furthermore, donor communities are increasingly demanding evidence of impact from the organizations they support. With increasing attention on the importance of evaluation, the global health community is turning their efforts to the improvement of metrics, data, and evidence.

Barriers to the evaluation of global health programs are not insignificant. Evaluations are expensive, time consuming and highly political. For one thing, donors typically want their contributions to go to strengthening a health program rather than to a study of it. Furthermore, many government and non-government organizations fear finding evidence that their programs are not successful and potentially lose funding. Despite the difficulty, expense, and potential for undesirable results, conducting rigorous evaluations of our programs is one of the most valuable things we can do. Only with evidence can we attempt to answer the question of “does it work?”

-Jessica Crawford

Program Associate

Ultimately, what role does social media play in promoting the work of NGOs?

My name is Alexa and I am first year college student currently volunteering for VillageReach.  I have spent the last week researching different blog sites maintained by practitioners (examples researched include The Acumen Fund and Frontline SMS), research centers (The Center for Global Development), foundations (The Skoll Foundation), academic blogging vehicles (The Stanford Social Innovation Review and The Duke Sanford School of Public Policy), donor communities (GiveWell and Tactical Philanthropy) and independent blogging sites (Good Intentions are Not Enough and Blood and Milk).  The purpose of this research was to identify online communities where VillageReach could engage with practitioners, researchers and donors, raise awareness of VillageReach and network with different organizations and individuals.

I found it compelling to learn more about how different organizations and individuals utilize blogs to promote new ideas, engage with their donors, make announcements and network.  Additionally, some of the most successful non-profits have revolutionized the new social media/network space for fundraising and to create awareness. For example, Kiva’s and Heifer International’s on-line models parallel a function offered by Facebook.  In its configuration, sending a friend a birthday gift of a goat graphic over Facebook does not differ greatly from giving a goat in the name of a friend through Heifer International. Kiva draws its strength through an emotional connection a donor draws through a photo of a recipient and through lender groups. The ever evolving web has changed the rules of the game when it comes to how non-profits structure their models, raise awareness of their cause and fundraise.

My research has guided me to raise several questions on how VillageReach should further engage and connect with other organizations, individuals and donors through the web as well as how other organizations view the web/social media as a tool for their overall mission.

1.)   How has your NGO modified their marketing/communications approach to engage in online opportunities and culture as well as to establish a social media presence?

2.)    What role do social networking sites such as Facebook/Facebook causes play in the work of NGOs?

3.)    How can the blogosphere help to unite fragmented divisions such as public/private sectors, multi-lateral sectors and NGOs?

4.)    Many non-profits such as Oxfam, Charity:Water and Amnesty International, etc. offer opportunities and resources for high school and college aged students to become informed on issues pertaining to poverty and human rights, to take action for a cause and to fundraise on behalf of the NGO.  What role should youth play in VillageReach’s mission/work?

Certainly this concept is not new as non-profits flood their year-end marketing appeals with individual sponsorship programs and passionate stories of hardship and suffering. Kiva is a perfect example of the power of connecting to the individual as they market to donors with a personalized story and photo of each borrower.  So what is our debate about you ask? The question in the office is: If this type of marketing works, and research shows that it does, shouldn’t VillageReach change its message to focus more on individual stories and less about systems?

Up until now, VillageReach has presented its message and for that matter, our brand, as a social enterprise. We are capacity builders and have been recognized as one of the top non-profits for our cost effectiveness in strengthening healthcare systems.  Our work directly impacts those children who so successfully appeal to donors. Yet, this is our challenge. If we focus on the child who we are able to save through immunization, do we lose the opportunity to educate donors on the critical importance of systems building and social change? Do we lose the chance to describe the complicated and sophisticated nature of what we do?

Perhaps this is not black or white and just as any good fund development or marketing manager knows, you need different messages for different audiences. Yet, Professor Singer’s book has given us an opportunity to step back and discuss how we tell our story to current and potential donors.  And wherever we land at the end of this debate, we know either way the most important message of all is in our ability to save a life. That we all agree on. What are your thoughts?  Join us in this debate.

The workshop highlighted inadequacies of the cold chain in addressing the government’s immunization goals, critical gaps in energy, communications, and transportation infrastructure that exist for many last mile communities; and that health workers are overburdened due to the scope and scale of their day-to-day work.  Not surprisingly, our experiences in sub-Saharan Africa are addressing these same issues.

How did we get from rural to urban?  As part of the workshop we included discussions on India’s population growth and industrial development that is leaving millions behind in rural communities throughout the country.   The three states of Bihar, Madhya Pradesh and Uttar Pradesh make up over 50% of the country’s estimated 10 million unimmunized children.  That’s the largest unprotected population of any one country on the globe. The great majority of theses states’ children are the very rural poor.   But two of these states also have the highest child mortality rates for urban poor communities in the country, which highlights the correlation between the rural poor and the rapid urbanization of the country, as desperate migrating families look for employment.  Dr. Siddharth Agarwal of the Urban Health Resource Centre, based in Dehli, is a passionate and persuasive advocate for these urban slum communities.

Not unlike remote rural communities, urban slum populations are difficult to reach.  There is weak physical infrastructure, limited reliable health services and poor documentation of the populations.

The plight of India’s city slum dwellers certainly isn’t a new topic … Mumbai’s Dharavi Slum of “Slumdog Millionaire” fame helped ensure that, but that the task of improving last mile healthcare access for the urban underserved is strikingly similar to that of rural communities, certainly may be.

As we move forward into fall, we look forward to even more good news…

VidaGas was started in 2002 to support the health system improvement program developed by VillageReach and FDC.  More than 80% of Mozambique’s population is rural and depends on charcoal and wood for basic cooking and heating.  Safe and reliable propane from VidaGas enables health centers to provide critical health services including vaccinations, equipment sterilization and nighttime birthing.  Additionally, propane from VidaGas is a clean and affordable alternative to charcoal and wood for households, small businesses, and light industry clients.  VidaGas has grown over 500% since 2002 and is now the largest propane distributor in northern Mozambique.

Oasis Fund is a Luxembourg investment fund which finances innovative, growth stage, commercially viable enterprises that deliver basic goods and services that improve the lives of low-income communities.  This investment is the first investment in Africa for the Oasis Fund.  Oasis Fund is advised by Bamboo Finance, a Geneva based investment advisory firm.

VillageReach’s Social Business Director, Craig Nakagawa, will be speaking with Keely Stevenson, of Bamboo Finance, about the investment at SoCap tomorrow.

So, this begs the question- how do you define and measure sustainability and how important should it be as a goal of a program?  This is especially important when you consider that sustainability often ends up being a trade-off with other qualities- for example, in order to make our program as sustainable as possible, we try to only include elements that we know the government is capable of carrying on after we leave- this can mean sacrificing impactful elements  because they are too expensive, too labor intensive or just too unprecedented for the government to assume control of.   GiveWell concludes that sustainability should be considered “a desirable goal, but not a reasonable requirement.”  The goal of sustainability is fundamental to VillageReach but a more candid conversation about what this truly means could be of enormous benefit to funders and implementers alike, both of whom tend to through the word around without really questioning its value.  On a similar theme- Phillip LaRocco has a humorous note to the “development posse” about lightening up- admitting when things are difficult and cutting through the clichés (of which sustainability certainly must be one of the most ubiquitous!) to truly impact the communities we serve.

GiveWell also explores investing in a small charity.  They note that “giving to VillageReach is a high-risk, high-upside proposition” and honestly, we couldn’t agree more.  VillageReach is proud of its dynamic and innovative approach- our President Allen is fond of saying that our theory of change boils down to “we see, we do, they see, they do.”  Basically, we recognize the problem and because we are small and agile we can create a customized model to address it and then advocate for both the recognition of the problem and the adoption of our model to others.  GiveWell was impressed by our rigorous monitoring and evaluation of our program, an expensive proposition that many small non-profits forgo because they view it as a luxury.  We, however, see quantitative evidence of impact a fundamental necessity in order to convince others of the value of our model.   But it is true that the same things that make us high-impact and allow for change on a scale disproportionate to our size also mean that we don’t have the security of a large, highly diversified non-profit working in a well understood area.  We rely on donors to recognize both the problem of last mile health system strengthening and the value of our solution.  We are thrilled to have GiveWell endorse our approach and we hope to report more success in the future.