Muhammad Yunus, September 26, 2013.

Measuring Aid Effectiveness Effectively: Being clear about objectives

This blog post was first published on the CGAP Microfinance blog.

It seems extraordinary that after 50 years of international aid, there is still no consensus on whether it works. Zambian economist Dambisa Moyo (Dead Aid) has argued that aid is not only ineffective, but is actually detrimental to development.  Bill Easterly (The Elusive Quest for Growth) says that ‘trillions of dollars’ of aid have had little effect.  Others, notably Jeff Sachs (The End of Poverty) and Roger Riddel (Does Foreign Aid Work?), have argued that there is plenty of evidence of the success of individual aid projects, and that it has brought about substantial improvements in people’s lives.  If we cannot even agree on whether aid works at all, how can we address the more important and nuanced questions such as how to make that aid more effective?

At the heart of this disagreement is not a dispute about the impact of aid but about what we mean when we ask whether ‘aid works.’

Microfinance is an example which mirrors the issues in the wider aid industry. Microfinance has often been touted as a bottom up solution to poverty. The Nobel Peace Prize 2006 was awarded jointly to Muhammad Yunus and Grameen Bank “for their efforts to create economic and social development from below.”  Give people access to credit, the story went, and they will be able to invest in businesses of their own. Instead of needing long-term support, the poor will be able to stand on their own two feet.  The Acumen Fund promises “Dignity not Dependence. Choice not charity.” This attractive prospect is one reason that microfinance has been so successful in raising donor funding, especially from foundations and private giving.

Following dozens of studies of microcredit and microfinance, there is little credible evidence that microcredit itself lifts people out of poverty. The two good randomized controlled trials find no impact of microcredit on poverty (though to be fair they have not yet been running for very long). As the evidence has challenged these grandiose claims, some in the microfinance industry have chosen to defend aspirations which are both more humble and more plausible.  First, there is growing recognition that much else besides access to credit is needed to enable poor people to run a successful business, and so microfinance can at best make a contribution to a wider set of circumstances needed for development.  Second, there is recognition that, even if microfinance is often used for consumption rather than investment, it is still a significant improvement in people’s lives if they have more control over their finances and are better able to deal with uncertainty and volatility in their incomes.  Third, microfinance may lubricate the process of experimentation and failure whichmay help successful firms and enterprises to emerge.

The rest of the aid industry would also benefit from a more nuanced account of its objectives. We often talk about aid as if it falls into two categories: humanitarian aid and development aid. But in reality this is a false dichotomy: most aid falls into neither category. More than 60 percent of aid is a long-term contribution to the provision of key services such as education, health, water and sanitation, and an investment in the institutions needed to provide them in the future. Improving people’s lives is a realistic and laudable goal. Measured against this more humble (but still very important) objective, there is plenty of evidence of the success of aid.  Aid has helped to abolish smallpox, to increase the number of children in primary school, and to give families access to clean water and improved sanitation.  Charles Kenny (‘Getting Better’) has convincingly argued that when measured by almost every standard other than income, the quality of life has improved substantially in developing countries. Foreign aid has made a significant contribution to these improvements.

It is tempting to make the bolder claim that investments in education and health also improve growth and development in the long run.  Perhaps they do – but, as with microcredit, the evidence for this relationship is weak. Why is it not sufficient to say that people everywhere should have access to these services – including financial services – whether or not this leads to long-term transformation of their economy and society?

Everyone wants developing countries to escape aid dependency, and most people recognized that this requires sustainable growth and jobs.  Because this is such a compelling objective, the development industry has been tempted to justify aid on these grounds.  But the evidence from opinion polls and focus groups suggests that the public is willing to support aid which demonstrably meets immediate human needs irrespective of whether this contributes to long-run growth.  By setting excessively ambitious objectives for aid, the industry risks alienating the public from their emotional connection with what aid can achieve, and asks to be measured by standards that it is unlikely ever to be able to show that it meets.

There are many flows of finance to developing countries which will contribute to investment and growth, including direct investment, portfolio capital flows and remittances.  The main drivers of growth will come from the country itself through private and public investment.  Aid is a small proportion of the finance for developing countries. But it is a precious resource because, unlike other sources of finance, it can help meet the needs of the most marginalized communities, women and girls, and people living in long-term chronic poverty.  If we want to see aid used effectively, we should demand that it is used for these purposes for which it has a unique contribution to make.  Just because growth is a priority does not mean it is a priority for aid.

Measured against reasonable claims about what aid can achieve, it is demonstrably effective.  As we have seen with microfinance, the industry damages rather than enhances its case by overstating what aid can achieve. By setting realistic objectives, we can both make aid more effective, and demonstrate the difference it makes.

14 comments on “Measuring Aid Effectiveness Effectively: Being clear about objectives”

  1. I just wonder that the fundamental disconnect is that an aid mentality assumes that all problems can be solved by focusing on the problems themselves, i.e. on the proximate factors…when in actuality, most ‘development’ problems are the result of more generalized problems in the global political economy, which inherently favor the have’s (since they’re the ones making the rules).

    I suppose your point about having clear objectives is okay in some cases, but I also wonder that clear (and definite/finite) objectives very often constrains, rather than liberates. I am suspicious of any intervention that is premised on a specific idea of “what’s good,” which can very often be based on ideology. Seems to me that very often “having objectives” is inherently top-down if they are based on a certain ideological vision of how the world should be.

    1. @Stephanie – I largely agree with what you are saying. I think I’m mainly arguing against setting an objective of growth and transformation, rather than for an objective of improving quality of life.

  2. Thanks for the response. I do agree that well-being (improving quality of life) needs to be more broadly construed than it is (as well as in terms relevant to the particular place). That shift in perspective might lead to some very different kinds of programming.

  3. Hi Owen,

    I agree with everything you say except:

    It is tempting to make the bolder claim that investments in education and health also improve growth and development in the long run. Perhaps they do – but, as with microcredit, the evidence for this relationship is weak.

    When I did a literature review on this at work a few years back I came across a bunch of papers that provided some evidence to support the belief that improved health contributes to improved economic development, even if the impact wasn’t necessarily huge (some refs pasted in below) and only the famous Acemoglu paper that showed the opposite. Some of the positive findings may not be robust, but there are enough of them to suggest that the impact of health on economic development is generally positive, I think. Although maybe not in all cases or with all diseases.

    FWIW, I definitely think that we should fund health services even if we they don’t lead to economic development. But I’m not quite as pessimistic as you regarding whether the link exists.

    Papers finding some positive relationship:
    Arora, S. ‘Health Human Productivity, and Long-Term Economic Growth’, The Journal of Economic History, Vol. 61, No. 3
    Mayer, D. 2001, ‘The Long Term Impact of Health on Economic Growth in Latin America’, World Development, V 29, No. 6
    Gallup J & Sachs J. ‘The Economic Burden of Malaria’, American Journal of Tropical Medicine and Hygiene, 64(1, 2)8, 2001, pp85-96
    Sachs, J. 2003, Institutions Don’t Rule: Direct Effects of Geography on Per Capita Income, NBER Working Paper
    Bloom, Canning & Sevilla, 2004, ‘The Effect of Health on Economic Growth: A Production Function Approach’, World Development V 32, No 1
    Fogel, R. 2004. ‘Health, Nutrition, and Economic Growth’, Economic Development and Cultural Change; Apr 2004; 52, 3
    Bloom and Canning, 2005, ‘Health and Economic Growth: Reconciling the Micro and Macro Evidence’, CDDRL Working Paper, Online at: http://cddrl.stanford.edu/
    Alsan, Bloom & Canning, 2006, ‘The Effect of Population Health on Foreign Direct Investment Inflows to Low- and Middle-Income Countries’, World Development Vol. 34, No. 4
    Ashraf, Lester & Weil. 2008, ‘When Does Improving Health Raise GDP?’, Online at: http://www.brown.edu/Departments/Economics/Papers/2008/2008-7_paper.pdf

    1. @Terence – Thank you. It is very rare that I disagree with you, so I think twice about doing so in this case.

      I think it is pretty clear that most of the productivity gap between rich and poor countries is accounted for by place-specific total factor productivity, not by productivity differences inherent to workers. (Hall & Jones, QJE, 1999). This fits with intuition: when I visit a developing country, I do not get the impression that low incomes are a consequence of people there not having the smarts (education) or fitness (health) to work. They are stuck in a system that doesn’t give them the opportunities to be productive.

      We agree, of course, on the central point, which is that investments in health and education are desirable whether or not health and education are a driver of growth.

      all the best
      Owen

  4. Hi Owen,

    Am just curious about these opinion polls you mention. Whilst I am struggling to find a specific example to link to, my impression is that politicians, normally avid followers of opinion polls, are still very much talking up aid as relatively short term investments with the goal of alleviating poverty after which aid won’t be required any more. Is that, I wonder, because even if there is a majority happy to live with that more modest goal for aid, there is a rather more vocal minority who insist on an exit strategy?

    MJ

    1. @MJ – I learned this from Steven Kull, the director of the Program on International Policy Attitudes (PIPA), which manages WorldPublicOpinion.org. Unfortunately it was at a closed meeting and the paper he presented is not available online. But my understanding of his research was that the public are not much persuaded by the economic and security arguments for foreign assistance. (These arguments are apparently more powerful with members of Congress than the members of the public whom they represent.) – Owen

  5. Hi Owen
    A very interesting posting
    While compared to the growth and development claims of investment in education and health, the evidence of such claim with micro credit is weak as pointed out in the post. Why?. I agree to this reasonable query and share my thoughts on the status position in this regard
    1.AID to education and health helps growth and development since they are direct development input readily absorbable for bringing expected welfare and desired growth where as with the investment in micro credit, it is not a direct input and it represents only financial inputs in terms of legally accepted notes and currencies facilitating as medium of transaction of any kind of welfare goods and services.. Transformation or conversion of these legal papers /dollars into desired good and services need to be performed at poor’s household level by themselves. Here also lot of assumption is to be made on the availability of such welfare goods at the time of requirement . Particularly in the context of investment made in micro finance sector with the ultimate goal of poverty reduction , the functioning of micro credit alone is very weak and cannot work in vacuum unless needed goods and services for transaction through medium of micro credit are available in the given market condition adequately and timely . Besides availability factor, ‘Capability’ (Amartya Sen) of the poor client for such conversion and utilization of the good for gaining desired level of welfare assumes significant in this regard. Evidently in many occasions and in many regions the availability of goods and services and capability factor of the poor do not coalesce each other and consequentially a mere ‘access to credit services’ becomes a futile one showing a putative success in growth and pseudo development. Further in the functioning of investment in poverty sector, it is assumed that there is no misutilization of the micro credit or candid utilization taken for granted at poor household level. . These ground level realities in MF-poverty sector make micro credit less potential and weak for effecting any long term transformation in poverty sector and thereby contribute for AID ineffectiveness
    2. Regarding effectiveness of AID in MF arena, there is a need to appreciate the concept MF more clearly . MF is by and large wrongly represented as micro credit only where as conceptually MF is a package of pro poor financial services such as micro savings, micro insurance , micro pension, transfer services besides micro credit. All these micro financial services holistically facilitate effective poverty reduction. Earlier, Muhammad Yunus has had succeeded in his pilot project with micro credit only since his clients were all micro entrepreneurs mostly engaged in bamboo based cottage industry having ‘capabilities’ on the activities with some wherewithal except micro credit. . When credit is accessed it all worked. But it may be noted that the category of such poor are in the top layer in poverty pyramid. Then what about other poorer and the poorest having weak or no capability factor for productively using the micro credit found in the bottom layers of the pyramid .Poverty reduction or alleviation cannot take place unless the process starts from the bottom of them pyramid.. The great mistake in many aided project, the capability factor is assumed equal for all the layers of the pyramid resulting poor impact and ineffectiveness of the investment thereby producing little evidence for either short term or long term transformation in terms of making a dent in global poverty reduction
    3.Waht is way forward towards AID effectiveness effectively ? a) When the clear objective of investment in MF sector is for poverty reduction then the direction for prudent Investment need to focus on provision of all required diversified micro financial products without confining to micro credit services only to the poor segment for achieving said goal
    b) Micro credit is to be utilized for income generation purpose and not for consumption at poor house hold level for impacting candid development
    c) Sequencing MF services for enhancing capability of the bottom layer in the poverty pyramid for graduated development ( as being experimented in CGAP pilot projects in different countries )
    d) Effectiveness of AID can also be ensured in this MF sector if the investment is directed more for creating a conducive environ in terms of needed physical infrastructure for supporting productive functioning of micro credit in the project area/region and capacity building of the poor client and the institution at grass root level
    e) The ‘AID help’ need to be directed to extend short term support to MF sector in the process of poverty reduction in terms of nurturing and creating social capital like ‘Self Help Group (SHG)’ so that these SHGs (Federations)would take care of their needs with indigenous resources ( human and financial)on a long term basis independently . In such case AID need to move other demanding area for replicating similar action.
    g) Qualitative assessment of AID impact in terms of outreach of the poor in all the layers in poverty pyramid
    h)Last, one way of measuring the effectiveness of AID effectively is to encourage ‘ participatory social audit’ by the poor clients themselves to reveal their perception on the impact of micro finance investment.
    Thanks for sharing my views
    Dr. Rengarajan

  6. A few points came to mind.

    First, is “the industry” really making the claims you state anywhere outside of proposals to donors or implying them in promotional literature? It is my experience that most working in aid have humble and specific ambitions and are fully aware of its limitations. Journalists, however, confuse a desire to “save the world” with actually… saving the whole world.

    Second, I read several of the microcredit/microfinance stories. My understanding, when I was monitoring such projects, was that low interest rates made possible through group guarantees, and strong selection processes, were part of the very definition of MC/MF, and thus, the criticisms heaped on so-called MC/MF (especially in Bangladesh where some of the institutions studied looked more like loan sharks than development institutions) really were not appropriate.

    Finally, it is important that we separate the types of aid that make a difference. Bilateral and international aid doled out by banks and governments with specific political agendas may be more or less likely to have unintended negative consequences than what I would refer to more as “charity” which, when given in isolation, is much easier to understand the effects of because the cause-effect relationship is somewhat less complex. Most NGOs are a-political and many are also non-sectarian. The fact that they get their money through channels that are part of a broader political agenda (be it positive, negative, or neutral) is something that has to be considered in these discussions.

    1. @Elizabeth – Yes, I think the ‘industry’ really is making these claims – both in proposals to donors and more widely. The microfinance industry has been among the worst offenders at making excessive claims. And I think that this sort of overstating the case undermines the cause in the long run.

      On your second points, group guarantees are indeed part of some microfinance and microcredit schemes, but they are not ‘part of the very definition’: they are merely one approach, which has been successful in some circumstances (and not in others).

      I agree with you that the different ways that organisations raise money (e.g. governments with political agendas, NGOs trying to break through public consciousness) affects how they spend money and the effectiveness of their work. There are also returns to scale which have to be taken into account (it is not clear that ‘small is beautiful’ in this business).

      Owen

  7. Excellent article, Owen.

    @ Elizabeth – as a campaigner, I have to say making ludicrous and overbloated claims is endemic in our pucblic facing work. Thinking Make Poverty Histroy, Stop Cimate Chaos, and the most recent – make the east Africa crisis, “Africa’s last famine”. Campaigners and communicators often say, “when we MEAN is an aspiration – and we have to do that to get attention”. But that rests on a host of a crude readings of public opinion and, critically, motivations. This is something we’ve tried to address from a different angle with the Finding Frames report earlier this year (see eee.findingframes.org).

    I think Owen’s spot on when he says “By setting excessively ambitious objectives for aid, the industry risks alienating the public from their emotional connection with what aid can achieve, and asks to be measured by standards that it is unlikely ever to be able to show that it meets.” But it’s worse than that, too. Not only does is misrepresent reality (and validate the misrepresentation) but it reinforces all manner of negative assumoptions people have that can, in a horrifyingly common set of occasions, vere on the racist. If youn look at what undperins public opinion in the UK, there are these four commonly held and largely subconscious assumptions:

    1. Nothing much is changing. People were starving in African deserts 25 years ago and they’re still starving today.

    2. This is partly because mass global poverty is inevitable. “Just the way the world is”. So overblown claims that it can be overcome/beaten/made history reinforces the perception that those making claims to suggest otherwise are profoundly are dreamy idealists. Certinly not credible, professional and real-world pragmatists.

    3. Charity is a legitimate and complete response in the face of mass global poverty. At individual and national level. And this definition of charity doesn’t encompass large scale political action, of the scale that the aid industry has said since its earliest days is needed.

    4. Charity is defined by the “dominant giver/grateful receiver” dynamic that is tinged with a moral dimension in which wealth is equated with moral superiority, and poverty moral inferiority.

    These assumptions are built not only into the public mind but also into the mind of many NGO comms, fundraising, campaigning folk. It’s wedded into the system. So we constantly reinfoce them with a lot of what we say in the North.

    There’s a systemic and professional culture challenge here, and it’s going to take time to address. But I’m quite hopeful that it’s being looked into by some serious people at a serious level. Who knows how long it might take, but it’s better it’s being addressed than not. At least, that’s how I feel this morning.

Leave a Reply

Your email address will not be published. Required fields are marked *

Published by

Owen Barder

Owen is Senior Fellow and Director for Europe at the Center for Global Development and a Visiting Professor in Practice at the London School of Economics. Owen was a civil servant for a quarter of a century, working in Number 10, the Treasury and the Department for International Development. Owen hosts the Development Drums podcast, and is the author Running for Fitness, the book and website. Owen is on Twitter and