It is all decided by a Professor in New York

Jeff Marlow writes in the New York Times about Koraro, a Millennium Village Project village in Northern Ethiopia:

As the project’s first five years wind down, its ultimate goals remain elusive, and the five-year initiative has swelled to 10. The extension, naturally, will require more spending: The financial injections to date—over $5 million per year in a mix of cash and non-cash contributions—have not abolished poverty. Improvements in the five sectors targeted by the MVP are readily apparent, but their sustainability is still up in the air.

There are many people in the development set who are sceptical about the utility of the Millennium Village Project, for good reasons and for bad.  Village-level interventions have had a chequered past, and the conventional wisdom today is that development assistance should help to build capable and accountable states which can deliver services, from agriculture and education to security and health, and not provide these separately from the systems that are being established.

I don’t know as much about Koraro as Jeff, but G and I did visit the town, unannounced, one day when we happened to be driving past.  We struck up a conversation with a local shopkeeper which went like this:

O&G:                 What is it like being a Millennium Village?

Shopkeeper:    Very good. We have lots of things.

O&G:                 Does everything work well?

Shopkeeper:    No, not all of it.  But we are much better off now.

O&G:                 Who decides what to change? Do you have a village council, or is there an Elder who decides?

Shopkeeper:   It is all decided by a Professor in New York.

O&G:                 Really? Do you know his name?

Shopkeeper:   No. But he is a very famous man

I don’t have the same ideological objections to Potemkin Villages the Millennium Villages Project as some other people. As both Jeff Sachs and Nick Stern have arged, it seems plausible that there may be significant complementarities between interventions which mean that programmes work better if there are other successful programmes at the same time.  For example, there may be little value in increasing agricultural productivity to generate surpluses if there is no way to get those surpluses to market, which requires infrastructure.  That suggests that each community may need a big heave:  ensuring that all these things come in together may be more effective than a series of uncoordinated interventions spread thinly.

For me the most disappointing aspect of the Millennium Villages Project has been the steadfast refusal to subject it to rigorous evaluation.  (Their evaluation programme is described here.)  The most detailed study so far has been conducted by the Overseas Develoment Institute.  The problem is lack of a proper basis of comparison.  Ethiopia is changing quite rapidly, and Korkora would have changed with or without the Millennium Village Project. For example, there has been a 51 percent reduction in malaria cases in Koraro, Ethiopia. This has been touted as a success by the supporters of the project; and it sounds impressive until you find out that malaria cases have been more than halved across the whole country, not just in Koraro.  The improvement in the Millennium Village is apparently no greater than anywhere else in the country.

To evaluate the project, Millennium Villages need to be compared with some suitable control group, ideally through randomised controlled trials.   Ideally, the individual components of the project would also be randomised to test the hypothesis that the effects of interventions are complementary.  (It follows that I don’t agree with Chris Blattman’s view that it would be too hard.)

It would, as Chris says, be pretty surprising if the Millennium Villages Project does not make a difference. After all, it is spending money roughly equivalent to 100% of the villagers’ income. Furthermore, it has benefited from close personal attention from the Prime Minister, other ministers and officials, researchers and academics (and, of course, a famous Professor from New York).  A rigorous evaluation would help us to know how big that improvement is, and and what cost.  It might also give us insights into whether any particular parts of the progamme are particularly important.

7 thoughts on “It is all decided by a Professor in New York”

  1. Thanks for bringing this issue up Owen.

    It drives me nuts that they are trying an innovative (or, said differently, completely untried) and very expensive approach to development, and not evaluating it. Grrrr argh! Why do funders keep spending money on things and not requiring evaluation?

    For your and your readers’ reference, here are the funders of the project, who evidently don’t prioritize evaluation, even when they are supporting something completely new:

    Bilateral Government Support

    * Government of Japan
    * Government of South Korea

    Foundation Support

    * Kingdom Foundation
    * MacArthur Foundation
    * Nike Foundation
    * Packard Foundation
    * United Nations Foundation

    Private Sector Support

    * Becton, Dickinson and Company
    * Ericsson
    * GE
    * GlaxoSmithKline
    * JM Eagle
    * KPMG
    * Monsanto
    * Motorola
    * Novartis
    * Sanofi Aventis
    * Sony Ericsson
    * Sumitomo
    * TechnoServe

  2. It would be interesting to investigate what the separate effect of the national and international attention on its own would be. I honestly don think this should be considered as a blank: the power for change of a driven community that is listened to might be higher than any financial or technical investment.

  3. Owen, I’m going to call you out on your power calculations. Whether or not it’s too hard is simply a question of sample size (and corresponding research funds). But the minute you get get DFID to fund the Villenium Millages you know I am so there.

  4. Pingback: From Poverty to Power by Duncan Green » Blog Archive » Zero rupee notes; MLK and powerpoint; Millennium professors; new sites on globalization and politics of climate change and what Obama could learn from Bartlett: links I liked

  5. For comparison, the Rwandan government has set up its own plan, is putting up its own money for much of it, and is trying to get the donors to back it. There’s still not much explanation of how things will be evaluated, and I worry about the overly optimistic rhetoric, but it does look somewhat better in the self-determination department.

  6. Pingback: New Year’s Resolutions « Just Another Development Blog

  7. Pingback: I finally visited a Millennium Village: Some Reflections | Nancy Birdsall | Global Development: Views from the Center

Leave a Reply

Your email address will not be published. Required fields are marked *