Tag: monitoring and evaluation

The British lead . . . and who will follow?

David Cameron gave a speech yesterday at the Global Alliance for Vaccines and Immunisation conference.  It deserves to be read in full – I don’t agree with every word (and how I disagree with many of Cameron’s stances), but it is one of the clearest statements on why we must continue to deliver aid to the poorest and most vulnerable people in the world.
On the down side, Cameron starts out a bit too market triumphalist for my tastes:

At home we don’t tackle poverty by state hand-outs; we help people get into work, to stand on their own two feet and to take control of their own destiny.  The same should be true of development.  No country has ever pulled itself out of poverty through aid alone, so this government will take a new approach.  The same conditions create prosperity the world over.  They include access to markets, property rights, private-sector investment and they make up what I see as the golden thread of successful development.  Ultimately it’s the private sector that will be the engine for growth and that’s why this government’s efforts will increasingly focus on helping developing countries achieve that growth with the jobs and opportunities it will bring.

Well, this is a bit muddled.  First, last I checked England (and Great Britain more generally) was home to a robust welfare state (well, until various Tory governments from Thatcher to Cameron took a hatchet to it) that provided the safety net that enhanced the quality of life of its citizens.  On the other hand (and second), I agree that no country has ever been lifted out of poverty through aid alone – but then, that’s not what aid does.  At best, aid catalyzes much larger processes of change – and sometimes those changes play out constructively (I discuss this at length in Delivering Development).  Third, the only countries to have really changed their status in the last half century have done so by rejecting things like the open market and behaving in very politically repressive ways to get through a serious of difficult transitions that eventually made them competitive in global markets and able to productively take in foreign investment – so this claim about what works isn’t fully supported by the evidence.  Andy Sumner’s work on the New Bottom Billion suggests that this might be changing as a new pile of countries “graduate” from low-income to middle-income status, but this is still unclear as many of the new “graduates” from low to middle income status have just crept above that line, often with no transformation of their economic fundamentals (leaving them vulnerable to slip-back) and still containing huge numbers of very poor people (creating the same problem, and calling into question the very concept of “graduation” to middle income status).
This is not to say that I don’t think markets have any value – I just fear those who place absolute faith in them, especially given that the environment is the site of perhaps the most serious market failure we’ve ever seen.  However, as the speech progressed, I became somewhat more comfortable as, at least in the context of development, Cameron takes a somewhat more moderate tack:

We want people in Africa to climb the ladder of prosperity but of course when the bottom rungs of that ladder are broken by disease and preventable death on a massive scale, when countries can’t even get on the bottom rung of the growth ladder because one in seven of their children die before they reach their fifth birthday, we have to take urgent action.  We have to save lives and then we can help people to live.  So that’s where today’s announcement fits in.  Because there cannot really be any effective development – economic or political – while there are still millions of people dying unnecessarily.

Absolutely correct – the “bottom of the pyramid”, as it were, often finds itself left behind when economic growth programs rev up . . . this is well-understood in both academia and the development institutions.  Indeed, it is not controversial for my Bureau (DCHA – the folks who deal with disasters and conflict) to argue that its work is fundamental to creating a firm foundation for future development efforts because we address the needs of vulnerable populations who might otherwise be overlooked by Agency programming.
But what I most like is the kicking Cameron hands out to those who argue we don’t have the money for aid in these hard economic times.  The kicking comes in two parts – first a moral argument:

When you make a promise to the poorest people in the world you should keep it.  I remember where I was during the Gleneagles Summit and the Live 8 concert of 2005 and I remember thinking at the time how right it was that those world leaders should make such pledges so publicly.  For me it’s a question of values; this is about saving lives.  It was the right thing to promise; it was the right thing for Britain to do and it is the right thing for this government to honour that commitment.

So to those who point to other countries that are breaking their promises and say that makes it okay for us to do the same, I say no, it’s not okay.  Our job is to hold those other countries to account, not to use them as an excuse to turn our back on people who are trusting us to help them.  And to those who say fine but we should put off seeing through those promises to another day because right now we can’t afford to help, I say we can’t afford to wait.  How many minutes do we wait?  Three children die every minute from pneumonia alone; waiting is not the right thing to do and I don’t think that 0.7% of our gross national income is too high a price to pay for saving lives.

…

I actually think that most people in our country want Britain to stand for something in the world, to be something in the world.  And when I think about what makes me proud of our country, yes, I think of our incredibly brave service men and women that I have the honour to meet and see so often; and yes, I think of our capabilities as an economic and diplomatic power; but I also think of our sense of duty to help others.  That says something about this country and I think it’s something we can be proud of.

Where . . . the . . . hell . . . is . . . the . . . American . . . political . . . leadership . . . on . . . this?  Dammit, the British just took the “City on a Hill” mantle from us.  Most Americans want America to stand for something in the world, last I checked.
Oh, and Cameron addresses the unaddressable (for America, it seems) in his speech: that development, in reducing the need for future wars and humanitarian interventions, actually is cost-effective:

If we really care about Britain’s national interest, about jobs, about growth, about security, we shouldn’t break off our links with the countries that can hold some of the keys to that future.  If we invest in Africa, if we open trade corridors, if we remove obstacles to growth, it’s not just Africa that will grow but us too.  And if we invest in countries before they get broken we might not end up spending so much on dealing with the problems, whether that’s immigration or threats to our national security.

Take Afghanistan.  If we’d put a fraction of our current military spending on Afghanistan into helping Afghanistan 15 or 20 years ago just think what we might have been able to avoid over the last decade.  Or take Pakistan.  Let another generation of Pakistanis enter adult life without any real opportunities and what are the risks in terms of mass migration, radicalisation, even terrorism?  That’s why UK support over the next four years will get four million more children in Pakistan into school.  This could be life changing for those children and it can be part of the antidote to the extremism that threatens us all.   So it’s not just morally right to invest in aid, it’s actually in our own interests too.

God help us, Ron Paul seems to be the only candidate for anything willing to say that the wars we are in are costing a hell of a lot of money, and might not have been necessary.  Of course, Ron Paul doesn’t like aid, either . . . actually, he doesn’t seem to like much of anything.  Nobody is really taking his hobgoblin act all that seriously, which means he isn’t going to shift the debate here.  Cameron, though, really glues his fiscal conservativism to a rational argument for aid – maybe we just should have worked on the aid side of things, at a fraction of the cost, and averted the whole mess in the first place.  Lord help me, the Tories are sounding reasonable . . .
Now, Cameron’s ideas for transforming aid are vague, mostly about focusing on results and enhancing accountability.  This is all well and good, but amazingly thorny.  There’s been quite a bit of discussion about evaluation in the development community (great summary list here)  and this blog (here, here and here) of late, and if nothing else, the reader might come to grips with the huge challenges that we must address before we can get to a realization of Cameron’s otherwise nonoffensive ideas.
I suppose it was asking too much to hope a leader talking about transforming development might mention that the global poor might actually have ideas of their own that we should start learning about before we go barging in . . .

Posted on by Ed 0

Challenging development dogma

On his blog Shanta Devarajan, the World Bank Chief Economist for Africa, has a post discussing the debate about the performance and results of the Millennium Villages Project (MVP).  The debate, which takes shape principally in papers by Matt Clemens and Gabriel Demombynes of Center for Global Development and Paul Pronyk, John McArthur, Prabhjot Singh, and Jeffrey Sachs of the Millennium Villages Project, questions how the MVP is capturing the impacts of its interventions in the Millennium Villages.  As Devarajan notes, the paper by Clemens and Demombynes rightly notes that the MVP’s claims about its performance are not really that clearly framed in evidence, which makes it hard to tell how much of the changes in the villages can be attributed to their work, and how much is change driven by other factors.  Clemens and Demombynes are NOT arguing that the MVP has had no impact, but that there are ways to rigorously evaluate that impact – and when impact is rigorously evaluated, it turns out that the impact of MVP interventions is not quite as large as the project would like to claim.
This is not all that shocking, really – it happens all the time, and it is NOT evidence of malfeasance on the part of the MVP.  It just has to do with a simple debate about how to rigorously capture results of development projects.  But this simple debate will, I think, have long-term ramifications for the MVP.  As Devarajan points out:

In short, Clemens and Demombynes have undertaken the first evaluation of the MVP.  They have shown that the MVP has delivered sizeable improvements on some important development indicators in many of the villages, albeit with effects that are smaller than those described in the Harvests of Development paper.  Of course, neither study answers the question of whether these gains are sustainable, or whether they could have been obtained at lower cost.  These should be the subject of the next evaluation.

I do not, however, think that this debate is quite as minor as Devarajan makes it sound – and he is clearly trying to downplay the conflict here.  Put simply, the last last two sentences in the quote above are, I think, what has the MVP concerned – because the real question about MVP impacts is not in the here and now, but in the future.  While I have been highly critical of the MVP in the past, I am not at all surprised to hear that their interventions have had some measurable impact on life in these villages.  The project arrived in these villages with piles of money, equipment and technical expertise, and went to work.  Hell, they could have simply dumped the money (the MVP is estimated to cost about $150 per person per year) into the villages and you would have seen significant movement in many target areas of the MVP.  I don’t think that anyone doubts that the project has had a measurable impact on life in all of the Millennium Villages.
Instead, the whole point here is to figure out if what has been done is sustainable – that is the measure of performance here.  Anyone can move the needle in a community temporarily – hell, the history of aid (and development) is littered with such projects.  The hard part is moving the needle in a permanent way, or doing so in a manner that creates the processes by which lasting change can occur.  As I have argued elsewhere (and much earlier that in this debate), and as appears to be playing out on the ground now, the MVP was never conceptually framed in a way that would bring about such lasting changes.  Clemens and Demombynes’ work is important because it provides an external critique of the MVP’s claims about its own performance – and it is terrifying to at least some in the MVP, as external evaluations are going to empirically demonstrate that the MVP is not, and never was, a sustainable model for rural development.
While I would not suggest that Clemens and Demombynes’ approach to evaluation is perfect (indeed, they make no such claim), I think it is important because it is trying to move past assumptions to evidence.  This is a central call of my book – the MVP is exhibit A of a project founded on deeply problematic assumptions about how development and globalization work, and framed and implemented in a manner where data collection and evaluation cannot really question those assumptions . . . thus missing what is actually happening (or not happening) on the ground.  This might also explain the somewhat non-responsive response to Clemens and Demombynes in the Pronyk et al article – the MVP team is having difficulty dealing with suggestions that their assumptions about how things work are not supported by evidence from their own project, and instead of addressing those assumptions, are trying to undermine the critique at all costs.  This is not a productive way forward, this is dogma.  Development is many things, but if it is to be successful by any definition, it cannot be dogmatic.

Posted on by Ed 1

Required reading . . .

I’ve worked in the field of development studies for more than a decade now, mostly from the academic side.  In academia, we are very good at looking at the nuances of language and practice to try and detect why people do the things that they do.  As a result, in development studies we spend a lot of time thinking about discourses of development – the ways that we think about, speak about and act in the world – and how those shape the ways in which we “do development”.  Mostly, academics do this to explain why it is that development agencies and practitioners keep doing the same things over and over, hoping for a different result (which, you might remember, is how Einstein defined insanity).  There are some wonderful studies based in this approach that everyone should be reading, including Ferguson’s The Anti-Politics Machine, Scott’s Seeing Like A State, and Mitchell’s Rule of Experts (links in the sidebar to the right).  All help us to better understand why development doesn’t seem to work as well as we hope.  I suppose my forthcoming book (link also to the right) falls into this category as well, though I do not wade explicitly into social theory there (if you know the theory, you will see it in there – if you don’t, no worries, the argument is still perfectly intelligible).
What we academic types are not so good at is understanding the reality of life and work in a development organization.  Many of us have never worked in one, or did so a long time ago as a relatively low-ranking person.  However, when you rise in rank in an agency, you start to see the various organizational and political impediments to good work . . . and these impediments are at least as important for explaining development’s many failures as the (often-flawed) discursive framings of the world these agencies employ to understand the world.
With that in mind, I now strongly recommend you read The Clash of the Counter-bureaucracy and Development by former USAID Administrator Andrew Natsios.  Now, I don’t agree with a lot of the things that Natsios says about development in general – indeed, I think some of his logic with regard to economic growth as the core of development is very flawed – but I cannot argue at all with his gloves-off reading of how accountability measures, like monitoring and evaluation, are slowly choking USAID to death.  And it is gloves off – the man names names.  I was not AID under his leadership, but my colleagues all agree that he was a great administrator to work for, even if they did not agree with him all the time.  The man knows development . . . which is more than I can say about some previous administrators here.
By the way, even if you don’t work in development, you should read this – it is a wider lesson about how the best intentions related to accountability can go all wrong.  Those of you working for larger organizations will likely recognize parts of this storyline from where you sit.  And it is a pretty entertaining read, if for no other reason then to watch Natsios just lay it out there on a few people.  Must be nice to be retired . . .

Posted on by Ed 4