Category: Delivering Development

Where Quant and Qual meet: On speculation, price instability and food insecurity

UPDATE: Marc Bellemare pointed out some issues with this post, which I have addressed here.  These issues, though, strengthen the argument about strategic deglobalization . . .

§§§§§§

There have been an interesting series of blog posts going around about the issue of price speculation in food markets, and the impact of that speculation on food security and people’s welfare.  Going back through some of these exchanges, it seems to me that a number of folks are arguing past one another.
The most recent discussion was spurred by a post on the Guardian’s Global Development blog by John Vidal that took on the issue of speculation in food markets.  In the post, Vidal argues that food speculation is a key driver of price instability on global food markets, which results in serious impacts for the poorest people in the world – a sort of famine profiteering, as it were.
The weakness of this post, as I see it, are twofold.  First, it doesn’t take the issue of price arbitrage seriously – that is, how speculation is supposed to function.  Aid Thoughts, via one of the comments on Vidal’s post, takes Vidal to task for this.  As Aid Thoughts/the commenter point out, the idea behind speculation is to pull future price impacts of shortage into the present, stimulating responses to future shortages before they occur.  Thus, a blanket condemnation of speculation makes very little sense from the perspective of one who wants to see food security enhanced around the world – without speculation, there will be no market signal for future shortage, creating a world that addresses shortages in a reactive instead of proactive manner. This is a completely fair critique of Vidal, I think.
However, neither Vidal nor those responding to him actually address the evidence for significant market manipulation, and the intentional generation of instability for the purposes of profiteering.  This evidence first emerged in a somewhat anecdotal manner in Fredrick Kaufman’s “The Food Bubble: How Wall Street starved millions and got away with it.”  In this article, Kaufman uses a fairly limited number of informants to lay out a case for the intentional manipulation of wheat markets in 2008.  It is an interesting read, though I argued in an earlier post that it suffers from trying to be a parable for the pervasive presence of complex investment vehicles in the modern world.  And in the end, its findings can hardly be called robust.
Though Kaufman’s argument might, by itself, be less than robust, it received a serious empirical boost from the International Food Policy Research Institute (IFPRI) in the fall of 2010.  In a discussion paper that remains underreported and under-considered in food security circles (trust me, it is difficult to get anyone to even talk about speculation in program settings), Bryce Cooke and Miguel Robles demonstrate quantitatively that the dramatic price rises for food in 2008 is best explained by various proxies for speculation and activity on futures markets.  Now, we can argue about how large an impact that activity had on actual prices, but it seems to me that Cooke and Robles, when taken in concert with the Kaufman piece, have demonstrated that the speculation we see in the markets right now is not merely a normal market response to potential future shortage – indeed, the Food and Agricultural Organization (FAO) of the United Nations has been arguing for months that there are no likely supply issues that should be triggering the price increases we see.  In other words, while it is foolish to simply blame price arbitrage for food insecurity, it is equally blind to assume that all of those practicing such arbitrage are doing so in the manner prescribed in the textbooks.  Someone will always try to game the system, and in tightly connected markets, a few efforts to game a market can have radiating impacts that draw in honest arbitrage efforts.  There is need for regulatory oversight.  But regulation will not solve all our food problems.
But this all leaves one last question unanswered: what is the impact of price instability, whether caused by actual likely future shortages or by efforts to game markets for short-term profits, on the welfare of the poor?  Vidal, Kaufman and many others assume that the impacts are severe.  Well, maybe.  You see, where matters (again – yep, I’m a geographer).  In a very interesting paper, Marc Bellemare (along with Chris Barrett and David Just) demonstrates that, at least in Ethiopia:

contrary to conventional wisdom, the welfare gains from eliminating price volatility would be concentrated in the upper 40 percent of the income distribution, making food price stabilization a distributionally regressive policy in this context.

This finding may be a shock to those working in aid at first glance, but this finding is actually intuitive.  In fact, in my book (out tomorrow!) I lay out a qualitative picture of livelihoods in rural Ghana that aligns perfectly with this finding.  In Bellemare et al, I would bet my house that the upper 40% of the population is that segment of the population living in urban areas and/or wealthy enough to be purchasing large amounts of processed food.  Why does this matter?  This is the segment of the population that typically has the most limited options when food prices begin to get unstable.  On the other hand, the bottom 60% of the population, especially those in this cohort living in rural areas (it is unclear from the study how much of an overlap between poor and rural there is in the sample, but I am betting it is pretty high), has a much more limited engagement with global food markets.  As a result, when food prices begin to spike, they have the ability to effect a temporary partial, or even complete, disengagement from the global market.  In other words, much as I saw in Ghana, this study seems to suggest that temporary deglobalization is a coping strategy that at least some people in Ethiopia use to guard against the vagaries of markets.  Ironically, those best positioned to effect such a strategy are the poorest, and therefore they are better able to manage the impact of price instability on food markets.
In short, I would argue that Marc’s (and his co-authors’) work is a quantitative empirical demonstration of one of my core arguments in Delivering Development:

2. At globalization’s shoreline the experience of “development” is often negative. The integration of local economies, politics, and society into global networks is not the unmitigated boon to human well- being presented by many authors. Those living along the shores of globalization deal with significant challenges in their lives, such as degrading environments, social inequality that limits opportunity for significant portions of society, and inadequate medical care. The integration of these places into a global economy does not necessarily solve these problems. In the best cases such integration provides new sources of income that might be used to address some of these challenges. In nearly all cases, however, such integration also brings new challenges and uncertainties that come at a cost to people’s incomes and well- being. (pp.14-15)

I’m not suggesting Marc endorses this claim – hell, for all I know he’ll start throwing things when he sees it.  But there is an interesting convergence happening here.  I’m glad I met Marc at a tweet-up in DC a few weeks ago.  We’re going to have to talk some more . . . I see the beginning of a beautiful friendship.
In summary, while efforts to game global food markets do exist, and have very serious impacts on at least some people, they do not crush everyone in the Global South.  Instead, this instability will be most felt by those in urban areas – in the form of a disaffected middle and upper class, and a large cohort of the urban poor who, lacking alternative food sources, might be pushed over the brink by price increases.  The policy implications are clear:

  • We need to be watching the impact of price increases on urban food insecurity more than rural insecurity
  • Demanding that rural producers orient themselves toward greater and greater integration with global markets in the absence of robust fallback measures (such as established, transparent microinsurance and microsavings initiatives) will likely extend the impact of future price instability further into the poorest populations.
  • We need to better understand the scope of artificially-generated instability and uncertainty in global food markets, and establish means of identifying and regulating this activity without closing price arbitrage down entirely.
Posted on by Ed 9

A world with less poverty . . . maybe

Brookings has come out with a report suggesting a dramatic decrease in the number of people living in poverty (using the $1.25/day mark as a measure of poverty) since 2004.  The report suggests that where 1.3 billion people met this description in 2004, today less than 900 million are dealing with similar circumstances.  In short, we are on target to achieve the first Millennium Development Goal (MDG) of cutting the global rate of poverty to half of the 1990 rate – indeed, the report suggests that:

the MDG1a target has already been met—approximately three years ago. Furthermore, by 2015, we will not only have halved the global poverty rate, as per MDG1a, but will have halved it again. (p.4)

This is remarkable news.  Brookings notes that the rate of poverty reduction varies dramatically by region, with East and South Asia cutting rates by about 50% between 2005 and 2010, while sub-Saharan Africa’s rate fell just under 8% in that same period.  Further, just two countries can account for the majority of this drop: India and China.  So there are still big challenges out there to be addressed, but things are looking up.
Or are they?
A glance at the methodology employed by this study leads me to think that the error bars on this study are rather huge.  Indeed, the authors are fully aware of the data and analytic challenges related to any effort to estimate poverty levels.  As the authors note, in development

we find it remarkably difficult to measure whether it is happening, and if so how fast. This is especially the case when it comes to producing global poverty data, as the challenges of national poverty data collection are multiplied several times over and then further compounded by the tricky—and unsatisfactory—business of converting national results into internationally comparable terms.

In short, the authors know that the project on which they have embarked is likely to generate estimates with significant potential errors – “error bars” as it were, around their data points, in which reality might actually exist.  Oddly, the report makes no effort to present these error bars.  Instead, it makes rather bold claims about reductions in the level of poverty largely without caveat.  I am not convinced these claims are warranted.
First, there are major data issues here.  Their 2005-2010 measures are predicated on recent household survey data.  Here is the problem with household survey data in sub-Saharan Africa: a lot of it is junk.  I’ve tried to deal with such data in Ghana, a country that has a relatively robust infrastructure for this sort of work, and found their survey data to be a mess.  I suspect that in some regions (Latin America, parts of Asia) the data is actually quite good, on the whole.  But in a lot of places (most of SSA and Southeast Asia) the data is likely very problematic.  And even where the data infrastructure is pretty good, the survey methodologies are often found wanting.  I was part of a team that tried to get a handle on livelihoods near a forest reserve area in Southern Malawi – to do so, we sampled 300 households across four villages (75 households/village) quarterly for a year, to capture things like seasonality in our dataset.  2400 structured interviews had to be undertaken to do this, and those interviews were supplemented by semi-structured interviews with subsamples of the group to explore issues like household power and gender relations to give context to that larger dataset.  This was enormously labor-intensive . . . and necessary to really understand what was going on in those villages.  Most household surveys are not done in this manner, and thus are subject to seasonal bias, or the presentation of data as comparable across the country when, in fact, it has very locally-specific meanings rooted in local social context. I do not expect that all national household surveys will be as rigorous or labor-intensive as ours was, but one should acknowledge the limitations of the data.  No discussion of this in the paper, but that can put a pretty wide margin of error on your findings.
I won’t even wade into the issues with population data that they gloss over in this study – I spend a good bit of time in chapter 9 of Delivering Development talking about census issues and the problems of compounding data error in estimations of economic growth.  Let’s just say that there are significant uncertainties around census data that compound any other errors in the data – again, growing error bars.
Second, there is a moment in the analysis that I found stunning – their projections to 2015 predicated on a surprising assumption – that distribution of wealth will stay the same.  Well, given that economic growth is, by and large, predicated on unevenness within regions, countries and between countries, there is basically no chance that the distribution of wealth will remain the same in any place that is growing.  Generally speaking, the GINI coefficient goes up as growth goes up . . . and a lot of places they are talking about are meant to experience fairly robust rates of growth now and in the near future.  More error.
What does this mean?  Well, to me it means that the data they presented like this:

Really has a wide margin of error, even for past observed data (but compounded going forward) that should look be presented like this (with margins of error in red, and not to scale.  I did not calculate them, as this is just illustrative):

OK, so perhaps there should have been some error bars in there.  So what?  Well, this is a policy brief, with policy recommendations that might actually be followed by someone . . . and this brief is arguing that we are on top of the whole poverty reduction thing, which is sure to become an argument for looking for ways to trim development budgets.
Even if the budgetary ax does not fall because of this brief, there is a risk of reprioritization that may not yet be appropriate.  In the recommendation

aid donors must adapt to the evolving poverty landscape and update their policies and programming to reflect current needs and priorities


the brief implicitly argues that agencies should be reevaluating their programming based on the findings in the brief – toward a focus on Africa and fragile states, and away (apparently) from much of Asia and those parts of Latin America, the Caribbean, and the Pacific where we currently work.  However, this is a recommendation based on much thinner evidence than it seems.
The worst part is that I think this presentation of the data undermines one of their excellent policy points:

One final policy recommendation revealed by this analysis is the need to improve the quantity, quality and timeliness of poverty data, at both the national and the global level. For both developing country governments and aid agencies working to fight poverty, it is impossible to efficiently allocate resources toward this goal using poverty data that is incomplete, unreliable or out of date.

At the country level, there has already been a significant uptake in the use of household surveys and an improvement in their quality. Yet in remarkably few countries is there a standardized, recurrent—and therefore consistent—approach to household survey data collection and analysis.  A renewed, long-term commitment to build capacity in domestic statistical agencies would be a valuable use of aid agencies’ resources.

I agree completely, and have argued for this need, but by presenting the data as so clear and robust, they have essentially undermined this argument.  Any policy maker looking at this will wonder why s/he should give more funding to something that already works . . .
Folks, policy makers will never learn to deal with uncertainty until they are faced with it . . . if we keep copping out and “firming up” mushy results into single bold trendlines, they will expect certain outcomes from uncertain data indefinitely.

Posted on by Ed 1

Good lord, still trending up?

We’ve broken the 50k barrier!  And this is trending data . . . I bounced up and down between 50k and 70k today, so climbing into the 40k range is reflective of even more new sales . . .
Could this be sustainable?

Thanks to everyone!

Posted on by Ed 0

Liveblogging Dead Aid (Chapter 3)

And the beat goes on . . . ladies and gentlemen, Chapter 3.
p.29: Well, so much for starting brightly.  She has grossly oversimplified Diamond (which is hard to to, y’all) to argue that a country’s wealth and success depend on geography and topography.  Er, no, that would be a form of environmental determinism.  Diamond was writing an anti-racist history of the world, explaining how the conditions that would eventually result in the ability of some groups to colonize others, etc., was enabled by environmental and geographic situations – but Diamond does not simply erase colonialism from the equation, he is trying to set the stage for how it came about.  You could argue that he has a somewhat environmentally determinist take on the causes of colonialism, maybe . . .
Oh, and for Diamond’s purposes, Africa was not resource-rich . . . it lacked easily domesticable crops and animals when compared to other world regions.  The whole discussion of squandering natural riches on page 30 is a total non-sequitor in the context of Diamond.
Note: I really don’t love Diamond’s book . . . and I am defending it here.  Ugh.
p.30: OK, the geographer in me just screamed.  I can’t blame Moyo for this – it is all about Collier, who along with Sachs and a few others in the field of economics is slowly resurrecting environmental determinism (or at least geographical determinism) with their damn correlations between coastline, endowment of natural resources, and economic growth.  The connections between these three issues are so complex that any analysis that simply divides countries into three categories (resource poor/coastline, resource poor/no coast, resource rich) is going to over-aggregate different relationships and causes into gross oversimplifications and false correlations.  Further, the damn N for these analyses is going to be less than 20 for one or more categories (less than 60 countries in Africa, folks).  I mean, you can run non-parametric stats on this sort of thing, but for the love of God, why?  Just do the qualitative work, dammit.
p.31: Moyo seems to have completely and utterly missed the reason why colonialism had such a brutal impact on African development.  Sure, artificial countries were not great.  And the inherited governmental structures after colonialism often caused problems.  But this sort of thing only really mattered after independence.  By then, these places had been completely restructured into sources of primary materials for the industries of the Global North – infrastructure, agricultural innovation, etc., all of it was aimed at enriching someone else and ensuring the colonized never developed any economic power of their own.  This led to the perpetuation of colonial relationships by other means after independence (neocolonialism), and I have little doubt this is way more important than the borders or governmental structures when we try to understand the growth trajectories of Africa since independence.  Either she is stunningly ignorant of her own country’s history, or this is a very disingenuous reading of African history.
p.32: Wonderful, Paul Collier postulates that the more ethnically divided the country, the more likely the prospect of civil war.  In other news, people with guns are more likely to shoot one another.  How much more likely?  Is this a cause unto itself, or a variable mobilized to political ends that can be better explained by another variable (I’m looking at you, Rwanda)?
p.34: If you are going to use Botswana as an example of a place where growth and development were facilitated by good institutions (which it was), you still have to contextualize the huge growth numbers by noting the GIANT DIAMOND MINES in the country.  I’m just sayin’.
p.35: Nondiagnostic diagnoses make me crazy.  “Africa’s failure to generate any meaningful or sustainable long run growth must, ostensibly, be a confluence of factors: geographical, historical, cultural, tribal and institutional.”  Again, no kidding.  This is meaningless.  Of course, it also discounts her previous example of Botswana having meaningful economic growth. Or Ghana. Or South Africa.  In other words, her whole statement is an overgeneralized negative that doesn’t hold up to scrutiny (or, in fact, her own argument from a page ago).   Next part of the diagnosis: “No factor should condemn Africa to a permanent failure to grow.” I don’t know of anyone making that claim.  If we were, we wouldn’t really bother with development, would we?  We’d just give up and walk away . . .  And the final part: “for the most part, African countries have one thing in common – they all depend on aid.”  Er, and colonialism (except maybe Ethiopia, and then mostly on a technicality.  And don’t tell me about Liberia – for God’s sake, we carved the place out to resettle freed slaves).  And colonialism has a lot to do with what CAUSED the situations we now address with aid.
I cannot, for the life of me, understand how she is ignoring this.
p.40: Yes, I am skimming a bit here.  That first bit really killed me.  But here I can give her some credit for hammering the “democracy gives us development” crowd – at least that portion of the crowd who thinks the relationship is simple.  It is not, of course, and some of the new thinking on this examines how, for example, governments can make difficult decisions that balance needed reforms/changes and their electoral interests.  But sadly, much of the mainstream writing on the subject tends toward the simplistic.
p.42-43: OK, I am now uncomfortable with what seems to be a bit too much lauding of dictatorships.  Yeah, they produce great growth numbers, but growth is a means to an end . . . improving the human condition.  Dictatorships tend to create large tradeoffs in quality of life that seem, on balance, to have negative impacts on their populations.  Not a lot of Chileans think back on Pinochet as the good old days, you know?
p.44: Moyo is quite right – the timing of aid, and inappropriate aid, can do much more harm than good.  For example, having food aid arrive nine months after a famine (not all that uncommon), just as the new harvest comes in, crushes local food prices (oversupply of free food drives prices of locally-grown crops) and re-impoverishes the local farmers.  But this is not an inherent problem of aid – this is about timing, something people are well aware of, and trying to address.  Further, Moyo’s complaint about celebrities bringing mosquito nets to the continent, and thereby putting local producers out of buisiness – while valid – steps outside her definition of aid (government-to-government transfers) that she laid out earlier in the book.  Apparently her terms of reference are not stable.  Super.
p.46: Moyo does not know what I feel in my heart of hearts, despite her claims – I do think aid can work.  Her evidence against it has to do with aid’s impact on various economic indicators.  But this is just means to an end, and does not capture many of the benefits of aid in a clear manner (reduced illness means a better quality of life, and might be partially captured in a growing GDP via the extra days the individual can work . . . but maybe not very clearly).  This isn’t to say that aid is perfect.  Hell, I wrote a book arguing that we don’t really know what it is we are trying to fix in much of the world, so I have my issues with aid and development.  I just want an honest reading of their impacts and drawbacks.

Posted on by Ed 1

Liveblogging Dead Aid (Chapter 1)

Today, I begin an series of posts “live blogging” my reading of Dambisa Moyo’s Dead Aid. I had intended to read the book for some time, and over the weekend I finally was able to pick it up.  I got two chapters deep, felt deeply frustrated, and went back through to figure out why.  If I am frustrated, surely others are too.  So, over a series of posts (this is the first) I will offer my thoughts on Dead Aid as I read it.  Take them for what they are worth – I won’t correct the text, but I will raise concerns where I see them.  I am not doing this to tear anyone down – indeed, I see this exercise as an effort to either shore up the argument in this paper by cleaning up otherwise loose or problematic readings of development history and practice, or provide a clear basis for the rejection of the argument.  To that end, I hope that people will offer their own comments, argue with me, and argue with Moyo from a different perspective than my own . . . hopefully something good will come out of the mess.  So, away we go . . .
Chapter 1: The Myth of Aid
p.3 The book begins with the usual litany of positive developments and remaining challenges for Africa.  Fair enough, I have a bit of this at the outset of my book.  However, she ends this section by arguing that the reason Africa has not yet realized its potential has its roots in aid.  Ok, provocative.
p.7 Yikes, we are headed downhill almost right away, as Moyo defines aid.  She breaks aid into three types:

  • humanitarian/emergency aid (in response to disasters)
  • charity-based aid (disbursed by charitable organizations to people on the ground)
  • systematic aid (payments made directly to governments from other governments or multilateral institutions).

My issue with this typology is simple: it doesn’t clarify our understanding of aid, as the categories she uses overlap heavily: for example, humanitarian aid is often administered by charitable organizations, and may also consist of direct payments to governments.  Further, bilateral aid is often implemented through charitable organizations acting as implementing partners who conduct work on the ground – there does not seem to be any space for this sort of aid in her typology, or her analysis.   So, when Moyo then argues that the book is not concerned with emergency and charity-based aid, she is also (unwittingly) removing from play a lot of bilateral aid – a form of aid that she then reduces to concessional lending/granting. In short, it is not clear to me that Moyo actually understands the mechanics of aid and its implementation, which strikes me as a central part of any argument against it (or for it, for that matter).  We shall see how this plays out . . .
p.8 Ah, we finally come to the myth of aid (I think): a fundamental, pervasive mindset that aid, whatever its form, is a good thing.  Wait, what?  Really?  This strikes me as a very thin straw man, and it is supported by absolutely nothing.  It is a bald assertion about the “western mindset” that strikes me as oddly echoing the really embarrassing overgeneralized assertions about various African ethnicities on the part of early-to-mid 20th century ethnographers.  I’ll spare you the quotes.  Not only is this assertion embarrassing in a horribly ironic way, it is hardly the stuff of the central argument for a book like this.  Of course that attitude toward aid is a myth . . . it doesn’t really exist.  At least not anywhere of which I am aware.  It is really easy to prove something is a myth when nobody believes in it in the first place – which might have something to do with the success of this book: it is telling people something they already knew, which makes the reader feel good about themselves.

Posted on by Ed 2

Development isn't impossible, just hard to understand

A few comments on the blog related to some earlier posts on a Grand Challenge for Development have gotten me thinking a bit about development (the concept and the project) and if it is achievable.  There are those who would argue it is not, that development is an ill-conceived idea that invokes pathways of change that are now closed due to the changing global political economy, and treats life in the advanced economies as the apotheosis of human existence toward which everyone else is (and should be) marching.  To the extent development is taken to mean this sort of change, I agree completely – development is unattainable and meaningless.  There are not enough resources on Earth to allow everyone to live the way we do in the advanced economies, so the idea of a march toward that standard of living as a goal is gone regardless of how one might feel about it morally/ethically/etc.
But that does not mean that change cannot happen, that things cannot improve in a manner that is appreciated by people living in particular places.  Certainly, a shift from a post-subsistence income of $1 a day to $5 a day is a huge change that, in many parts of the world, would enable very different standards of health, education and well-being.  Surely this is worth striving for – and certainly, the people with whom I have worked in Ghana and Malawi would take that kind of a change over no change at all – and they would much rather than kind of change, than endless, pride-killing aid dependence. There is no doubt that this sort of change can be attained in many, if not most places.  Indeed, it has been accomplished.  Further, there are places where life expectancy has risen dramatically, infant mortality has fallen, nutrition and education levels have improved, and by any qualitative measure the quality of life has improved as a direct result of aid interventions (often termed development, but this should only count as development if the changes are sustained after the aid ends).  The real question at hand is not if it can be done, but why the results of our aid/development efforts are so erratic.
You see, for every case of improved life expectancy, there is the falling expectancies in Southern Africa.  For every case of improved nutrition and food availability, there are cases of increasing malnutrition and food insecurity (such that in sub-Saharan Africa, the balance has tipped toward less food availability per capita than two decades ago), and so on.  What works in one place often fails in another.  And the fact is that we don’t understand why this is in a systematic way.  I am a geographer and an anthropologist, so I am quite sympathetic to the argument that the local specificity of culture and society have a lot to do with the efficacy of particular interventions, and therefore explain a lot of the variability we see in project outcomes.  However, “local specificity” isn’t an answer, it is a blanket explanation that isn’t actionable in a specific way.  We persist in this answer because it pushes development (and aid) failure into the realm of the qualitative, the idiosyncratic.  And this attitude absolves us, the development community, from blame when things don’t work out.  Your project failed? Ah, well, who could have known that local land tenure rules would prevent the successful adoption of tree crops by women?  Subtly, we blame the victims with this mentality.
What it comes down to, I think, is a need to admit that we have at best a shaky idea of what works because in many areas (both geographic and technical) we really don’t understand what it is we are trying to transform when we engage in aid and development work.  We are better in some areas (health) because, frankly, they do a better job of gathering data and analyzing it than we do in, say, rural development (hey, don’t take my word for it – read some Robert Chambers, for heaven’s sake!).  But, in the end, we are driven by our myths about how markets and globalization work, how development/aid is linked to change, and how the problems we claim to address through development and aid came about in the first place.  This argument is the heart of my book (Amazon link here) – and I spend the first half using the story of two villages in Ghana to lay out how our assumptions about the world and how it works are mostly wrong, the next quarter explaining why this is a major problem for everything from economics to the environment, and the last quarter thinking about how to change things.
My take is but one take – and a partial one at that.  We need more people to think about our assumptions when we identify development challenges, design programs, and implement projects.  We need to replace assumptions with evidence.  And we need to be a lot more humble about our assumptions AND our evidence – so we stay open to new ideas and evidence as they inevitably flow in.

Posted on by Ed 3

Challenging development dogma

On his blog Shanta Devarajan, the World Bank Chief Economist for Africa, has a post discussing the debate about the performance and results of the Millennium Villages Project (MVP).  The debate, which takes shape principally in papers by Matt Clemens and Gabriel Demombynes of Center for Global Development and Paul Pronyk, John McArthur, Prabhjot Singh, and Jeffrey Sachs of the Millennium Villages Project, questions how the MVP is capturing the impacts of its interventions in the Millennium Villages.  As Devarajan notes, the paper by Clemens and Demombynes rightly notes that the MVP’s claims about its performance are not really that clearly framed in evidence, which makes it hard to tell how much of the changes in the villages can be attributed to their work, and how much is change driven by other factors.  Clemens and Demombynes are NOT arguing that the MVP has had no impact, but that there are ways to rigorously evaluate that impact – and when impact is rigorously evaluated, it turns out that the impact of MVP interventions is not quite as large as the project would like to claim.
This is not all that shocking, really – it happens all the time, and it is NOT evidence of malfeasance on the part of the MVP.  It just has to do with a simple debate about how to rigorously capture results of development projects.  But this simple debate will, I think, have long-term ramifications for the MVP.  As Devarajan points out:

In short, Clemens and Demombynes have undertaken the first evaluation of the MVP.  They have shown that the MVP has delivered sizeable improvements on some important development indicators in many of the villages, albeit with effects that are smaller than those described in the Harvests of Development paper.  Of course, neither study answers the question of whether these gains are sustainable, or whether they could have been obtained at lower cost.  These should be the subject of the next evaluation.

I do not, however, think that this debate is quite as minor as Devarajan makes it sound – and he is clearly trying to downplay the conflict here.  Put simply, the last last two sentences in the quote above are, I think, what has the MVP concerned – because the real question about MVP impacts is not in the here and now, but in the future.  While I have been highly critical of the MVP in the past, I am not at all surprised to hear that their interventions have had some measurable impact on life in these villages.  The project arrived in these villages with piles of money, equipment and technical expertise, and went to work.  Hell, they could have simply dumped the money (the MVP is estimated to cost about $150 per person per year) into the villages and you would have seen significant movement in many target areas of the MVP.  I don’t think that anyone doubts that the project has had a measurable impact on life in all of the Millennium Villages.
Instead, the whole point here is to figure out if what has been done is sustainable – that is the measure of performance here.  Anyone can move the needle in a community temporarily – hell, the history of aid (and development) is littered with such projects.  The hard part is moving the needle in a permanent way, or doing so in a manner that creates the processes by which lasting change can occur.  As I have argued elsewhere (and much earlier that in this debate), and as appears to be playing out on the ground now, the MVP was never conceptually framed in a way that would bring about such lasting changes.  Clemens and Demombynes’ work is important because it provides an external critique of the MVP’s claims about its own performance – and it is terrifying to at least some in the MVP, as external evaluations are going to empirically demonstrate that the MVP is not, and never was, a sustainable model for rural development.
While I would not suggest that Clemens and Demombynes’ approach to evaluation is perfect (indeed, they make no such claim), I think it is important because it is trying to move past assumptions to evidence.  This is a central call of my book – the MVP is exhibit A of a project founded on deeply problematic assumptions about how development and globalization work, and framed and implemented in a manner where data collection and evaluation cannot really question those assumptions . . . thus missing what is actually happening (or not happening) on the ground.  This might also explain the somewhat non-responsive response to Clemens and Demombynes in the Pronyk et al article – the MVP team is having difficulty dealing with suggestions that their assumptions about how things work are not supported by evidence from their own project, and instead of addressing those assumptions, are trying to undermine the critique at all costs.  This is not a productive way forward, this is dogma.  Development is many things, but if it is to be successful by any definition, it cannot be dogmatic.

Posted on by Ed 1