Ben Leo at ONE.org (formerly of CGD) put forth an intriguing proposal recently on Huffington Post Impact: It’s Time to Ask the World’s Poor What They Really Want.  In short, Ben is trying to argue that the current top-down definition of development goals, no matter how well-intentioned, is unlikely to reflect the views of the people these development goals are meant to benefit.

Hear, hear.  I made a similar point in Delivering Development. Actually, that sort of was one of the main points of the book.  See also my articles here and here.

But I am concerned that Leo is representing this effort a little too idealistically.  Just because we decide to ask people what they want doesn’t mean that we will really find out what they want.  Getting to this sort of information has everything to do with asking the right questions in the right way – there is no silver bullet for participation that will ensure that everyone’s voices will be heard.  To that end, what worries me here is that Ben does not explain exactly how ONE plans to develop the standardized survey they will put out there, or how exactly they will administer this survey.  So, here are a few preliminary questions for Ben and the ONE team:

1)   Does a standardized survey make sense? Given the very different challenges that people face around the world, and the highly variable capacity of people to deal with those challenges, it seems to me that going standardized is going to result in one of two outcomes: either you ask focused questions that only partially capture the challenges facing most people, or you ask really general questions that basically capture the suite of challenges we see globally, but do so in a manner that is so vague as to be unactionable.  How will ONE thread this needle?

2)   Who is designing the survey? To my point above, what questions are asked determine who will answer, and therefore determines what you will learn.  While the information gleaned from this sort of survey is likely to be very interesting, it is not the same thing as an open participatory process – full participation includes defining the questions, not just the answers.  Indeed, I would suggest that ONE needs to ditch the term participatory here, as in the end I fear it will be misleading.

3)   How will you administer the survey? Going out with enumerators takes a lot of time and money, and is subject to “investigator bias” – that is, the simple problem that some enumerators will do their job in a different manner than others, thus getting you different kinds/qualities of answers to the same questions.  On the other hand, if you are reliant on mobile technology, how will you incentivize those rural populations with mobile handsets to participate?  If you can’t do this, you will end up with a highly unrepresentative sample, making the results far less useful.

This is not to dismiss the effort Ben is spearheading – indeed, it is fantastic to see a visible organization make this argument and take concrete steps to actually get the voices of the global poor into the agenda-setting exercises.  However, this is not a participatory process – it is, instead, an information-driven process (which is good) that is largely shaped by the folks at ONE in the name of the global poor.  If ONE wants this to be more than information-driven, it needs to think about how it is going to let a representative sample of the global poor define the questions as well as the answers.  That is no easy task.

In all sincerity, I am happy to talk this through with anyone who is interested – I do think it is a good idea in principle, but execution is everything if you want it to be more than a publicity stunt…

18 September, 11:55pm Update: It seems I overlooked David Brooks’ piece in the New York Times today – all credit to him for picking up, rather explicitly, I think, on the themes below.  Even more credit to the center-right Brooks for eschewing politics to express his disturbance over Romney’s comments. Too bad the rest of the media seems to be ignoring the issues here.

+++++++++++++++++++++++++++++++++++++++

In the roar over Mitt Romney’s astonishingly impolitic (let’s just leave it at that for now) comments, it seems to me that the media, twitter…basically everyone has ignored one part of his statement:

All right, there are 47 percent who are with him, who are dependent upon government, who believe that they are victims, who believe the government has a responsibility to care for them, who believe that they are entitled to health care, to food, to housing, to you-name-it. That that’s an entitlement. And the government should give it to them. And they will vote for this president no matter what. (my emphasis).

Look, do we have entitlement issues in the United States?  Sure – hell, we have giant agricultural subsidies that are screwing up global markets, but heaven forbid anyone try to get rid of them.  And is it totally unreasonable to phase in a retirement age a year or two later over the next several decades, given rising life expectancies?  But this is one set of debates.  When you bring food, shelter and basic healthcare into play, you are not talking about the same issues.

I’m sorry, but is anyone in this country seriously arguing that having enough food to eat is not a right?  That access to shelter is not a right? That the hippocratic oath, and all that goes with it, is out the window? It is a national shame that anyone starves inside the boarders of this country.  It is an embarrassment that people die of exposure in this country. It is humiliating to have a higher infant mortality rate than CUBA. Politics aside, shouldn’t we, as members of the world’s largest economy, all agree that this sort of thing simply should not happen, and that it is our societal responsibility to prevent it from happening?

What religious creed, what political affiliation, what socially-acceptable philosophy argues that we should crap on the poor? None. There is no excuse for this sort of statement, this sort of argument – it is just wrong.  And the fact that everyone seems to be letting it slide because they are myopically focused on “the 47%” is equally inexcusable.  Given these comments, someone damn well better ask candidate Romney where he stands on social safety nets…

There has mixed response to my posts on disaster awareness among college students (well, the Horn of Africa drought among my current students) – see posts here and here.  Some see something hopeful and interesting in the idea that the students want more complex explanations for the problems they see.  Others are significantly more negative, suggesting that people such as my students are just symptomatic of a larger societal, if not species-level, lack of empathy for distant others.  I fall on the optimistic side of things, perhaps because I am a geographer.  Let me explain…

Geography, as a discipline, spends a hell of a lot of time thinking about how places are created and maintained.  Places are not locations (folks get this mixed up all the time) – places are our experiences of particular locations – at least this is how I choose to think about it.  And when you think of it that way, it becomes impossible to see life in a particular place as independent from events in other places.  The experience of living in Columbia, South Carolina is shaped by the weather, the cost of living, the infrastructure, the schools (I am a parent), etc.  But each of these is in turn shaped by other factors that transcend Columbia.  The cost of living and state of the infrastructure are intimately tied to the history of the state of South Carolina within the United States (where the South has historically been the underdeveloped agrarian other of the industrialized Northeast), but are also tied to the global economy. South Carolina is now the last stopping point for large-scale manufacturing before it heads out of the US to find the most favorable conditions of production possible.  The overseas shift of the textile industry wrought devastation on the state’s economy…and relatively few in the state seem able to come to grips with the fact they were ground up in the jaws of a new global economy that has already spit them out.  Even the weather is being reshaped by global factors that drive climate change, as a new regime of reduced rainfall seems to be settling in.  At what point do you stop calling a prolonged rainfall deficit a drought and start calling it the new normal?  Turns out about three decades. We are about 20 years into a significant decline in precipitation, so we are getting there.  Thus, the policy decisions (regarding industrial policy and emissions policy) of actors in China and India drive shifts in the economy and environment of the State of South Carolina.  We are thoroughly tied up in larger global forces here.  To understand South Carolina today, we have to understand the larger world today – there is simply no way around this.

As soon as this lesson settles in (and it can take a while), it becomes obvious that these forces flow both ways – that is, as Columbia, SC is constituted by global forces, so too what we do here in Columbia contributes to global forces that play out in other places.  Thus, when we vote for federal lawmakers who keep absurd ethanol subsidies in place no matter what the price/maize production conditions, we create a driver of food price increases that can radiate around the world.  And while we in Columbia feel those increases, when the price of a loaf of bread goes up by a dollar, most of us are inconvenienced and annoyed.  For someone who was already living on less than $2/day, this same price increase blows up their capacity to feed themselves.

All of this then goes back to my earlier point about what the students wanted – complex explanations.  The kids already get it, folks – they already understand an interconnected world (to some extent), and they mistrust oversimplified explanations.  When you feed them simple explanations, you often have to root out the interconnections that connect us to events in other parts of the world – the very things that students would grab on to.  In short, by oversimplifying things, we are making it harder for people to feel connected to the places in which things like famine happen.

The lesson: find yourself a geographer, work with them to tell the damn story in all its complex glory, and get out of the way.  The kids are waiting…

I am in the midst of (finally) reading James Scott’s The Art of Not Being Governed.  The book is a fascinating exercise in anarchist political geography – basically, examining how state power was limited by/shaped by various geographic factors in upland Southeast Asia.  As with many Scott books, it is a readably huge, sweeping view of a long timespan and a lot of diverse people, but thus far it hangs together well.

I was struck by a particular passage in chapter 3 that speaks rather directly to a lot of food security/agricultural development work being done in the world today, not least under Feed the Future.  The passage comes in a section where Scott is reviewing the historical efforts of various tenuous states to consolidate their control over the surrounding population, but contains an important message for those who are pushing the newest era of agricultural modernization (I call it part II, but there are compelling arguments for this being part III, or IV, depending on how you parse the history of development):

At about the same time as the Peloponnesian War, the early Chinese state was doing everything in its power to prevent the dispersal of population. Manuals of statecraft urged the king to prohibit subsistence activities in the mountains and wetlands “in order to increase the involvement of the people in the production of grain.” . . . The objective of this policy was, it seems, to starve the population into grain farming and subjecthood by separating them from the open commons.

There is no leap at all from this ancient Chinese case to what the inadvertent outcomes of much of our “efficiency-driven” food security and agricultural development work might do.  By compelling greater and greater market integration, and doing so through a focus on fewer and fewer crops, we are effectively closing the commons and prohibiting/constraining subsistence activities among the affected populations.  The result, in the best case, is improved agricultural outcomes and incomes that translate into improved well-being for all involved.  The worst case is a scenario where various marginal populations who have developed some expertise in managing the uncertainty of their particular contexts lose adaptive capacity, making them much more vulnerable to state violence and control.  Critically, these are not mutually exclusive scenarios – what works in one year or season to improve the quality of life for rural farmers might, in another year or season with different market and weather conditions, work to extend state control over marginal populations who already receive little for their status as citizens. I am sure that nobody who works on the donor side wants to be part of a campaign of state violence (at the worst) or part of a project that results in the further marginalization of poor, marginal populations (in the better to middle-case scenarios).  So, at least for Feed the Future, perhaps now would be a good time to call the Democracy, Human Rights and Governance folks (they’re just one floor up!) and have them take a look at this?

Not everyone wants a resilient population, folks.

The larger message here: incorporating people into agricultural markets is about much more than economic efficiency – there are much broader considerations, from state-level political economic issues to sub-household gender roles, that come into play when we radically rework existing agricultural systems and the livelihoods that go with them.  I seriously doubt we are doing enough to capture the wide suite of challenges that comes along with market engagement to ensure that our agricultural development programs are not enhancing vulnerability and stripping resilience from some of the most marginal people we are working with.  Output per hectare is not the only relevant metric. It’s probably not even the most relevant metric, given the fact we are not suffering from a production crisis at the global scale (despite what you hear from various outlets).

Incidentally, if you are at all interested in rural development, you should buy every book James Scott has ever written and get to reading.  Now.  Links below.

Weapons of the Weak is a classic – shame on you if you do rural development/community-based development and are not familiar with it. Or, shame on the people who taught you…

The Moral Economy of the Peasant is Scott’s most underrated text.

Domination and the Arts of Resistance is Scott’s first “meta” text – huge topic, giant sweep, really interesting.

Seeing Like A State shows up on everyone’s must read list…pretty much because you should read it.

So, given the twitter/blog/social media/whatever response to my post expressing shock at my students’ lack of awareness of the Horn of Africa drought, I did a little follow-up with them today.  This was the first day of real lecture content in the class, and as it happens one of the first examples I hit on (while trying to demonstrate the concept of interdependence – how every part of the world is inextricably linked to other parts of the world, for better or for worse) was the food price spikes of 2008 and 2011 (and the imminent spike coming this fall as a result of the drought that has devastated the US maize crop).  Since we were on food insecurity, I pivoted a bit and decided to just talk to them directly.  A summary, for those of you interested in how the hell a bunch of college students/college-bound high school students could have missed a crisis of this size:

1)   The crisis was horribly branded: I think talking about the Horn of Africa confused the few people who did know something had happened.  When I started casting about (around this time last year, Kenya, Ethiopia, Somalia, starving African babies…) a few students did remember seeing something on the news.  As one student put it, he saw it on a major network, but the anchor wasn’t reporting.  I suspect more of the students were briefly aware of the crisis at the time, but it has since been lost to time because of the sheer volume of calls for help/mentions of crisis to which they are exposed (see point #2).

2)   In general, the students disliked most current “disaster messaging.”  Yes, it grabs their attention…and then it overwhelms them.  First, there are a lot of bad things that happen, and therefore a lot of news stories/PSAs/etc. coming down the line all the time.  They become hard to differentiate, such that students just tune out the PSAs entirely.  Second, the messaging largely seems to be a competition to horrify people even more…but the explanations for the problem are simplistic or, worse, nonexistent.  The students don’t understand why the crisis is happening, and they are turned off by “solutions” that amount to “send me $5 and I will fix it.”  These are young, idealistic, energetic people – this particular constituency has a greater interest in acting directly than many others.  To summarize: screaming “IT’S A DISASTER!!! SEND ME MONEY TO FIX IT!!!” is rarely going to generate deep interest and engagement (and we need both, for a lot of reasons – see below).  Most messaging around the Horn was of this genre, and as a result it quickly receded into the daily noise of news feeds and celebrity weddings.

3)   Students (or at least some students) don’t need to be spoken down to – they can handle hearing that a crisis has complex causes, that it is often difficult to identify anyone who is to blame.  In short, they are looking for the opposite of the FWD campaign, which shied away from the really complex, big causes of the Horn crisis.  Complexity, unto itself, will not scare students off.  Instead, if you can get people to give clear, concise, interesting reviews of the complex causes of the crisis, this group of people will get more engaged.  Think about it – not everyone is into Africa, or into food security, or into relief work.  So when we yell “African famine!”, we are yelling to a small but dedicated fanbase.  If, however, we unpack the causes of the Horn crisis, we find out that we have to address climate change/climate science, global markets, the politics of failed states, the regional geopolitics of East Africa, the workings of the US Government, the international politics of aid, etc., etc.  In short, when we engage complexity, we find there is something that can draw in almost anyone on their terms.  After the conversation with the students today, it seems really clear to me that they would like to be engaged in this manner – stop treating them like apathetic idiots who just don’t/won’t understand.  Why?

• Crowdsourcing: folks, there is a big world outside the aid and development community, and some of those people actually have interesting ideas.  Maybe those ideas can only address part of one of the many causes of the crisis (i.e. adjusting a market’s function for one commodity in one part of the world), but with a lot of people acting in this manner, it becomes possible to identify a wide range of potential options to address a given crisis/prevent its recurrence.
• Politics: not one person in my classroom, or really any person anywhere who has a clean bill of mental health, wants to see 100,000 people die for any reason.  I believe that the vast majority of them would support spending tax dollars to prevent this from happening.  But when we fail to explain what needs to be done, in all its complexity, we are turning off a key constituency that can be mobilized and can make its voice heard – they have something that all politicians want. Votes. I can’t guarantee that my students would use those votes to shape policy, but they can’t do this until someone gives them a reasonable, actionable explanation for the events in the world that we would all like to address.

4)   Message management is anathema to social media: let me state the obvious – in the social media era, controlling the message is only possible if the message is so insipid that nobody cares about it at all.  A lot of the Horn messaging was about controlling the message, which is the equivalent of lecturing people via social media.  Ugh.  One student who wrote to me after class argued, more or less, that our role should be a catalyst for social media – we light the fire, but count on the fire to catch and build in its own way once it is started.  Social media that tries to message top-down, instead of evolving with a viral situation, will fail…it will be ignored.  I just realized what I am going to assign my students to do in my absence next week – I am going to make them follow a few official twitter feeds and critique them…oh, the horror!  This will be fun…

5)   Explain why the crisis at hand is important to their self-interest.  Yes, this sounds crass, but self-interest is a broad thing that can be mobilized with decent messages.  To pull an example from my own work, I can sell using development dollars on forest conservation because it has an important impact on the functioning of ecosystems that limit the pace of climate change – climate change that is raising sea levels along the South Carolina coast and producing drought across the state, and eventually will negatively impact the tourism industry in South Carolina (one of the few sectors here that is going well).  The students got that right away.  But nobody really did this for the Horn.  Which is pathetic. Hell, I did an off-the-cuff 2 minute explanation of why they care about the failed state in Somalia in terms of piracy in class today, by referencing the various ways in which piracy is raising shipping costs and therefore commodity prices…which hits their pocketbooks, impacts job growth, etc.  From there, it is easy to get into a reasoned conversation about the relative cost of the development and aid work that could change things in Somalia and end piracy as a viable livelihood versus doing nothing and bearing the cost of piracy.  It is all about entry points and catalysts, folks.

There were several other points that the students made – the one that sticks with me now is one student’s observation there is real experiential distance between their lives and what is happening in a famine that limits their engagement.  While we cannot bring students to a food crisis, we need to start thinking about how to create this experiential engagement.  For me, this happened when I became a parent…I will never again be able to objectively stomach an infant mortality statistic, because I flash to one or more of my children lying dead on the ground and I start to get the shakes.  I’m not sure what would do that for an 18-22 year old, but that sort of visceral connection spurs action.

To summarize: I think I was right in my initial post.  My students’ failure to recall the Horn of Africa crisis was not really their fault.  The messaging went awry in all sorts of ways because it assumed a lot about the audience (they had no interest in the issue, and only wanted simple stories with simple solutions) that was simply wrong.  Not everyone is going to care about every crisis – everyone has limited bandwidth – and so bad messaging just fell back into the everyday noise of social and old media, another data point among many, but nothing new or engaging.  Good messaging won’t make everyone care about every crisis, but it could engage enough of the right people each time to get us different outcomes, and fewer crises in the future.  That alone should make the effort worthwhile – so I guess I am disagreeing with J over at AidSource. Or the hopelessly realistic optimist in me is just winning out again…

Today, I reentered the classroom for the first time in two years.  That’s not completely accurate, actually – I lectured at the Foreign Service Institute several times while I was in DC, and I have a number of lectures, so I am not totally out of practice.  And after you’ve spent over 1000 hours (!!!) in front of a classroom, it really is like riding a bike…

Despite my classroom experience, I was seriously thrown by a moment in class today – I was discussing the different climates we see in East Africa, and mentioned the Horn of Africa famine in an offhand way…then realized there were too many blank stares.  So I asked the class directly how many of them were aware of the famine.  Not a single hand went up – 70 students, no hands.  Now, maybe someone put up a hand in that half-shrug, uncomfortable sort of way and I missed it.  And perhaps a few people had heard of the famine, but had not heard of it as something going on in the Horn of Africa.  But…at best, that is a few people.  Out of 70.

HOW THE HELL COULD THIS HAPPEN?  Somewhere between 50,000 and 100,000 people died in this famine – actually, that is a very low estimate, given that we were looking at 20,000-30,000 under-5 deaths in August 2011, and things stayed bad for quite a while after.  This is probably the single biggest human catastrophe since the Boxing Day Tsunami in 2004 (that killed 230,000 people).

I don’t blame the students.  Honestly.  They are wired in – they get all kinds of media all day long.  The simple fact is that the story of this famine was never sold very well, or very widely.  I thought the PSA campaign around the famine was terrible – a bunch of B-list celebrities, at best, in really dull clips (more on that in a later post).  Media coverage was confused.  Most could not separate drought from famine (which led me to write mymost-viewed post ever), attributing the causes completely to the weather.  Others played up the Somalia terrorism angle with al-Shabab, a heterogenous and not terribly effective fundamentalist group in Somalia that decided to turn itself into drone bait by aligning with al-Qaeda.  But the whole story was much more than could be compressed into 2 minutes on the nightly news.

That these students didn’t know about the famine is a lost opportunity – an opportunity to illustrate how complex the world is, how climate change compromises development efforts, how relief work is very hard, and very political, and how there are a hell of a lot of really heroic people doing amazing work that probably saved as many lives as were lost, if not many, many more.  These are the people who will become educated voters, who will shape America’s place in the world through who they elect and what sorts of priorities they express – and they have no idea that America has a tool like FEWS-NET, which now can predict when and where famine will break out months in advance in several African countries…this is an astonishing accomplishment, and the envy of the world.  And if the foreign aid cutters in Congress get their way, it could go away.

Maybe many more people paid attention to the famine on other campuses, in other states…but somehow, I have a feeling that my class was not all that much of an anomaly.  Simply put, we in the relief and development community suck at messaging.  Between the frantic and often disingenuous fundraising that imprint television viewers with the belief that the situation is hopeless, the confused media reporting as everyone looks for their unique angle, and the near-total failure of messaging from the donor institutions, it is no wonder my students were clueless – hell, they almost certainly knew about the famine, at least in passing, but the completely disjointed storytelling probably prevented any meaningful understanding of the causes of the events or how to address these causes and their impacts.

I have no idea how to fix this, but somebody has to fix this. It is too important to be lamented and then ignored in favor of “doing the work” of development and relief.  Messaging is the work of development and relief – telling the story of what we do, why it needs to be done, and how we could do less of it in the future if we just addressed some root causes now is fundamental to getting the societal buy-in we need to do our jobs right.  Somebody do this right.  I can only reach 70 people at a time…

A while back, I was musing about the end of IDA, especially given the interesting work of Andy Sumner on the “New Bottom Billion” and the increasing rate of country “graduation” from IDA eligibility.  For those unfamiliar with the term, IDA is the International Development Association, a branch of the World Bank Group focused on assistance for the world’s poorest countries.  Countries eligible to receive support from IDA have access to low- or no-interest loans that can be paid back over long periods of time – this is justified by the assumption that IDA-eligible countries will not have access to other forms of credit on reasonable terms, mostly because of conditions in the country that would drive away commercial lenders and other sources of credit.  Of course, IDA loans often came with significant conditionality – terms that recipients had to agree to in order to receive the funds, ideally intended to remedy the problems that kept traditional creditors away.  While these conditions were meant to help recipient countries, they often proved disastrous (structural adjustment had a lot of collateral damage among the people it was meant to help) and certainly challenged recipient country’s sovereignty, as the conditions effectively moved economic policy decision-making out of the national government and into multilateral donor organizations.  Both situations created a lot of tension between the multilaterals and the IDA-eligible countries.

Today, for a lot of reasons, it seems that IDA might be coming to a close in much of the world.  Countries are moving beyond the GNI per capita thresholds between low-income and middle-income status all the time (Ghana recently did so), and thus losing eligibility for IDA loans.  Further, ever-cheaper sources of capital have been extended to nearly every country in the world, a process that seems to have accelerated since the recession in the Global North (I surmise this has something to do with the fact that the giant pool of global money that no longer resides in real estate speculation needs somewhere else to go).  In any case, for most low income countries IDA is no longer the only credit game in town, and considering other sources of credit, especially the Chinese, put very few conditions on their loans, IDA is less attractive all the time.  There is still a need for IDA, for a very few countries, but it seems that the tide has turned against it.  This is good news – after all, in development our job should be to put ourselves out of business.  I think there is significant debate about whether IDA put itself out of business (considering attribution of graduation to IDA assistance is weak, at best), but at least we are seeing a situation where a development institution might finally be losing its purpose because good things have happened in the world.

Having worked for USAID, and now working with the World Bank, I have had the chance to sound out a lot of people in the donor world about this shift.  Most people I talk to who have given it any thought recognize that this is part of a much broader shift, one in which the days of “development as financial assistance” (IDA or otherwise) are coming to an end.  Going forward, it is clear that development assistance will increasingly be about technical assistance and less about lending (the Obama administration’s pick of Jim Kim as head of the World Bank is a clear signal that they see the World Bank of the future as a technical assistance organization, not a lending organization).  It is a fascinating transformation that, among other things, is going to obviate a lot of traditional critiques of aid, which revolved around the economic imperialism that aid dollars allowed – the conditionalities of lending that enabled structural adjustment and its many disastrous outcomes.

But the question of graduation is, of course, a tricky one.  Andy’s work, at least when I have been present for his presentations, has drawn questions about what seems to be the arbitrary nature of the lines between low-income and middle-income countries – that is, the movement to “middle income status” might represent a real shift in economic activity, but not a significant change in people’s lives.  I’ve expressed concerns about the robustness of “graduation”, as what appears to be a positive income trend in many countries might not have the strongest of foundations, or might be compromised by the impacts of climate change going forward.

So how can we productively track this change in the world’s financial needs and ensure that development takes a relevant shape as we seek to address the evolving needs of the world’s poor (wherever they might be)?  Perhaps another, more productive way to think about “graduation” is to examine the point at which investment dollars provided by donors is swamped by private investment such that the aid dollars are no longer effective means of leveraging change.  One million dollars can effectively leverage 10 million dollars of investment, but can 1 million dollars leverage 100 million dollars?  In most cases, probably not.  It would be interesting to look at various countries where aid dollars are of declining importance (Ghana, Zambia, etc.) and determine a) if their situation vis a vis aid investment has shifted such that donor dollars no longer effectively leverage other investments and b) when and under what conditions that shift occurred.  It seems to me that this would be a more effective marker in terms of thinking of transitioning from aid as investment to aid as technical assistance, as opposed to a largely arbitrary line in the GNI sand, one subject to rather large margins of error in its calculation.   Further, understanding the conditions of a transition from dependence on donor investment to stand-alone FDI recipient would be a key lesson for transitioning other countries out from aid dependence.

This is a rough sketch, but it seems to me this could be done.  It would require a bit of research in a couple of key places, but organizations like the World Bank are well-positioned to conduct such research and disseminate the findings.  Academia stands ready to help…

Alright, last post I laid out an institutional problem with M&E in development – the conflict of interest between achieving results to protect one’s budget and staff, and the need to learn why things do/do not work to improve our effectiveness.  This post takes on a problem in the second part of that equation – assuming we all agree that we need to know why things do/do not work, how do we go about doing it?

As long-time readers of this blog (a small, but dedicated, fanbase) know, I have some issues with over-focusing on quantitative data and approaches for M&E.  I’ve made this clear in various reactions to the RCT craze (see here, here, here and here). Because I framed my reactions in terms of RCTs, I think some folks think I have an “RCT issue.”  In fact, I have a wider concern – the emerging aggressive push for quantifiable data above all else as new, more rigorous implementation policies come into effect.  The RCT is a manifestation of this push, but really is a reflection of a current fad in the wider field.  My concern is that the quantification of results, while valuable in certain ways, cannot get us to causation – it gets us to really, really rigorously established correlations between intervention and effect in a particular place and time (thoughtful users of RCTs know this).  This alone is not generalizable – we need to know how and why that result occurred in that place, to understand the underlying processes that might make that result replicable (or not) in the future, or under different conditions.

As of right now, the M&E world is not doing a very good job of identifying how and why things happen.  What tends to happen after rigorous correlation is established is what a number of economists call “story time”, where explanation (as opposed to analysis) suddenly goes completely non-rigorous, with researchers “supposing” that the measured result was caused by social/political/cultural factor X or Y, without any follow on research to figure out if in fact X or Y even makes sense in that context, let alone whether or not X or Y actually was causal.  This is where I fear various institutional pushes for rigorous evaluation might fall down.  Simply put, you can measure impact quantitatively – no doubt about it.  But you will not be able to rigorously say why that impact occurred unless someone gets in there and gets seriously qualitative and experiential, working with the community/household/what have you to understand the processes by which the measured outcome occurred.  Without understanding these processes, we won’t have learned what makes these projects and programs scalable (or what prevents them from being scaled) – all we will know is that it worked/did not work in a particular place at a particular time.

So, we don’t need to get rid of quantitative evaluation.  We just need to build a strong complementary set of qualitative tools to help interpret that quantitative data.  So the next question to you, my readers: how are we going to build in the space, time, and funding for this sort of complementary work? I find most development institutions to be very skeptical as soon as you say the words qualitative…mostly because it sounds “too much like research” and not enough like implementation. Any ideas on how to overcome this perception gap?

(One interesting opportunity exists in climate change – a lot of pilot projects are currently piloting new M&E approaches, as evaluating impacts of climate change programming requires very long-term horizons.  In at least one M&E effort I know of, there is talk of running both quantitative and qualitative project evaluations to see what each method can and cannot answer, and how they might fit together.  Such a demonstration might catalyze further efforts…but this outcome is years away)

One of the things I have had the privilege to witness over the past two years is the movement of a large donor toward a very serious monitoring and evaluation effort aimed at its own programs.  While I know some in the development community, especially in academia, are skeptical of any new initiative that claims to want to do a better job of understanding the impact of programs, and learning from existing programs, what I saw in practice leads me to believe that this is a completely sincere effort with a lot of philosophical buy-in.

That said, there are significant barriers coming for monitoring and evaluation in development.  I’m not sure that those making evaluation policy fully grasp these barriers, and as a result I don’t see evidence that they are being effectively addressed by anyone.  Until they are, this sincere effort is likely to underperform, if not run aground.

In this post, I want to point out a huge institutional/structural problem for M&E: the conflict of interest that is created on the implementation side of things.  On one hand, donors are telling people that we need to learn about what works, and that monitoring and evaluation is not meant to be punitive, but part of a learning process to help all of us do our jobs better.  On the other hand, at most donors the budgets are under pressure, and the message from the top is that development must focus on “what works.”  Think about what this means to a mission director or a chief of party.  On one hand, they are told that M&E is about learning, and failure is now to be expected and can be tolerated as long as we learn why the failure occurred and can remedy the problem and prevent that problem in the future in other places.  On the other, they are told that budgets will focus on what works.  So if they set up rigorous M&E, they are likely to identify programs that are underperforming (and perhaps learn why)…but there is no guarantee that this learning won’t result in that program being cut, with a commensurate loss of staff and budget.  I have yet to see anyone meaningfully address this conflict of interest, and until someone figures out how to do so, there will be significant and creative resistance to the implementation of rigorous M&E.

Any ideas, folks?  Surely some of you have seen this at work…

Simply put, the donors are going to have to decide what is more important – learning what works, and improving on development’s 60+ year track record of spotty results with often limited correlation to programs and projects, or maintaining the appearance of efficiency and efficacy by cutting anything that does not seem to work, and likely throwing out a lot of babies with the bathwater. I know which one I would choose.  It remains unclear where the donors’ choices will fall.  In a politically challenging environment, the pressure to go with the latter approach is high, and the protection of a learning agenda that will really change how development works will require substantial political courage.  That courage exists…but whether or not it comes to the fore is a different question.

Ah, I have to play catch up…the things I have heard that I wanted to comment on but could not over the past few months…

Back in June, USAID hosted the Frontiers in Development Conference.  I was fortunate to attend the conference as an author of a chapter in the associated conference publication.  Overall, it was an interesting but worthwhile couple of days.

That said, the opening day was marked by what I saw as a really troubling comment by Senator Dick Lugar, who generally speaking is a big supporter of foreign aid and USAID.

While foreign assistance investments often require significant time before demonstrating impacts, funding should flow to programs that demonstrate results. I raise this point, because a percentage of foreign assistance funding to some countries is moving away from traditional purposes — including education, food security, and disease prevention — toward climate change.

I have expressed concerns about individual USAID climate change projects and the growing share of these projects within our development budget . . . My concern simply is that climate change projects are among the least likely to offer measurable development results and the most likely to be politically motivated.

Here is the problem – Senator Lugar’s comments fundamentally misconstrue what climate change funding does in development.  It is not funding “in place of” existing programs, nor is climate change programming meant to be stand-alone.  Climate change programs ensure the enduring impact of the work that USAID and other development organizations are doing – for example, there is little point to building a road between a food surplus and a food deficit area to facilitate trade if the food surplus area is likely to stop producing a surplus in the near future.  Climate change programs and projects are new, and mostly focused on learning about how to integrate climate sensitivity into development programs and projects – but again, these are not meant to be stand-alone programs and projects.

So how could Senator Lugar come to the conclusion that climate change programs were somehow supplanting other development activities, aside from an overly-brief reading of the USAID budget request?  There are two options: either the Senator, who is retiring and therefore does not need to worry much about what his constituency thinks, has decided to pander to the climate-skeptical in his state (not terribly likely), or organizations like USAID are doing an absolutely terrible job messaging their climate change work to the legislative branch.  The evidence points toward the latter, which is horrible – it is an easy sell, even to fiscal conservatives, when you can demonstrate how climate sensitivity makes programs and projects more effective, and increase the duration of the impact of the dollars that are spent in the foreign aid budget.

(Incidentally, this sort of work is going on at every major donor right now, so if USAID doesn’t have great examples to pull from yet – most programs and projects are very, very new – there are plenty out there from other donors that could make this point).

C’mon, folks, let’s get the messaging right.  Or at least better.  We shouldn’t be in a place where a supporter of foreign aid and development is questioning these budgets.