OK, ok, you say: I get it, global environmental change matters to development/aid/relief.  But aside from thinking about project-specific intersections between the environment and development/aid/relief, what sort of overarching challenges does global environmental change pose to the development community?  Simply put, I think that the inevitability of various forms of environmental change (a level of climate change cannot be stopped now, certain fisheries are probably beyond recovery, etc.) over the next 50 or so years forces the field of development to start thinking very differently about the design and evaluation of policies, programs, and projects . . . and this, in turn, calls into question the value of things like randomized control trials for development.
In aid/development we tend to be oriented to relatively short funding windows in which we are supposed to accomplish particular tasks (which we measure through output indicators, like the number of judges trained) that, ideally, change the world in some constructive manner (outcome indicators, like a better-functioning judicial system).  Outputs are easier to deliver and measure than outcomes, and they tend to operate on much shorter timescales – which makes them perfect for end-of-project reporting even though they often bear little on the achievement of the desired outcomes that motivated the project in the first place (does training X judges actually result in a better functioning judicial system?  What if the judges were not the problem?).  While there is a serious push in the development community to move past outputs to outcomes (which I generally see as a very positive trend), I do not see a serious conversation about the different timescales on which these two sorts of indicators operate.  Outputs are very short-term.  Outcomes can take generations.  Obviously this presents significant practical challenges to those who do development work, and must justify their expenditures on an annual basis.
This has tremendous implications, I think, for development practice in the here and now – especially in development research.  For example, I think this pressure to move to outcomes but deliver them on the same timescale as outputs has contributed to the popularity of the randomized control trials for development (RCT4D) movement.  RCT4D work gathers data in a very rigorous manner, and subjects it to interesting forms of quantitative analysis to determine the impact of a particular intervention on a particular population.  As my colleague Marc Bellemare says, RCTs establish “whether something works, not how it works.”
The vast majority of RCT4D studies are conducted across a few months to years, directly after the project is implemented.  Thus, the results seem to move past outputs to impacts without forcing everyone to wait a very long time to see how things played out.  This, to me, is both a strength and a weakness of the approach . . . though I never hear anyone talking about it as a weakness.  The RCT4D approach seems to suggest that the evaluation of project outcomes can be effectively done almost immediately, without need for long-term follow-up.  This sense implicitly rests on the forms of interpretation and explanation that undergird the RCT4D approach – basically, what I see as an appallingly thin approach to the interpretation of otherwise interesting and rigorously gathered data. My sense of this interpretation is best captured by Andrew Gelman’s (quoting Fung) use of the term “story time”, which he defines as a “pivot from the quantitative finding to the speculative explanation.” It seems that many practitioners of RCT4D seem to think that story time is unavoidable . . . which to me reflects a deep ignorance of the concerns for rigor and validity that have existed in the qualitative research community for decades.  Feel free to check the methods section of any of my empirically-based articles (i.e. here and here): they address who I interviewed, why I interviewed them, how I developed interview questions, and how I knew that my sample size had grown large enough to feel confident that it was representative of the various phenomena I was trying to understand.  Toward the end of my most recent work in Ghana, I even ran focus groups where I offered my interpretations of what was going on back to various sets of community members, and worked with them to strengthen what I had right and correct what I had wrong.  As a result, I have what I believe is a rigorous, highly nuanced understanding of the social causes of the livelihoods decisions and outcomes that I can measure in various ways, qualitative and quantitative, but I do not have a “story time” moment in there.
The point here is that “story time”, as a form of explanation, rests on uncritical assumptions about the motivations for human behavior that can make particular decisions or behaviors appear intelligible but leave the door open for significant misinterpretations of events on the ground.  Further, the very framing of what “works” in the RCT4D approach is externally defined by the person doing the evaluation/designing the project, and is rarely revised in the face of field realities . . . principally because when a particular intervention does not achieve some externally-defined outcome, it is deemed “not to have worked.”  That really tends to shut down continued exploration of alternative outcomes that “worked” in perhaps unpredictable ways for unexpected beneficiaries.  In short, the RCT4D approach tends to reinforce the idea that development is really about delivering apolitical, technical interventions to people to address particular material needs.
The challenge global environmental change poses to the RCT4D randomista crowd is that of the “through ball” metaphor I raised in my previous post.  Simply put, identifying “what works” without rigorously establishing why it worked is broadly useful if you make two pretty gigantic assumptions: First, you have to assume that the causal factors that led to something “working” are aspects of universal biophysical and social processes that are translatable across contexts.  If this is not true, an RCT only gives you what works for a particular group of people in a particular place . . . which is not really that much more useful than just going and reading good qualitative ethnographies.  If RCTs are nothing more than highly quantified case studies, they suffer from the same problem as ethnography – they are hard to aggregate into anything meaningful at a broader scale.  And yes, there are really rigorous qualitative ethnographies out there . . .
Second, you have to assume that the current context of the trial is going to hold pretty much constant going forward.  Except, of course, global environmental change more or less chucks that idea for the entire planet.  In part, this is because global environmental change portends large, inevitable biophysical changes in the world.  Just because something works for improving rain-fed agricultural outputs today does not mean that the same intervention will work when the enabling environmental conditions, such as rainfall and temperature, change over the next few decades.  More importantly, though, these biophysical changes will play out in particular social contexts to create particular impacts on populations, who will in turn develop efforts to address those impacts. Simply put, when we introduce a new crop today and it is taken up and boosts yields, we know that it “worked” by the usual standards of agricultural development and extension.  But the take-up of new crops is not a function of agricultural ecology – there are many things that will grow in many places, but various social factors ranging from the historical (what crops were introduced via colonialism) to gender (who grows what crops and why) are what lead to particular farm compositions.  For example, while tree crops (oil palm, coconut, various citrus, acacia for charcoal) are common on farms around the villages in which I have worked in Ghana, almost none of these trees are found on women’s farms.  The reasons for this are complex, and link land tenure, gender roles, and household power relations into livelihoods strategies that balance material needs with social imperatives (for extended discussions, see here and here, or read my book).
Unless we know why that crop was taken up, we cannot understand if the conditions of success now will exist in the future . . . we cannot tell if what we are doing will have a durable impact.  Thus, under the most reliable current scenario for climate change in my Ghanaian research context, we might expect the gradual decline in annual precipitation, and the loss of the minor rainy season, to make tree crops (which tend to be quite resilient in the face of fluctuating precipitation) more and more attractive.  However, tree crops challenge the local communal land tenure system by taking land out of clan-level recirculation, and allowing women to plant them would further challenge land tenure by granting them direct control over access to land (which they currently lack).  Altering the land tenure system would, without question, set off a cascade of unpredictable social changes that would be seen in everything from gender roles to the composition of farms.  There is no way to be sure that any development intervention that is appropriate to the current context will be even functional in that future context.  Yet any intervention we put into place today should be helping to catalyze long-term changes . . .
Simply put: Global environmental change makes clear the limitations of our current thinking on aid/development (of which RCT4D is merely symptomatic).   Just like RCTs, our general framing of development does not move us any closer to understanding the long-term impact of our interventions.  Further, the results of RCTs are not generalizable past the local context (which most good randomistas already know), limiting their ability to help us transform how we do development.  In a world of global environmental change, our current approaches to development just replicate our existing challenges: they don’t really tell us if what we are doing will be of any lasting benefit, or even teach us general lessons about how to deliver short-term benefits in a rigorous manner.
 
Next up: The Final Chapter – Fixing It