Open The Echo Chamber

I have a confession. For a long time now I have found myself befuddled by those who claim to have identified the causes behind observed outcomes in social research via the quantitative analysis of (relatively) large datasets (see posts here, here, and here).  For a while, I thought I was seeing the all-to-common confusion of correlation and causation…except that a lot of smart, talented people seemed to be confusing correlation with causation.  This struck me as unlikely.

Then, the other day in seminar (I was covering for a colleague in our department’s “Contemporary Approaches to Geography” graduate seminar, discussing the long history of environmental determinism within and beyond the discipline), I found myself in a similar discussion related to explanation…and I think I figured out what has been going on.  The remote sensing and GIS students in the course, all of whom are extraordinarily well-trained in quantitative methods, got to thinking about how to determine if, in fact, the environment was “causing” a particular behavior*. In the course of this discussion, I realized that what they meant by “cause” was simple (I will now oversimplify): when you can rule out/control for the influence of all other possible factors, you can say that factor X caused event Y to happen.  Indeed, this does establish a causal link.  So, I finally get what everyone was saying when they said that, via well-constructed regressions, etc., one can establish causality.

So it turns out I was wrong…sort of. You see, I wasn’t really worried about causality…I was worried about explanation. My point was that the information you would get from a quantitative exercise designed to establish causal relationships isn’t enough to support rigorous project and program design. Just because you know that the construction of a borehole in a village caused girl-child school attendance to increase in that village doesn’t mean you know HOW the borehole caused this change in school attendance to happen.  If you cannot rigorously explain this relationship, you don’t understand the mechanism by which the borehole caused the change in attendance, and therefore you don’t really understand the relationship. In the “more pure” biophysical sciences**, this isn’t that much of a problem because there are known rules that particles, molecules, compounds, and energy obey, and therefore under controlled conditions one can often infer from the set of possible actors and actions defined by these rules what the causal mechanism is.

But when we study people it is never that simple.  The very act of observing people’s behaviors causes shifts in that behavior, making observation at best a partial account of events. Interview data are limited by the willingness of the interviewee to talk, and the appropriateness of the questions being asked – many times I’ve had to return to an interviewee to ask a question that became evident later, and said “why didn’t you tell me this before?”  (to which they answer, quite rightly, with something to the effect of “you didn’t ask”).  The causes of observed human behavior are staggeringly complex when we get down to the real scales at which decisions are made – the community, household/family, and individual. Decisions may vary by time of the year, or time of day, and by the combination of gender, age, ethnicity, religion, and any other social markers that the group/individual chooses to mobilize at that time.  In short, just because we see borehole construction cause increases in girl-child school attendance over and over in several places, or even the same place, doesn’t mean that the explanatory mechanism between the borehole and attendance is the same at all times.

Understanding that X caused Y is lovely, but in development it is only a small fraction of the battle.  Without understanding how access to a new borehole resulted in increased girl-child school attendance, we cannot scale up borehole construction in the context of education programming and expect to see the same results.  Further, if we do such a scale-up, and don’t get the same results, we won’t have any idea why.  So there is causality (X caused Y to happen) and there are causal mechanisms (X caused Y to happen via Z – where Z is likely a complex, locally/temporally specific alignment of factors).

Unfortunately, when I look at much quantitative development research, especially in development economics, I see a lot of causality, but very little work on causal mechanisms that get us to explanation.  There is a lot of story time, “that pivot from the quantitative finding to the speculative explanation.”  In short, we might be programming development and aid dollars based upon evidence, but much of the time that evidence only gets us part of the way to what we really need to know to really inform program and project design.

This problem is avoidable –it does not represent the limits of our ability to understand the world. There is one obvious way to get at those mechanisms – serious, qualitative fieldwork.  We need to be building research and policy teams where ethnographers and other qualitative social scientists learn to respect the methods and findings of their quantitative brethren such that they can target qualitative methods at illuminating the mechanisms driving robust causal relationships. At the same time, the quantitative researchers on these teams will have to accept that they have only partially explained what we need to know when they have established causality through their methods, and that qualitative research can carry their findings into the realm of implementation.

The bad news for everyone…for this to happen, you are going to have to pick your heads up out of your (sub)disciplinary foxholes and start reading across disciplines in your area of interest.  Everyone talks a good game about this, but when you read what keeps getting published, it is clear that cross-reading is not happening.  Seriously, the number of times I have seen people in one field touting their “new discoveries” about human behavior that are already common conversation in other disciplines is embarrassing…or at least it should be to the authors. But right now there is no shame in this sort of thing, because most folks (including peer reviewers) don’t read outside their disciplines, and therefore have no idea how absurd these claims of discovery really are. As a result, development studies gives away its natural interdisciplinary advantage and returns to the problematic structure of academic knowledge and incentives, which not only enable, but indeed promote narrowly disciplinary reading and writing.

Development donors, I need a favor. I need you to put a little research money on the table to learn about whatever it is you want to learn about. But when you do, I want you to demand it be published in a multidisciplinary development-focused journal.  In fact, please start doing this for all of your research-related money. People will still pursue your money, as the shrinking pool of research dollars is driving academia into your arms. Administrators like grant and contract money, and so many academics are now being rewarded for bringing in grants and contracts from non-traditional sources (this is your carrot). Because you hold the carrot, you can draw people in and then use “the stick” inherent in the terms of the grant/contract to demand cross-disciplinary publishing that might start to leverage change in academia. You all hold the purse, so you can call the tune…

*Spoiler alert: you can’t.  Well, you probably can if 1) you pin the behavior you want to explain down to something extraordinarily narrow, 2) can limit the environmental effect in question to a single independent biophysical process (good luck with that), and 3) limit your effort to a few people in a single place. But at that point, the whole reason for understanding the environmental determinant of that behavior starts to go out the window, as it would clearly not be generalizable beyond the study. Trust me, geography has been beating its head against this particular wall for a century or more, and we’ve buried the idea.  Learn from our mistakes.

**by “more pure” I am thinking about those branches of physics, chemistry, and biology in which lab conditions can control for many factors. As soon as you get into field sciences, or starting asking bigger questions, complexity sets in and things like causality get muddied in the manner I discuss below…just ask an ecologist.

Alright, last post I laid out an institutional problem with M&E in development – the conflict of interest between achieving results to protect one’s budget and staff, and the need to learn why things do/do not work to improve our effectiveness.  This post takes on a problem in the second part of that equation – assuming we all agree that we need to know why things do/do not work, how do we go about doing it?

As long-time readers of this blog (a small, but dedicated, fanbase) know, I have some issues with over-focusing on quantitative data and approaches for M&E.  I’ve made this clear in various reactions to the RCT craze (see here, here, here and here). Because I framed my reactions in terms of RCTs, I think some folks think I have an “RCT issue.”  In fact, I have a wider concern – the emerging aggressive push for quantifiable data above all else as new, more rigorous implementation policies come into effect.  The RCT is a manifestation of this push, but really is a reflection of a current fad in the wider field.  My concern is that the quantification of results, while valuable in certain ways, cannot get us to causation – it gets us to really, really rigorously established correlations between intervention and effect in a particular place and time (thoughtful users of RCTs know this).  This alone is not generalizable – we need to know how and why that result occurred in that place, to understand the underlying processes that might make that result replicable (or not) in the future, or under different conditions.

As of right now, the M&E world is not doing a very good job of identifying how and why things happen.  What tends to happen after rigorous correlation is established is what a number of economists call “story time”, where explanation (as opposed to analysis) suddenly goes completely non-rigorous, with researchers “supposing” that the measured result was caused by social/political/cultural factor X or Y, without any follow on research to figure out if in fact X or Y even makes sense in that context, let alone whether or not X or Y actually was causal.  This is where I fear various institutional pushes for rigorous evaluation might fall down.  Simply put, you can measure impact quantitatively – no doubt about it.  But you will not be able to rigorously say why that impact occurred unless someone gets in there and gets seriously qualitative and experiential, working with the community/household/what have you to understand the processes by which the measured outcome occurred.  Without understanding these processes, we won’t have learned what makes these projects and programs scalable (or what prevents them from being scaled) – all we will know is that it worked/did not work in a particular place at a particular time.

So, we don’t need to get rid of quantitative evaluation.  We just need to build a strong complementary set of qualitative tools to help interpret that quantitative data.  So the next question to you, my readers: how are we going to build in the space, time, and funding for this sort of complementary work? I find most development institutions to be very skeptical as soon as you say the words qualitative…mostly because it sounds “too much like research” and not enough like implementation. Any ideas on how to overcome this perception gap?

(One interesting opportunity exists in climate change – a lot of pilot projects are currently piloting new M&E approaches, as evaluating impacts of climate change programming requires very long-term horizons.  In at least one M&E effort I know of, there is talk of running both quantitative and qualitative project evaluations to see what each method can and cannot answer, and how they might fit together.  Such a demonstration might catalyze further efforts…but this outcome is years away)

I will be speaking about my book and research at the University of Florida on Friday as part of the Glen R. Anderson Visiting Lectureship.  Poster here:

Hope to see folks there!

I’ll be running my mouth about the book again at Chatham University on December 2nd.  Chatham has some very cool stuff going in sustainability and the environment (a new school!), including a new Eden Hall Campus in Richland Township, PA.  My talk will actually be out on that campus, and not in the Shadyside campus . . . directions are here.

The flyer (they’ve done a nice job on it):

Hope to see some of you there . . .

I’ve made a few changes to my personal homepage (www.edwardrcarr.com).  This included cleaning up a few things, adding a few book reviews for Delivering Development, and updating my CVs.  However, today, for the first time since I set my homepage up, I have added a page . . . there is now a page for pre-prints.  I have become thoroughly fed up with the gatekeeping and slow pace of academic publishing – I was annoyed to start with, but after more than a year in an agency, and about 18 months engaged with a much wider environment/development community via the blog and twitter, I have come to realize that academic publishing, for all its rigor and legitimacy, is something of a liability.  There is no way anyone is going to wait around for my work, or anyone else’s work, to wend its way through peer review and the inevitable publication delays before it appears in print.

To address this, I am now posting work that I have submitted for review – it is polished, and sometimes it has seen a round of peer review already (those will be marked revised and resubmitted).  However, they are not fully finished, peer-approved work – which means they will likely change a little before they come out in final form.  My goal is to make this stuff available more or less as soon as I submit it.  I am open to comments and suggestions – I can still work them in before the final version goes out!

Some of you might wonder how this could affect the idea of double-blind peer review.  Well, in my experience, double-blind peer review in development studies – or indeed in any of the qualitative social sciences – is largely a joke.  In my field, we tend to invest a lot of time and effort working in a particular place, and so it is very, very easy to figure out who is writing about what.  I often know who the author of a piece is as soon as I read the abstract – and there are always enough details in any manuscript to facilitate a quick Google search that will identify the author.  Both pieces that I currently have on my website work from material for which I am well-known within my field.  For example, just mentioning the villages of Dominase and Ponkrum in Ghana in the livelihoods piece pretty much tells everyone who it is.  And the piece on academic engagement with development practice comes directly from a panel at last year’s Association of American Geographers Annual Meeting which was attended by more than 100 people, as well as an extended listserv exchange in the fall of 2010 that was sent out to several thousand subscribers of various lists.  Again, pretty much everyone will be able to figure out who wrote it.

So, the work is now up there for your perusal.  Have a look, and let me know what you think . . .

So, it seems I have been challenged/called out/what-have-you by the folks at Imagine There Is No . . . over what I would do (as opposed to critique) about development.  At least I think that is what is going on, given that I received this tweet from them:

@edwardrcarr what would You do with 1 Billion $ for #development? bit.ly/rQrUOd #The.1.Bill.$.Question

In general, I think this is a fair question.  Critique is nice, but at the end of the day I strive to build something from my critiques.  As I tell my grad students, I can train a monkey to take something apart – there isn’t much talent to that.  On the other hand, rebuilding something from whatever you just dismantled actually requires talent.  I admit to being a bit concerned about calling what I build “better”, mostly because such judgments gloss over the fact that any development intervention produces winners and losers, and therefore even a “better” intervention will probably not be better for someone.  I prefer to think about doing things differently, with an eye toward resolving some of the issues that I critique.

So, I will endeavor to answer – but first I must point out that asking someone what s/he would do for development with $1 billion is a very naive question.  I appreciate its spirit, but there isn’t much point to laying down a challenge that has little alignment with how the world works.  I think this is worth pointing out in light of the post on Imagine There Is No . . ., as they seem to be tweaking Bill Easterly for not having a good answer to their question.  However, for anyone who has ever worked for a development agency, the question “on what would you spend a billion dollars” comes off as a gotcha question because it is sort of nonsensical.  While the question might be phrased to make us think about an ideal world, those of us engaged in the doing of development who take its critique and rethinking seriously immediately start thinking about the sorts of things that would have to happen to make spending $1 billion possible and practical.  Those problems are legion . . . and pretty much any answer you give to the question is open to a lot of critique, either from a practical standpoint (great idea that is totally impractical) or from the critique side (and idea that is just replicating existing problems).  When caught in a no-win situation, the best option is not to answer at all.  Sure, we should imagine a perfect world (after all, according to A World Of Difference, I am “something of a radical thinker”), but we do not work in that world – and people live in the Global South right now, so anything we do necessarily must engage with the imperfections of the now even as we try to transcend them.

Given all of this, I offer the following important caveats to my answer:

1) I am presuming that I will receive this money as individual and not as part of any existing organization, as organizations have structures, mandates and histories that greatly shape what they can do.

2) I am presuming that I have my own organization, and that it already has sufficient staff to program $1 billion dollars – so a lot of contracting officers and lawyers are in place.  Spending money is a lot harder than you’d think.

3) I am presuming that I answer only to myself and the folks in the Global South.  Monitoring and evaluation are some of the biggest constraints on how we do development today.  As I said in my talk at SAIS a little while ago, it is all well and good to argue that development merely catalyzes change in complex systems, which makes its outcomes inherently unpredictable.  It is entirely another to program against that understanding – if the possible outcomes of a given intervention are hard to predict, how do you know which indicators to choose?  How can you build an evaluation system that allows you to capture unintended positive and negative outcomes as the project matures without looking like you are fudging the numbers?  This sounds like constrained thinking, but it is reality for anyone working in a big donor agency, and for all of the folks who implement the work of those agencies.

4) I am presuming there are enough qualified staff out there willing to quit what they are doing and come work for this project . . . and I am going to need a hell of a lot of staff.

5) I am presuming that I am expected to accomplish something in the relatively short term – i.e. 3-5 years, as well as trigger transformative changes in the Global South over the long haul.  If you don’t produce some results relatively soon, people will bail out on you.

All of these, except for 5), are giant caveats that basically divorce the question and its answer from reality.  I just need to point that out.  Because of these caveats, my answer here cannot be interpreted as a critique of my current employer, or indeed any other development organization – an answer that would also serve as a critique of those institutions would have to engage with their realities, blowing out a lot of my caveats above . . . sorry, but that’s reality, and it is really important to acknowledge the limits of any answer to such a loaded question.

So, here goes.  If I had $1 billion, I would spend it 1) figuring out what people really do to manage the challenges they face day-to-day, 2) identifying which of these activities are most effective at addressing those challenges and why, 3) evaluating whether any of these activities can be brought to scale or introduced to new places, and 4) bringing these ideas to scale.

Basically, I would spend $1 billion dollars on the argument “the new big idea is no more big ideas.”

Why would I do this, and do it this way?  Well, I believe that in a general way those of us working in development have very poor information about what is actually happening in the Global South, in the places where the challenges to human well-being are most acute.  We have a lot of assumptions about what is happening and why, but these are very often wrong.  I wrote a whole book making this point – rather convincingly, if some of the reviews are to be believed.  Because we don’t know what is happening, and our assumptions are wide of the mark, a lot of the interventions we design and implement are irrelevant (at best) or inappropriate (at worst) to the intended beneficiaries.  Basically, the claim (a la Sachs and the Millennium Villages Project) that there are proven development interventions is crap.  If we had known, proven interventions WE WOULD BE USING THEM.  To assume otherwise is to basically slander the bulk of people working on development as either insufficiently motivated (if we weren’t so damn lazy, and we really cared about poor people, we could fix all of the problems in the world with these proven interventions) or to argue that there simply needs to be more money spent on these interventions to fix everything (except in many cases there is little evidence that funding is the principal cause of project failure).  Of course, this is exactly what Sachs argues when asking for more support for the MVP, or when he is attacking anyone who dares critique the project.

The only way to really know what is happening is to get out there and talk to people.  When you do, what you find is that the folks we classify as the “global poor” are hardly helpless.  They are remarkably capable people who make livings under very difficult circumstances with very little resource and limited fallback options.  They know their environments, their economy, and their society far better than anyone from the outside ever will.  They are, in short, remarkable resources that should be treated as treasured repositories of human knowledge, not as a bunch of children who can’t work things out for themselves.  $1 billion would get us a lot of people in a lot of places doing a lot of learning . . . and this sort of thing can be programmed to run over 6 months to a year to run fieldwork, do some data analysis, and start producing tailored understandings of what works and why in different places . . . which then makes it relatively easy to start identifying opportunities for scale-up.  Actually, the scale-up could be done really easily, and could be very responsive to local needs, if we would just set up a means of letting communities speak to one another in a free and open manner – a network that let people in the Global South ask each other questions, and offer their answers and solutions, to one another.  Members of this project from the Global North, from the Universities and from development organizations, could work with communities to convey the lessons the project has gleaned from various activities in various places to help transfer ideas and technology in a manner that facilitates their productive introduction in new contexts.  So I suppose I would have to carve part of the $1 billion off for that network, but it would come in under the scale-up component of my project.  Eventually, I suspect this sort of network would also become a means of learning about what is happening in the Global South as well . . .

With any luck at all, by year 3 we would see the cross-fertilization of all kinds of locally-appropriate ideas and technology happening around the world and the establishment of a nascent network that could build on this momentum to yield even more information about what people are already doing, and what challenges they really face.  We would have started a process that has immediate impacts, but can work in tandem with the generational timescales of social change that are necessary to bring about major changes in any place.  We would have started a process that likely could not be stopped.  How it would play out is anyone’s guess . . . but it would sure look different than whatever we are doing now.

Whenever you write something, you hope that other people will like it . . . or perhaps hate it so much it spurs them to do something useful in response.  In any case, you want feedback.  A vast, echoey silence just sucks.  I have a weird version of this with my own academic work.  More often than not, I write things that land in the literature with a huge thud.  One or two people notice, read and cite it in the first two or so years it is out . . . and then all of a sudden lots of people start citing it in all kinds of places, ranging from academic journals to UN Reports.  This has become a pretty regular pattern for me, which to some extent reflects the fact that I have a habit of writing stuff on the edges of my discipline(s), and also reflects how long it takes new ideas to get into people’s work and show up in print (generally speaking, it takes between 9 months and a year, at least, from the acceptance of an article to its appearance in print – so any new idea has to be read, processed and incorporated into a new article, which takes a few months.  Then the article has to be accepted, and review typically takes 3-6 months.  Finally, after it is accepted, another 9-12 month wait.  Add it up, and you realize that it takes anywhere from 14-24 months for the first people who read a new idea to start responding in print).

Delivering Development has been a little different, as it is being reviewed in different kinds of venues – a lot of blog attention, for example.  I also had the good fortune of having two people review the piece for the back cover, so I got some feedback before the book even came out.  In any case, the reviews are now starting to flow in, and overall they are really kind.  Best of all, they seem to get what I was trying to do with the book – which are the best kind of reviews one can get as an author.  The reviews (with links to full reviews):

Back Cover

Carr’s concern is that development and globalization, as currently pursued, are creating more poverty than they solve, needlessly producing economic and environmental challenges that put everyone on Earth at risk. Confronting this paradoxical outcome head-on, Carr questions the “wisdom” of the traditional development-via-globalization strategy, a sort of connect-the-development-dots, by arguing that in order to connect the dots one must first see the dots. By failing to do so, agencies do not understand what they are connecting and why. This fundamental questioning of Post WWII development strategies, grounded in life along “Globalization’s Shoreline,” sets his approach to development in the age of globalization apart from much of the contemporary development literature.

— Michael H. Glantz, Director, CCB (Consortium for Capacity Building), INSTAAR, University of Colorado

Over the fifty years since the end of the colonial era, rich nations have granted Africa billions of dollars in development aid—the equivalent of six Marshall Plans—and yet, today, much of the continent is as desperate as ever for help. In Delivering Development, Edward Carr delves into the question of why the aid system has failed to deliver on its promises, and offers a provocative thesis: that economic development, at least as international donors define it, is not necessarily equal to advancement. Unlike many combatants in the debate over the causes of global poverty, who jet in and out of these countries and offer the view from 10,000 feet, Carr takes a novel approach to the problem. He examines the aid system as it is actually experienced by poor Africans.Delivering Development focuses on a pair of Ghanaian villages, which despite their poverty by statistical measures have nonetheless managed to construct sophisticated systems of agricultural cultivation and risk management. Carr doesn’t argue that these places hold the secret to ending poverty. On the contrary, his point is that there are no overarching solutions, that each community holds a unique set of keys to its own future. By delving into development at the grassroots, Carr reveals the rich and bedeviling complexity of a problem that, all too often, is reduced to simplistic ideological platitudes.”

— Andrew Rice, author of The Teeth May Smile but the Heart Does Not Forget: Murder and Memory in Uganda

Summaries of Recent Reviews (with links to full reviews)

The book is a riveting read, horizon broadening and . . . takes a somewhat unusual path towards challenging the dominant paradigm that complements other, parallel efforts . . . All-in-all, a must read for aid wonks everywhere.

— Andy Sumner, Global Dashboard

Development often fails. This is not a new premise. Many have written about it. But Edward Carr offers a fascinating perspective on why he believes this is true in Delivering Development.”

— Robin Pendoley, Thinking Beyond Borders

This book makes an important contribution to critical literatures on globalization and development . . . [providing] an often overlooked perspective within critical development literature: the real possibility for positive change and for a more active role of development’s target population to participate and shape the direction of change in their communities.

— Kelsey Hanrahan, Africa Today

Yep, no sooner do I post on failure and how we account for it and learn from it, then I come upon a big fail of my own.  That I can learn from. Irony, anyone?

As many of you know, I have been working in Ghana since 1997.  I’ve spent some 20 months there, though it has been a while since I was last on the ground (I need to change that) – basically, the last meaningful research trip I took was in the summer of 2006.  That work, along with the fieldwork that came before it, was so rich that I am still working through what it all means – and it has led me down the path of a book about why development doesn’t work as we expect, and now a (much more academic) complete rethinking of the livelihoods framework that many in development use to assess how people make a living.

One of my big findings (at least according to some of my more senior colleagues) is that inequality and (depending on how you look at it) injustice are not accidental products of “bad information” or “false consciousness” in livelihoods strategies, but integral parts of how people make a living (article to this effect here, with related work here and here, as well as a long discussion in Delivering Development).  One constraint specific to the livelihoods in the villages in which I have been working is the need to balance the material needs of the household with the social requirement that men make more money than their wives.  I have rich empirical data demonstrating this to be true, and illustrating how it plays out in agricultural practice (which makes up about 65% of most household incomes).

In other words, I know damn well that men get very itchy about anything that allows women to become more productive, as this calls one of the two goals of existing livelihoods strategies into question.  Granted, I figured this out for the first time around 2007, and have only very recently (i.e. articles in review) been able to get at this systematically, but still, I knew this.

And I completely overlooked it when trying to implement the one village improvement project with which I have been involved.  Yep, I totally failed to apply my own lessons to myself.

What happened?  Well, to put it simply, I had some money available after the 2006 fieldwork for a village improvement project, which I wanted the residents of Dominase and Ponkrum to identify and, to the extent possible, design for themselves.  We had several community meetings that meandered (as they do) and generally seemed to reflect the dominant voices of men.  However, at the end of one of these meetings, one of my extraordinarily talented Ghanaian colleagues from the University of Cape Coast had the experience and the awareness to quietly wander off to a group of women and chat with them.  I noticed this but did not say anything.  A few minutes later, he strolled by, and as he did he said to me “we need to build a nursery.”  Kofi had managed to elicit the womens’ childcare needs, which were much more practical and actionable than any other plans we had heard.  At the next community meeting we raised this, and nobody objected – we just got into wrangling over details.  I left at the end of the field season, confident we could get this nursery built and staffed.

Five years later, nothing has happened.  They formed the earth blocks, but nobody cleared the agreed-upon area for the nursery.  It was never a question of money, and my colleagues at the University of Cape Coast checked in regularly.  Each time, they left with promises that something would get going, and nothing ever did.  I don’t fault the UCC team – the community needed to mobilize some labor so they would have buy-in for the project, and would take responsibility for the long-term maintenance of the structure. This is on the community – they just never built it.

And it wasn’t until yesterday, when talking about this with a colleague, that I suddenly realized why – childcare would lessen one demand on women that limits their agricultural productivity and incomes.  Thus, with a nursery in place women’s incomes would surely rise . . . and men have no interest in that, as this is not the sort of intervention that would drive a parallel increase in their own incomes.  I have very robust data that demonstrates that men move to control any increase in their wives incomes that might threaten the social order of the household, even if that decreases overall household income and access to food.

So why, oh why, did I ever think that men would allow this nursery to be built?  Of course they wouldn’t.

I can excuse myself between 2006-2008 for missing this, as I was still working through what was going on in these livelihoods.  But for the last three years I knew about this fundamental component of livelihoods, and how robust this aspect of livelihoods decision-making really is, even under conditions of change such as road construction.  I have been looking at how others misinterpret livelihoods and design/implement bad interventions for years, all the while doing that very thing myself.

Healer, heal thyself.

In our last installment of this series (in which I try to lend some transparency to the publication process), I had requested (in great detail) further clarification before I revised and resubmitted an article to Development and Change. Yesterday, I received a reply.

DECH-11-094 – Livelihoods as Governmentality: Reframing the Logic of Livelihoods for Development

Dear Edward,

Thank you for your detailed response to the referee reports that were sent to you in connection with your paper.

The editors discussed your e-mail during their editorial board meeting last week, and have asked me to contact you. They found your response to the referee comments generally reasonable, and a good foundation for revising your paper. Assuming that you are still interested in doing so, they invite you to go ahead with the revisions as proposed.

So, this is good news.  But then the editor went an extra step – in other words, she made some editorial suggestions:

I would like to mention a couple of additional points. We ask all authors who submit revised manuscripts to include a list of revisions made. The response that you have sent to us indicates that some of the comments will not be addressed in the revised paper: it is important that your list of revisions also includes a rationalization for what you have NOT done, as well as for the changes that you have made. However, you might also consider whether any of those ‘non-responses’ need to be covered in some way in the revised manuscript itself. For instance, in your point 2 d) (ii) you suggest adding a footnote regarding (the lack of) remittances; it might be helpful to consider something similar for the point above it, on religious activity. And more broadly, rather than dismissing the report of Reader 1 because s/he seems to be asking for a totally different paper, you might consider whether you could reformulate any elements of your paper to ensure that similar criticisms could not be levelled against it by other readers, in the event of it being published. Finally, with regard to the point about precipitation: there is clearly a difference of opinion here, although the referee mentions an IPCC report (as well as other statistics), and you tell us that you are on the IPCC and have access to rain gauge data. Since the issue at stake seems to revolve around widely available quantitative data, presumably it can be resolved through the use of recognized sources and citations?

There is nothing wrong with this at all – it is a clear signal from the editor of the preferred tack to take while editing this piece.  Contrast this with the initial editorial statement I received. Which would you rather operate from?

The editor also acknowledged my concerns regarding one of the reviewers:

For the record I would like to stress that correspondence about the revision process does not imply any kind of commitment to a revised version of the paper. As stated in Friedl’s original e-mail, revised manuscripts are subject to further review and refereeing. In practice, the editors are often selective when it comes to sending revised papers back to referees, but they reserve the right to approach any of the original referees, and/or one or more new referees, as they see fit.

That is as close as an editor is going to come to saying “yeah, we’re not going back to reviewer 1.”

Had I not pushed back and laid out my concerns, I would be operating with the initial very vague guidance.  Now I have a very clear, achievable path to getting this done.  There is no guarantee it will be accepted, of course, but now my odds are greatly improved.

In the interest of transparency, this was my response:

Thank you for getting back to me, and doing so in such a detailed manner.  I greatly appreciate your editorial involvement in the revision process.  I will, of course, submit a cover letter that details all the revisions made in detail, and explain what issues the revisions were intended to address – and I will, of course, explain why I have not chosen to address certain comments.  I take your point with regard to reviewer 1 – in fact, if I look back at my notes on his/her review, I do see one or two things that I could address from that review, and I will try to think about how I might phrase things in the introduction to address the sort of reading I recieved from this reviewer.  I have no problem with footnoting the religion issue, and I will, of course, document my claims about the climate and the rain gauges.  I believe the reviewer was looking at the data for the Sahel, which is rather different than in coastal areas.

[Note: There is no harm in conceding a point to an editor, especially when that point is valid! It also signals to the editor that you are taking their points seriously.]

I fully expected my resubmission to be reviewed, which of course carries with it the risk of rejection or further revisions – though hopefully I will revise the paper in a manner that avoids this outcome.  I of course recognize your editorial right to select whatever reviewers you see fit to assess this revision – I merely wanted to voice my concerns with reviewer 1 for the record.  I appreciate that you and your colleagues do try to be selective about who reviews revisions, and having raised my concerns I am more than happy to proceed through this process with the reviewers you see fit.

[Note: read the last two sentences as me more or less saying "I'm glad you agree that reviewer 1 sucked."]

Again, my thanks for taking the time to send along detailed comments.  I will get to work on the revision shortly, and hopefully turn it back to you before the end of the month.

Upward and onward, y’all.

Yesterday I posted about a less-than-productive set of comments from at least one reviewer, and some contradictory comments from the other reviewers, related tof a manuscript I have in submission to a major development journal.  This is not at all uncommon – I’ve only had two or three articles ever accepted without revision, and have gone three or more rounds on a few.  However, to manage this sort of thing successfully requires good editorial guidance – i.e. what comments are relevant, whose comments are more appropriate (when there is a conflict in the comments), how much revision is really needed?

Well, my polite-but-firm 1500-word missive to the editors of the journal has resulted in immediate action – from my inbox this morning:

Dear Edward

Thank you for your email. We have an editorial board meeting next week during which I will circulate your comments. I will get back to you shortly thereafter.

Best wishes,

XXXX

Now, this guarantees nothing . . . however, now my very detailed response and suggested edits will end up in front of the editorial board, and hopefully I will get a sense of what is actually needed to make this publishable.  I also might get some sense of how they plan to review a resubmission . . . I really don’t want this to take another 5-6 months, and I really don’t want Reviewer 1 to ever see this piece again!

More as it happens . . .