Mon 24 Oct 2011

What should peer review do?

Posted by Ed under Higher Education, research, Uncategorized
No Comments

Colleague Ben Neimark at ODU recently asked me a tough question: “What makes for good (helpful to get published, strengthened,  intellectually creativity, etc.) peer review?”  I figured this might be of wider interest to academic colleagues, as well as those who see the entire academic publishing world as somewhat opaque.  So . . .

I think the challenge in producing a good peer review is to balance its dual imperative .  There is the part of peer review that ensures quality and offers constructive criticism (and I have received some in the case of my current livelihoods work – see here, here and here -, and have had some reviewers offer great stuff in the past).  Then there is the disciplinary policing that goes on through peer review, where reviewers don’t examine the quality of the data or argument, but simply argue against it because it challenges convention (which the reviewer likely belongs to or established) – see my comments about reviewer 1 at the bottom of this post.  This second function makes innovation very challenging unless you are very, very hardheaded (which I am).

In a nutshell, though, I think good peer review is that which looks at a paper for its stated aims and evaluates

  1. are those stated aims actually new and interesting and
  2. did the paper achieve the stated aims.

If standard 1) is not met, a good peer reviewer should be able to suggest where the real contribution of the paper lies – i.e. by suggesting literatures into which the author should place the manuscript.  If standard 2) is not met, the reviewer should explain exactly how and why this happened, and what sorts of remedial steps might solve the problem(s).  That is my minimum take . . .

I’m happy to hear the opinions of others . . .

Tags: , , ,

 

Thu 13 Oct 2011

Ed Fail

Posted by Ed under Africa, Delivering Development, development, Livelihoods, policy, research, sustainable development
[4] Comments

Yep, no sooner do I post on failure and how we account for it and learn from it, then I come upon a big fail of my own.  That I can learn from. Irony, anyone?

As many of you know, I have been working in Ghana since 1997.  I’ve spent some 20 months there, though it has been a while since I was last on the ground (I need to change that) – basically, the last meaningful research trip I took was in the summer of 2006.  That work, along with the fieldwork that came before it, was so rich that I am still working through what it all means – and it has led me down the path of a book about why development doesn’t work as we expect, and now a (much more academic) complete rethinking of the livelihoods framework that many in development use to assess how people make a living.

One of my big findings (at least according to some of my more senior colleagues) is that inequality and (depending on how you look at it) injustice are not accidental products of “bad information” or “false consciousness” in livelihoods strategies, but integral parts of how people make a living (article to this effect here, with related work here and here, as well as a long discussion in Delivering Development).  One constraint specific to the livelihoods in the villages in which I have been working is the need to balance the material needs of the household with the social requirement that men make more money than their wives.  I have rich empirical data demonstrating this to be true, and illustrating how it plays out in agricultural practice (which makes up about 65% of most household incomes).

In other words, I know damn well that men get very itchy about anything that allows women to become more productive, as this calls one of the two goals of existing livelihoods strategies into question.  Granted, I figured this out for the first time around 2007, and have only very recently (i.e. articles in review) been able to get at this systematically, but still, I knew this.

And I completely overlooked it when trying to implement the one village improvement project with which I have been involved.  Yep, I totally failed to apply my own lessons to myself.

What happened?  Well, to put it simply, I had some money available after the 2006 fieldwork for a village improvement project, which I wanted the residents of Dominase and Ponkrum to identify and, to the extent possible, design for themselves.  We had several community meetings that meandered (as they do) and generally seemed to reflect the dominant voices of men.  However, at the end of one of these meetings, one of my extraordinarily talented Ghanaian colleagues from the University of Cape Coast had the experience and the awareness to quietly wander off to a group of women and chat with them.  I noticed this but did not say anything.  A few minutes later, he strolled by, and as he did he said to me “we need to build a nursery.”  Kofi had managed to elicit the womens’ childcare needs, which were much more practical and actionable than any other plans we had heard.  At the next community meeting we raised this, and nobody objected – we just got into wrangling over details.  I left at the end of the field season, confident we could get this nursery built and staffed.

Five years later, nothing has happened.  They formed the earth blocks, but nobody cleared the agreed-upon area for the nursery.  It was never a question of money, and my colleagues at the University of Cape Coast checked in regularly.  Each time, they left with promises that something would get going, and nothing ever did.  I don’t fault the UCC team – the community needed to mobilize some labor so they would have buy-in for the project, and would take responsibility for the long-term maintenance of the structure. This is on the community – they just never built it.

And it wasn’t until yesterday, when talking about this with a colleague, that I suddenly realized why – childcare would lessen one demand on women that limits their agricultural productivity and incomes.  Thus, with a nursery in place women’s incomes would surely rise . . . and men have no interest in that, as this is not the sort of intervention that would drive a parallel increase in their own incomes.  I have very robust data that demonstrates that men move to control any increase in their wives incomes that might threaten the social order of the household, even if that decreases overall household income and access to food.

So why, oh why, did I ever think that men would allow this nursery to be built?  Of course they wouldn’t.

I can excuse myself between 2006-2008 for missing this, as I was still working through what was going on in these livelihoods.  But for the last three years I knew about this fundamental component of livelihoods, and how robust this aspect of livelihoods decision-making really is, even under conditions of change such as road construction.  I have been looking at how others misinterpret livelihoods and design/implement bad interventions for years, all the while doing that very thing myself.

Healer, heal thyself.



Tags: , , , , , , , , ,

 

Thu 6 Oct 2011

At what scale can we fail?

Posted by Ed under development, Development Institutions, policy, research
[10] Comments

I am a big fan of the idea of admitting failure and trying to learn from it.  I like ambitious projects with potentially huge payoffs, but a lot of risk of failure – they’re just much more interesting than going at things incrementally.  Besides, if you are going to fail, why not fail spectacularly?  As I tell my grad students, if you are going to ride it all the way to the ground, you might as well dig a big hole when you get there.  At least people will notice the hole, and try to figure out what the hell you were up to . . . of course, I am an academic (with tenure), so I have a pretty big cushion to land on these days.

All that said, I wonder about the utility of these admitting failure efforts that I see coming from groups like Engineers without Borders.  I had the good fortune to catch up with Tom Murphy (or, as the twitterati know him, @viewfromthecave) the other day while he was here in DC, and we started talking about learning from failure.  In the course of our conversation, we came around to two key problems.  First, really admitting failure requires reframing the public image of development as an inherently do-no-harm effort, where just doing something is better than nothing.  Second, given this first problem, when we really start talking about what failure means, even in the most constructive of settings, we will call the entire development enterprise into question. How do we avoid throwing the baby out with the bathwater?

We have long allowed ourselves and our donor constituencies to believe that development work should never have bad outcomes – there is a pervasive belief (under challenge right now, at least by some) that, at worst, a failed project will not change anything – that is what development failure means. Of course, this is simply untrue – development efforts can make things much, much worse for people if they are poorly framed, designed, and implemented – a point I try to make in Delivering Development.  This has a lot to do with the very imagery of a helpless and oppressed global poor the aid world relies upon to raise funds.  When people see someone in a situation that difficult, they assume things could not get worse.  There is no discussion of what is working in the lives of the poor, and therefore the public has little sense that there are fragile things in peoples’ lives and livelihoods that should be protected as we bring new programs and projects to ground. As a result, development takes on the image of a low-risk enterprise in which social protection and “do no harm” safeguards are superfluous, as the worst we could do is leave people as they were.

Up against that worldview, admitting failure seems just fine – “hey, we didn’t really move the needle with that project, but we’ll figure out what we did wrong and try again” sounds much better than “we are incredibly sorry for utterly devastating the physical basis of your livelihoods and forcing many of you to abandon your farms because we ignored your existing land management practices.”  Unfortunately, admitting failure means a lot of the latter, and I am not at all convinced that anyone has the stomach to really wade into that.

This issue has to be combined with a concern for the scale of failure.  It is all well and good to admit failure, even ugly failures, at the project level – stuff happens.  A failed project can usually be traced to concrete causes that can then be addressed and remedied.  But how can a bilateral aid agency, or even a multilateral agency, do the same for its programs?  It is one thing for such huge organizations to talk about the failure of individual projects, and learn from them, but how can we talk about learning from entire programs that don’t live up to expectations without attracting serious challenges to the aid budget that end up wrecking even successful programs, or preventing the scale-up of things that we know work? Put another way, how can we create an environment where learning from our activities is truly possible, and balance that environment with the political reality of aid agencies and NGOs that answer to (different) constituencies that expect only good things to happen?

This framing of global poverty, and the persistent need to justify aid budgets, puts everyone involved with development on a terrible tightrope – at least for those of us interested in evidence-based programming and policy.  Just saying that admitting failure is good does not begin to get us to a world in which we can see that as more than a slogan.  We will have to unwind decades of public relations and fundraising practice, and back out of some very long-standing and pervasive views of global poverty, before we have any real hope of bringing real learning to the fore of development practice.

Or, we could just give everyone tenure . . .



Tags: , , , ,

 

Thu 6 Oct 2011

More tales of publication: revision negotiations part 2

Posted by Ed under Adaptation, development, Higher Education, research
[5] Comments

In our last installment of this series (in which I try to lend some transparency to the publication process), I had requested (in great detail) further clarification before I revised and resubmitted an article to Development and Change. Yesterday, I received a reply.

DECH-11-094 – Livelihoods as Governmentality: Reframing the Logic of Livelihoods for Development

Dear Edward,

Thank you for your detailed response to the referee reports that were sent to you in connection with your paper.

The editors discussed your e-mail during their editorial board meeting last week, and have asked me to contact you. They found your response to the referee comments generally reasonable, and a good foundation for revising your paper. Assuming that you are still interested in doing so, they invite you to go ahead with the revisions as proposed.

So, this is good news.  But then the editor went an extra step – in other words, she made some editorial suggestions:

I would like to mention a couple of additional points. We ask all authors who submit revised manuscripts to include a list of revisions made. The response that you have sent to us indicates that some of the comments will not be addressed in the revised paper: it is important that your list of revisions also includes a rationalization for what you have NOT done, as well as for the changes that you have made. However, you might also consider whether any of those ‘non-responses’ need to be covered in some way in the revised manuscript itself. For instance, in your point 2 d) (ii) you suggest adding a footnote regarding (the lack of) remittances; it might be helpful to consider something similar for the point above it, on religious activity. And more broadly, rather than dismissing the report of Reader 1 because s/he seems to be asking for a totally different paper, you might consider whether you could reformulate any elements of your paper to ensure that similar criticisms could not be levelled against it by other readers, in the event of it being published. Finally, with regard to the point about precipitation: there is clearly a difference of opinion here, although the referee mentions an IPCC report (as well as other statistics), and you tell us that you are on the IPCC and have access to rain gauge data. Since the issue at stake seems to revolve around widely available quantitative data, presumably it can be resolved through the use of recognized sources and citations?

There is nothing wrong with this at all – it is a clear signal from the editor of the preferred tack to take while editing this piece.  Contrast this with the initial editorial statement I received. Which would you rather operate from?

The editor also acknowledged my concerns regarding one of the reviewers:

For the record I would like to stress that correspondence about the revision process does not imply any kind of commitment to a revised version of the paper. As stated in Friedl’s original e-mail, revised manuscripts are subject to further review and refereeing. In practice, the editors are often selective when it comes to sending revised papers back to referees, but they reserve the right to approach any of the original referees, and/or one or more new referees, as they see fit.

That is as close as an editor is going to come to saying “yeah, we’re not going back to reviewer 1.”

Had I not pushed back and laid out my concerns, I would be operating with the initial very vague guidance.  Now I have a very clear, achievable path to getting this done.  There is no guarantee it will be accepted, of course, but now my odds are greatly improved.

In the interest of transparency, this was my response:

Thank you for getting back to me, and doing so in such a detailed manner.  I greatly appreciate your editorial involvement in the revision process.  I will, of course, submit a cover letter that details all the revisions made in detail, and explain what issues the revisions were intended to address – and I will, of course, explain why I have not chosen to address certain comments.  I take your point with regard to reviewer 1 – in fact, if I look back at my notes on his/her review, I do see one or two things that I could address from that review, and I will try to think about how I might phrase things in the introduction to address the sort of reading I recieved from this reviewer.  I have no problem with footnoting the religion issue, and I will, of course, document my claims about the climate and the rain gauges.  I believe the reviewer was looking at the data for the Sahel, which is rather different than in coastal areas.

[Note: There is no harm in conceding a point to an editor, especially when that point is valid! It also signals to the editor that you are taking their points seriously.]

I fully expected my resubmission to be reviewed, which of course carries with it the risk of rejection or further revisions – though hopefully I will revise the paper in a manner that avoids this outcome.  I of course recognize your editorial right to select whatever reviewers you see fit to assess this revision – I merely wanted to voice my concerns with reviewer 1 for the record.  I appreciate that you and your colleagues do try to be selective about who reviews revisions, and having raised my concerns I am more than happy to proceed through this process with the reviewers you see fit.

[Note: read the last two sentences as me more or less saying "I'm glad you agree that reviewer 1 sucked."]

Again, my thanks for taking the time to send along detailed comments.  I will get to work on the revision shortly, and hopefully turn it back to you before the end of the month.

Upward and onward, y’all.



Tags: , , , ,

 

Wed 28 Sep 2011

Oh, Ghana . . .

Posted by Ed under Africa, development, environment, Food Security, Livelihoods, policy
1 Comment

Ah, that familiar refrain – a mix of love and derision provoked by the vagaries of life in my favorite West African country: the power cuts out randomly in the midst of a big soccer match, “Oh, Ghana!” The new road washes out because of inadequate culverts? “Oh, Ghana!” And now, the country’s economy grows 34% in the second quarter of 2011 – expanding the GDP by 3.4 percent in that quarter alone (h/t to Andy Sumner for pointing this out to me)?

Wait, isn’t that good news?

Well, on its face, yes – this surge in growth suggests there is a lot more money at play in Ghana, and that will hopefully result in new and better jobs, greater revenues for the state, and eventually better services for the population.  But there are two big caveats that really, really worry me here.

  1. The growth was driven mostly by growth in the mining and quarrying sector – of which oil has about a 2/3 share. So the economy has grown, but it is still commodity-dependent.  Admittedly, they now have oil on top of cocoa and gold, but these don’t exactly track independently of one another.  Building your whole economy on three commodities is not a path to a stable, sustainable future.
  2. Ghana does not seem to have a plan to spend all of this new revenue in a manner that will trigger the virtuous process I was describing above.  Without a plan, the possibility of misuse and redirection of funds into private accounts rises dramatically (h/t to Mark Weston).

Oh, Ghana!

Even the oddly good news – agricultural (economic) growth seems to be matching the growth of mining and quarrying – isn’t really that good.  At first glance, this news seems to suggest that ag production is increasing, or that more of that production is getting to market before spoiling, trends that would benefit much of the Ghanaian population.  Maybe not, though – Ghana’s light-crop cocoa crop doubled over the same period last year, suggesting this increase is largely pegged to cocoa.  Worse, a big chunk of this improvement is tied to good weather, which is difficult to gamble on year-to-year.

Oh, Ghana!



Tags: , , , , ,

 

Mon 26 Sep 2011

On agricultural productivity and food security

Posted by Ed under Adaptation, Delivering Development, Food Security, policy, research, sustainable development
[6] Comments

I was on a panel at the Organic Trade Association‘s research series at the Natural Products Expo East in Baltimore last Friday, discussing the issue of organic farming and the need to feed the world.  As I heard over and over from proponents of organic agriculture, the argument “you can’t feed the world on organic” is something thrown at them all the time.  As I argued, though, this is a production-based argument: that is, organic farming often has somewhat lower levels of productivity than industrial farming (though there are several cases where this does not seem to hold, and a number of confounding factors that make it entirely possible that the productivity difference is actually quite small).  Well, that would be a relevant argument if we were already using our food resources carefully.  Except we aren’t.  Consider:

  • We still produce more than enough food globally to feed everyone a very healthy number of calories, and probably enough that those calories could be accompanied by adequate nutrients.  The current problems of food insecurity are primarily about distribution, not production.
  • Anywhere between 20% and 40% of all food grown globally spoils before it reaches market.  The figures are lower for grains (which tend to travel well) and much higher for vegetables.
  • In the US, we throw away roughly 30% of all food we purchase.
  • Consider those two numbers together: In the US, we probably lose a lot less of the crop between farm and purchase at market, but then throw 30% of it away.  In other places, the food that reaches the table is nearly completely eaten, but we could lose up to 40% of that food before it reaches market.  In other words, no matter where you go on Earth, there is a hell of a lot of waste in the food system.
  • Finally, consider that 33% of all farmland is used for animal feed, one of the less efficient ways of getting calories out of the environment.  It is unclear to me if this 33% includes biofuel crops, but in any case biofuels would only add a few percentage points to this at most.

In short, we have distribution problems and an astonishing amount of waste in our food systems, but it seems that a lot of the food security debate in policy circles is driven by production arguments.  Enhancing production is not a low hanging fruit.  Enhancing production is often used as an excuse for ignoring local knowledge and capacity in favor of reworking entire agroecological systems (which usually ends badly).  Those of us working in development would be well-served to consider all the ways we might address hunger, including waste and distribution, rather than focus myopically on one cause for what might be a phantom problem.  Welcome to another central theme of Delivering Development: misunderstanding/misidentifying the development challenge, and then trying to solve the wrong thing.

One caveat: there are places in the world in absolute production crises – that is, they lack market access to facilitate the movement of needed food, and their agricultural systems are no longer resilient in the face of current challenges.  In these places, waste may be less of an issue, and distribution solutions may be years in the future (good infrastructure and markets require good governance, which is no easy fix), and therefore the application of new agricultural technologies might become the low hanging fruit solution for the time being, until the other challenges can be met. It’s about finding the right tool for the job (and knowing exactly what the job is, too).



Tags: , , , , , ,

 

Tue 20 Sep 2011

See, being polite and organized helps

Posted by Ed under development, Higher Education, Livelihoods, research
[2] Comments

Yesterday I posted about a less-than-productive set of comments from at least one reviewer, and some contradictory comments from the other reviewers, related tof a manuscript I have in submission to a major development journal.  This is not at all uncommon – I’ve only had two or three articles ever accepted without revision, and have gone three or more rounds on a few.  However, to manage this sort of thing successfully requires good editorial guidance – i.e. what comments are relevant, whose comments are more appropriate (when there is a conflict in the comments), how much revision is really needed?

Well, my polite-but-firm 1500-word missive to the editors of the journal has resulted in immediate action – from my inbox this morning:

Dear Edward

Thank you for your email. We have an editorial board meeting next week during which I will circulate your comments. I will get back to you shortly thereafter.

Best wishes,

XXXX

Now, this guarantees nothing . . . however, now my very detailed response and suggested edits will end up in front of the editorial board, and hopefully I will get a sense of what is actually needed to make this publishable.  I also might get some sense of how they plan to review a resubmission . . . I really don’t want this to take another 5-6 months, and I really don’t want Reviewer 1 to ever see this piece again!

More as it happens . . .



Tags: , , , ,

 

Mon 19 Sep 2011

How to deal with peer review

Posted by Ed under Adaptation, development, Livelihoods, research
[6] Comments

Remember the article on livelihoods I referenced in my last post?  The one that has been through 13-odd reviewers over the past 6 months?  Well, it came back this morning with that least satisfying of responses: revise and resubmit.  The reviewers disagreed on the strengths and weaknesses of the paper, but all of them wanted at least some small changes.

Now, this is hardly the first time I’ve had to deal with this.  Happens all the time.  But this review, like a number of others I have encountered in my career, highlights for me the need for strong editors at journals.  The email from the journal read, in part:

By the time of [the editorial board] meeting, we had received the reports of the external referees. You will find copies of their comments in the attached document. The editors feel that the referees have raised some important points and highlighted some shortcomings in the paper, which would need to be addressed through a round of revisions before we could proceed further. Please note that the editors are forwarding these comments as you might find them helpful; individual comments do not necessarily represent the views of the editors, or a consensus regarding the direction of any possible revision. [my emphasis].

Well, that’s just super.  It amounts to “here are some comments, which might or might not be relevant. Good luck!”  This gives the author nothing to work from, especially when some of the comments are contradictory.

So, a quick lesson for all of you budding academics out there.  Letters like this require a response asking for clarification.  I hammered out an email requesting a conversation with the editor.  Yes, that’s right, you can talk to the editors!  My first paragraph:

Thank you for forwarding the reviewer comments on my paper (Manuscript ID DECH-11-094 entitled “Livelihoods as Governmentality: Reframing the Logic of Livelihoods for Development”).  I appreciate the time and effort the reviewers put into the paper, and the consideration the editorial board has given my submission.  I would very much like to revise and resubmit this paper, but I need a bit of guidance if I am to do so in a productive manner.  Ideally, I would like to talk with the editor in charge of this submission to resolve some of my ideas for revision, and some larger concerns before moving forward.  I have summarized them, in a general sense, below:

(Note that this is polite – always be polite! It’s amazing how many people fail to do this.)

I then wrote 1500 words on what I proposed to do – with great specificity.  I hope to speak directly to the editor myself soon.  That way, I will know how s/he plans to treat the manuscript when they receive revisions, and how s/he views my suggestions – this is huge, as it will tell me what I can and cannot skip in the comments.

Oh, and one of the reviewers was a disaster – did not get the paper, wanted it to be the paper they would have written, etc.  I headed this person off with the following paragraph:

The comments of reviewer 1 are external critique – that is, they demand that the paper be something other than what it is, do not acknowledge the stated goals of the paper, or whether or not the paper achieved those goals.  The reviewer clearly did not engage with the core theory of the paper (for example in arguing the lit review was too confusing and should be cut down), instead demanding it be reframed as a gender analysis (which it is not) with relevance to policy (not the goal of this piece, which is more about foundational theory upon which policy statements might be constructed – policy relevance is also not a criteria for Development and Change).  The reviewer appears to be upset because the paper is not the gender and development paper s/he would have written (incidentally, I already wrote a paper closer to what they wanted and published it in World Development about three years ago – this paper is in part an effort to move past the limitations of that analysis), and instead of looking at the literature I am drawing on, they simply demanded I reframe and cite all the things they think are important.  Generally speaking, I try to take responsibility for moments when the reader becomes confused, treating this as a symptom of unclear writing or thinking on my part – I hope this is clear from my responses to the other reviewers’ comments.  However, the complete disengagement of this review with the paper as it was written, coupled with reviewer 3’s assessment of the paper as well-written, suggests to me that this reviewer’s concerns cannot be addressed without writing a completely different paper – and that paper would not make much of a contribution anymore (something they acknowledge). Suggested edits: none

Yep, you can make the case for the removal of reviewers from a revise and resubmit – especially if they will never approve the paper for what it is.

I’ll keep you posted on progress . . .



Tags: , , , , , , ,

 

Sun 18 Sep 2011

On data and its caveats

Posted by Ed under Adaptation, Africa, Delivering Development, Food Security, Livelihoods, research
1 Comment

Marc Bellemare at Duke has been using Delivering Development in his development seminar this semester.  On Friday, he was kind enough to blog a bit about one of the things he found interesting in the book: the finding that women were more productive than men on a per-hectare basis.  As Marc notes, this runs contrary to most assumptions in the agricultural/development economics literature, especially some rather famous work by Chris Udry:

Whereas one would expect men and women to be equally productive on their respective plots within the household, Udry finds that in Burkina Faso, men are more productive than women at the margin when controlling for a host of confounding factors.

This is an important finding, as it speaks to our understanding of inefficiency in household production . . . which, as you might imagine given Udry’s findings, is often assumed to be a problem of men farming too little and women farming a bit too much land.  So Marc was a bit taken aback to read that in coastal Ghana the situation is actually reversed – women are more productive than men per unit area of land, and therefore to achieve optimal distributions of agricultural resources (read:land) in these households we would actually have to shift land out of men’s production into women’s production.

I knew that this finding ran contrary to Udry and some other folks, but I did not think it was that big a deal: Udry worked in the Sahel, which is quite a different environment and agroecology than coastal Ghana.  Further, he worked with folks of a totally different ethnicity engaged with different markets.  In short, I chalked his findings up to the convergence of any number of factors that had played out somewhat differently in my research context.  I certainly don’t see my findings as generalizable much beyond Akan-speaking peoples living in rural parts of Ghana . . .

All of that said, Marc points out that with regard to my findings:

Of course, this would need to be subjected to the proper empirical specification and to a battery of statistical tests . . .

Well, that is an interesting question.  So, a bit of transparency on my data (it is pretty transparent in my refereed pubs, but the book didn’t wade into all of that):

Weaknesses:

  • The data was gathered during the main rainy season, typically as the harvest was just starting to come in.  This required folks to make some degree of projection about the productivity of their fields at least a month into the future, and often several months into the future
  • The income figures for each crop, and therefore for total agricultural productivity, were self-reported. I was not able to cross-check these reported figures by counting the actual amount of crop coming off each farm.
    • I also gathered information on expenses, and when I totaled up expenses and subtracted them from reported income, every household in the village was running in the red.  I know that is not true, having lived there for some 18 months of my life.
    • There is no doubt in my mind that production figures were underestimated, and expenses overestimated, in my data – this fits into patterns of income reporting among the Akan that are seen elsewhere in the literature.
    • Therefore, you cannot trust the reported figures as accurate absolute measures of farm productivity.

Strengths:

  • The data was replicated across three field seasons.  The first two field seasons, I conducted all data collection with my research assistant.  However, in the final year of data collection, I lead a team of four interviewers from the University of Cape Coast, who worked with local guides to identify farms and farmers to interview – in the last year, we interviewed every willing farmer in the village (nearly 100% of the population).
    • It turns out that my snowball sample of households in the first two years of data collection actually covered the entire universe of households operating under non-exceptional household circumstances (i.e. they are not samples, they are reports on the activities of the population).
      • In other words, you don’t have to ask about my sampling – there was no sampling.  I just described the activities of the entire relevant population in all three years.
      • This removes a lot of concerns people have about the size of my samples – some household strategies only had 7 or 8 households working with them in a given year, which makes statistical work a little tricky :)  Well, turns out there is no real need for stats, as this is everyone!
      • The only exception to this: female-headed households.  I grossly underinterviewed them in years 1 and 2 (inadvertently), and the women I did interview do not appear to be representative of all female-headed households.  I therefore can only make very limited claims about trends in these households.
    • Even with completely new interviewers who had no preconceived notions about the data, the income findings came in roughly the same as when I gathered the data. That’s replicability, folks! Well, at least as far as qualitative social science gets in a dynamic situation.
    • Though the data was gathered at only one point in the season, at that point farmers were already seeing how the first wave of the harvest was doing and could make reasonable projections about the rest of the harvest.

I’m probably forgetting other problems and answers . . . Marc will remind me, I’m sure!  In any case, though, Marc asks a really interesting question at the end of his post:

Assuming the finding holds, it would be interesting to compare the two countries given that Burkina Faso and Ghana share a border. Is the change in gender differences due to different institutions? Different crops?

The short answer, for now, has to be a really unsatisfying “I don’t know.”  Delivering Development lays out in relatively simple terms a really complex argument I have building for some time about livelihoods, that they are motivated by and optimized with reference to a lot more than material outcomes.  The book builds a fairly simple explanation for how men balanced the need to remain in charge of their households with the need to feed and shelter those households . . . but I have elaborated on this in a piece in review at the Development and Change.  I will send them an email and figure out where this is in review – they have been struggling mightily with reviewers (last I heard, they had gone through 13!?!) and put up a preprint as soon as I am able.  This is relevant here because I would need a lot more information about the Burkina setting to work through my new livelihoods framework before I could answer Marc’s question.

Stay tuned!

 

Tags: , , , , , , , ,

 

Tue 13 Sep 2011

Food security and conflict done badly . . .

Posted by Ed under Africa, Food Security, policy, research, sustainable development
[3] Comments

Over at the Guardian, Damian Carrington has a blog post arguing that “Food is the ultimate security need.”  He bases this argument on a map produced by risk analysts Maplecroft, which sounds quite rigorous:

The Maplecroft index [represented on the map], reviewed last year by the World Food Programme, uses 12 types of data to derive a measure of food risk that is based on the UN FAO’s concept. That covers the availability, access and stability of food supplies, as well as the nutritional and health status of populations.

I’m going to leave aside the question of whether we can or should be linking food security to conflict – Marc Bellemare is covering this issue in his research and has a nice short post up that you should be reading.  He also has a link to a longer technical paper where he interrogates this relationship…I am still wading through it, as it involves a somewhat frightening amount of math, but if you are statistically inclined, check it out.

Instead, I would like to quickly raise some questions about this index and the map that results. First, the construction of the index itself is opaque (I assume because it is seen as a proprietary product), so I have no idea what is actually in there.  Given the character of the map, though, it looks like it was constructed from national-level data.  If it was, it is not particularly useful – food insecurity is not only about the amount of food, but access to that food and entitlement to get access to the food, and these are things that tend to be determined locally.  You cannot aggregate entitlement at the national level and get a meaningful understanding of food insecurity – and certainly not actionable information.

Further, you can’t aggregate food markets or prices at the national level and get anything meaningful with regard to food security – let’s compare Maplecroft’s map with FEWS-NETs maps for the immediate future (August-September 2011):

First Maplecroft:

Now FEWS-NET:

While FEWS-NET does not have global coverage, compare their maps to those of Maplecroft and you see two things: One, FEWS is clearly working at a much finer geographic scale, because they have on-the-ground information about actual markets and access, as well as a deep understanding of climate and livelihoods through which to contextualize their grounded data.  This is what it takes to represent variable vulnerability within a country.  The variability you see on their map illustrates my point about the problems of national-level statistics – clearly food insecurity is a regional-to-local problem in every country, even Somalia.  Two, FEWS is not projecting major risk in the same places as Maplecroft, whose map has painted most of equatorial and dryland Africa as problematic at best.  Now, FEWS-NET’s medium-term projections (October-December 2011):

Again, no real resemblance to the Maplecroft map.

Now, you can argue that the Maplecroft map is aimed at a different goal than the FEWS-NET maps, as Maplecroft is trying to create a risk-assessment picture of food security in the region.  However, Maplecroft’s timescale is unclear (does it cover the next 6 months? 1 year? 5 years?), and its data is so over-aggregated as to be non-actionable.  You can’t build policy or programs from this, and I would argue that you can’t really assess the risk of food insecurity from the map or the underlying index either.  FEWS-NET’s maps are what actionable information looks like . . .

I appreciate the point Carrington is trying to make on his blog – food security is a really important issue.  But if we are to address the challenges of hunger and conflict, we need to build our understanding of the connection between them from meaningful data . . . and probably work from the outstanding material already available via FEWS-NET and others.



Tags: , , , , , ,

 

« Previous Page — Next Page »