Andy Sumner and Charles Kenny (disclosure – Andy and Charles are friends of mine, and I need to write up my review of Charles’ book Getting Better . . . in a nutshell, you should buy it) have a post on the Guardian’s Poverty Matters Blog addressing the two most recent challenges to the idea of the “poverty trap”: Ghana and Zambia’s recent elevations to middle-income status (per capita GNIs of between $1,006 and $3,975) by the World Bank.
Quick background for those less versed in development terminology: GNI (Gross National Income) is the value of all goods and services produced in a country, as well as all overseas investments and remittances (money sent home from abroad). Per capita GNI divides this huge number by the population to get a sense of the per-person income of the country (there is a loose assumption that the value of goods and services will be paid in the form of wages). So, loosely speaking, a per capita GNI of $1006 is roughly equivalent to $2.75/day. Obviously $2.75 buys a lot more in rural Africa than it does basically anywhere inside the US, but this is still a pretty low bar at which to start “Middle Income.”
I do not want to engage an argument about where Middle Income should start in this post – Andy and Charles take this up near the end of their post, and nicely lay out the issues. The important point that they are making, though, is that the idea that there are a lot of countries out there mired in situations that make an escape from food insecurity, material deprivation, absence of basic healthcare, and lack of opportunity (situations often called “poverty traps”) is being challenged by the ever-expanding pool of countries that seem to be increasing economic productivity rapidly and significantly. The whole point of a “poverty trap”, as popularized by Paul Collier’s book on The Bottom Billion and Jeffrey Sach’s various writings, is that it cannot be escaped without substantial outside aid interventions (a la Sachs) or may not be escapable at all. Well, Ghana certainly has received a lot of aid, but its massive growth is not the product of a new “big push”, a massive infusion of aid across sectors to get the country up into this new income category. Turns out the poorest people in the world might not need us to come riding to their rescue, at least not in the manner that Sachs envisions in his Millennium Villages Project.
That said, I’ve told Andy that I am deeply concerned about fragility – that is, I am thrilled to see things changing in places like Ghana, but how robust are those changes? At least in Ghana, a lot of the shift has been driven by the service sector, as opposed to recent oil finds (though these will undoubtedly swell the GNI figure in years to come) – this suggests a broader base to change in Ghana than, say, Equatorial Guinea . . . where GNI growth is all about oil, which is controlled by the country’s . . . problematic . . . leader (just read the Wikipedia post). But even in Ghana, things like climate change could present significant future challenges. The loss of the minor rainy season, for example, could have huge impacts on staple crop production and food security in the country, which in turn could hurt the workforce, exacerbate class/ethnic/rural-urban tensions, and generally hurt social cohesion in what is today a rather robust democracy. Yes, things have gotten better in Ghana . . . but this is no time to assume, a la Rostow, that a largely irreversible takeoff to economic growth has occurred. Aid and development are important and still needed in an increasingly middle-income world, but a different aid and development that supports existing indigenous efforts and consolidates development gains.
Category: Development Institutions
If you are uncomfortable, you are probably doing it right (Part 2)
In part 1, I argued that most academics who study development and aid have a very weak understanding of the processes they critique and seek to influence . . . and the only real way to build that understanding is to engage more seriously with development agencies. Why, then, have so few academics in the social sciences sought out such engagement – that is, why do so few academics work in development agencies as part of their training/research/practice? I think it has something to do with an unachievable desire to alter development practice and outcomes without unsettling ourselves. For example, many academics limit themselves to the critique of development practice to preserve some distance between themselves and the messy world of practice and policy. However, limiting oneself to critique still invokes an ethics of engagement, for if these critiques come too late to be acted upon, or do not speak to the institutional context from which these practices spring, the end result will be writing accessible only by other academics that has little if any benefit to those with whom we work in the Global South. This de facto extractive knowledge industry can hardly be seen as progressive, and its existence should upset us.
At the same time, holding ourselves apart from development practice out of a concern for being co-opted by (or used to legitimize) problematic political-economic agendas only makes sense if we treat development organizations as largely unchanging monoliths. This is a terribly ironic failure for a body of critical scholarship that otherwise spends so much time identifying and celebrating difference. Development agencies are not monoliths. For example, within these agencies are individuals deeply concerned about the rights of those affected by new forest carbon programmes, who object to the framing of development objectives in terms of economic growth, and who lament and struggle against the historical amnesia that marks the cyclical re-emergence of problematic and failed development initiatives. When we see development organizations as sites of contestation, unsettling questions arise. What is the point of critically-informed scholarship if not to provide support to individuals in their struggles to reshape policy, budget and programming into something more productive? What good will the most progressive, community-level effort come to if it can be plowed under by a single bad Country Development Cooperative Strategy (USAID) or Poverty Reduction Strategy Paper (World Bank)? What is the point of studying development, if not to intervene?
We cannot alter development without unsettling ourselves, as development requires us to think about the ideas of change and progress, and our role in both. I wrestle with this when I find myself arguing that the application of critical social theory to ‘development challenges’ can result in different and arguably more productive empirical understandings of events in the world (see here, here, here and here). This struggle helps me evaluate of my own positionality, motivations and expectations for such interventions. It is not a struggle that will come to a neat resolution. If indeed the path of the critical development geographer is between the equally untenable poles of uncritical self-justifying judgement and self-promoting intellectual resource extraction, then it is a path that is constantly fraught with tension. If you are unsettled, it means you are paying attention to this tension and trying to address it. If you are uncomfortable, you are probably doing it right.
If you are uncomfortable, you are probably doing it right (Part 1)
Back in April, I participated in a session on the role of geographers (and indeed academics more broadly) in development agencies. Though many outside of academia do not seem to know this, engagement with development agencies by those of us working in geography, anthropology and sociology tends to provoke both strong feelings and some controversy. Given geography’s and anthropology’s historical connection to colonialism, many academics fear that engagement with these agencies risks a return to these old relationships, where the work of academics serves to legitimize or even further neocolonial efforts. I thought the session was outstanding – the discussion was probably the most spirited I’d seen at an AAG, but it never degenerated into name-calling or other unproductive behavior.
Due to the success of and interest in the session, the participants in my panel decided to put together a forum of brief position pieces to be published in the Transactions of the Institute of British Geographers, hopefully later this year (screaming fast by academic standards). In my short piece, I took up the argument that we should be engaging with agencies more (probably not that surprising, considering where I work these days) – a position I supported in a distressingly well-read email exchange on a few big listservs this past fall (see a related blog post here). Before I submitted it, I had to get it cleared by Legislative and Public Affairs (LPA), which led to several people reading it. It was cleared without comment, which I believe only serves to support Bill Easterly’s claim (made in the context of the World Bank) that nobody really cares what we write in the academic journals, because they don’t think anyone reads them.
Along the way, though, my office director read it. Or, more to the point, he read it three times, because, as he put it, it was “impenetrable.” He did not say this dismissively, but instead to point out that the jargon in which I engaged in the piece (and I fully admit that my piece is very, very jargon-laden) made it nearly impossible to follow for the non-academic. To his credit, he read it three times to get my point . . . how many people do you know who are willing to do that?
So, in the spirit of his intervention, I offer a translation of my piece, in two parts. This is part 1.
Engagement with international development is fraught with tension. On one side lies a belief in improvement that carries with it judgment of the lives of others. At its worst, this judgment can become a justification for the lifestyles and foreign policy of “the developed” by placing both at the top of a pyramid of human progress to which everyone should aspire. On the other side is the peril of an extractive intellectual industry. When academic research and writing on development has no impact on policy and practice, it serves only to further the career of the researcher who gains from those s/he researches. It is not possible for an academic to engage development and remain unsullied by one, the other, or both. I see the job of the academic in development as walking between these extremes, balancing the risks of each. Therefore it is incumbent upon each of us to evaluate critically the path we walk between them.
It is very difficult for the contemporary academic to make such a critical evaluation. Critical development studies are often based upon a surprisingly thin understanding of the object of research. I can count on the fingers of one hand the development geographers who have worked in a development agency (receiving a contract from a development agency as a consultant or subcontractor does not count, as in that case one is only seeing the end product of a long process of policy building, budgeting, programming and contracting). Yet without an understanding of mundane bureaucratic moments such as budgeting, contracting and monitoring and evaluation it is simply impossible to understand why agencies do what they do, or reliably to identify points of intervention that might change practice in the world.
Though it was a book that brought me to critical development studies, Ferguson’s The Anti-Politics Machine is exemplary of this problem. Ferguson’s analysis of the Canadian International Development Agency’s (CIDA) Thaba-Tseka project is constrained largely to the reports and field programmes that are the outputs of this complex process. There is no doubt that he is correct about the ways in which CIDA’s representation of Lesotho and its challenges bore little resemblance to events on the ground. However, without a link to the institutional practices and structures that are inextricably bound up with these (mis)representations, Ferguson’s explanation for development failure comes to rest on a vague sense that language/representations (largely reflected in documents related to development projects and agencies) shape action. But this language, and these representations, are produced and reproduced in the often-byzantine interplay of policy, budget, programme and contracting that currently happens outside the scope of analysis for the bulk of academics. Pointing out the problematic character of CIDA’s representations of Lesotho is not in itself a productive intervention – we must know when this construction was put into play, by whom, and to what end. This information cannot be inferred from an organizational chart or a history of organizational actions. Instead, it requires ethnographic attention in its own right.
A very large proportion of critical development studies rests on this sort of incomplete analysis, resulting in critiques and questions that often have limited relevance to the experience of development practice. The mismatch of the products of such analysis with the experiences of those who occupy positions in development institutions is a source of the widening gulf between academic studies of development and the work of the development agencies we criticize and seek to influence. This suggests that productive critical interventions require greater direct engagement with development agencies.
Next up, Part 2: Why does this failure of understanding prevent serious engagement?
Open Data: This is a very big deal
It appears that the World Bank, at long last, is going to really make a huge portion of its data publicly available. The New York Times has a story that outlines some of the trials and tribulations that brought us to this point, some of which will probably seem arcane to the development outsider. However, as a development researcher/practitioner hybrid, I cannot tell you how exciting or important this is – the Bank is sitting on a giant pile of interesting data. Not all of it is going to be high quality (a lot of data from the Global South is not – see chapter 9 of Delivering Development or a parallel discussion in Charles Kenny’s Getting Better). But until very recently the data you could easily access from the Bank was worthy of a lower-division undergraduate project – and getting to the really interesting stuff was brutally difficult. The new datasets are more detailed and comprehensive, but still not everything the Bank has. Andy Sumner has been trying to get at the Bank’s core data to refine and test his ideas about the New Bottom Billion (which you should all be reading, by the way), with little success because of security requirements.
I really like a quote, at the end of the NY Times piece, from Bitange Ndemo, Kenya’s permanent secretary for information. When asked if there would be resistance to public dissemination of government data, he argued that transparency was inevitable because:
Information is valuable, he says, and people will find a way to get it: “This is one of those things, like mobile phones and the Internet, that you cannot control.”
The British lead . . . and who will follow?
David Cameron gave a speech yesterday at the Global Alliance for Vaccines and Immunisation conference. It deserves to be read in full – I don’t agree with every word (and how I disagree with many of Cameron’s stances), but it is one of the clearest statements on why we must continue to deliver aid to the poorest and most vulnerable people in the world.
On the down side, Cameron starts out a bit too market triumphalist for my tastes:
At home we don’t tackle poverty by state hand-outs; we help people get into work, to stand on their own two feet and to take control of their own destiny. The same should be true of development. No country has ever pulled itself out of poverty through aid alone, so this government will take a new approach. The same conditions create prosperity the world over. They include access to markets, property rights, private-sector investment and they make up what I see as the golden thread of successful development. Ultimately it’s the private sector that will be the engine for growth and that’s why this government’s efforts will increasingly focus on helping developing countries achieve that growth with the jobs and opportunities it will bring.
Well, this is a bit muddled. First, last I checked England (and Great Britain more generally) was home to a robust welfare state (well, until various Tory governments from Thatcher to Cameron took a hatchet to it) that provided the safety net that enhanced the quality of life of its citizens. On the other hand (and second), I agree that no country has ever been lifted out of poverty through aid alone – but then, that’s not what aid does. At best, aid catalyzes much larger processes of change – and sometimes those changes play out constructively (I discuss this at length in Delivering Development). Third, the only countries to have really changed their status in the last half century have done so by rejecting things like the open market and behaving in very politically repressive ways to get through a serious of difficult transitions that eventually made them competitive in global markets and able to productively take in foreign investment – so this claim about what works isn’t fully supported by the evidence. Andy Sumner’s work on the New Bottom Billion suggests that this might be changing as a new pile of countries “graduate” from low-income to middle-income status, but this is still unclear as many of the new “graduates” from low to middle income status have just crept above that line, often with no transformation of their economic fundamentals (leaving them vulnerable to slip-back) and still containing huge numbers of very poor people (creating the same problem, and calling into question the very concept of “graduation” to middle income status).
This is not to say that I don’t think markets have any value – I just fear those who place absolute faith in them, especially given that the environment is the site of perhaps the most serious market failure we’ve ever seen. However, as the speech progressed, I became somewhat more comfortable as, at least in the context of development, Cameron takes a somewhat more moderate tack:
We want people in Africa to climb the ladder of prosperity but of course when the bottom rungs of that ladder are broken by disease and preventable death on a massive scale, when countries can’t even get on the bottom rung of the growth ladder because one in seven of their children die before they reach their fifth birthday, we have to take urgent action. We have to save lives and then we can help people to live. So that’s where today’s announcement fits in. Because there cannot really be any effective development – economic or political – while there are still millions of people dying unnecessarily.
Absolutely correct – the “bottom of the pyramid”, as it were, often finds itself left behind when economic growth programs rev up . . . this is well-understood in both academia and the development institutions. Indeed, it is not controversial for my Bureau (DCHA – the folks who deal with disasters and conflict) to argue that its work is fundamental to creating a firm foundation for future development efforts because we address the needs of vulnerable populations who might otherwise be overlooked by Agency programming.
But what I most like is the kicking Cameron hands out to those who argue we don’t have the money for aid in these hard economic times. The kicking comes in two parts – first a moral argument:
When you make a promise to the poorest people in the world you should keep it. I remember where I was during the Gleneagles Summit and the Live 8 concert of 2005 and I remember thinking at the time how right it was that those world leaders should make such pledges so publicly. For me it’s a question of values; this is about saving lives. It was the right thing to promise; it was the right thing for Britain to do and it is the right thing for this government to honour that commitment.
So to those who point to other countries that are breaking their promises and say that makes it okay for us to do the same, I say no, it’s not okay. Our job is to hold those other countries to account, not to use them as an excuse to turn our back on people who are trusting us to help them. And to those who say fine but we should put off seeing through those promises to another day because right now we can’t afford to help, I say we can’t afford to wait. How many minutes do we wait? Three children die every minute from pneumonia alone; waiting is not the right thing to do and I don’t think that 0.7% of our gross national income is too high a price to pay for saving lives.
…
I actually think that most people in our country want Britain to stand for something in the world, to be something in the world. And when I think about what makes me proud of our country, yes, I think of our incredibly brave service men and women that I have the honour to meet and see so often; and yes, I think of our capabilities as an economic and diplomatic power; but I also think of our sense of duty to help others. That says something about this country and I think it’s something we can be proud of.
Where . . . the . . . hell . . . is . . . the . . . American . . . political . . . leadership . . . on . . . this? Dammit, the British just took the “City on a Hill” mantle from us. Most Americans want America to stand for something in the world, last I checked.
Oh, and Cameron addresses the unaddressable (for America, it seems) in his speech: that development, in reducing the need for future wars and humanitarian interventions, actually is cost-effective:
If we really care about Britain’s national interest, about jobs, about growth, about security, we shouldn’t break off our links with the countries that can hold some of the keys to that future. If we invest in Africa, if we open trade corridors, if we remove obstacles to growth, it’s not just Africa that will grow but us too. And if we invest in countries before they get broken we might not end up spending so much on dealing with the problems, whether that’s immigration or threats to our national security.
Take Afghanistan. If we’d put a fraction of our current military spending on Afghanistan into helping Afghanistan 15 or 20 years ago just think what we might have been able to avoid over the last decade. Or take Pakistan. Let another generation of Pakistanis enter adult life without any real opportunities and what are the risks in terms of mass migration, radicalisation, even terrorism? That’s why UK support over the next four years will get four million more children in Pakistan into school. This could be life changing for those children and it can be part of the antidote to the extremism that threatens us all. So it’s not just morally right to invest in aid, it’s actually in our own interests too.
God help us, Ron Paul seems to be the only candidate for anything willing to say that the wars we are in are costing a hell of a lot of money, and might not have been necessary. Of course, Ron Paul doesn’t like aid, either . . . actually, he doesn’t seem to like much of anything. Nobody is really taking his hobgoblin act all that seriously, which means he isn’t going to shift the debate here. Cameron, though, really glues his fiscal conservativism to a rational argument for aid – maybe we just should have worked on the aid side of things, at a fraction of the cost, and averted the whole mess in the first place. Lord help me, the Tories are sounding reasonable . . .
Now, Cameron’s ideas for transforming aid are vague, mostly about focusing on results and enhancing accountability. This is all well and good, but amazingly thorny. There’s been quite a bit of discussion about evaluation in the development community (great summary list here) and this blog (here, here and here) of late, and if nothing else, the reader might come to grips with the huge challenges that we must address before we can get to a realization of Cameron’s otherwise nonoffensive ideas.
I suppose it was asking too much to hope a leader talking about transforming development might mention that the global poor might actually have ideas of their own that we should start learning about before we go barging in . . .
3.36 Billion Africans in 2100?
Schuyler Null has a post up on The New Security Beat on the 2010 revision of the United Nations (UNDESA) World Population Prospects, noting that this new revision suggests that by 2100 roughly 1 in every 3 people in the world will live in sub-Saharan Africa – a total of 3.36 billion people. It is far too early to pick apart these projections, especially as the underlying assumptions used to guide their construction are not yet available to the public. Null is quite right to note:
the UN’s numbers are based on projections that can and do change. The range of uncertainty for the sub-Saharan African region, in particular, is quite large. The medium-variant projection for the region’s total population in 2100 is 3.36 billion people, but the high variant projection is 4.85 billion and the low variant is 2.25 billion.
A few preliminary thoughts, though. I pulled up the data for a country I know reasonably well – Ghana. Under this new revised projection, Ghana’s population is expected to reach more than 67 million by 2100. Peak population growth is supposed to take place between 2035 and 2040, with steady declines in population growth after that. With life expectancies projected to rise to 79 years by 2100, certainly a lot more Ghanaians will be around for a lot longer than they are today (current life expectancy is just shy of 60 years). That said, these numbers trouble me. First, I don’t quite see how Ghana will be able to sustain a population of this size at any point in the future – the number is just too massive. Second, it seems to me that the life expectancy estimates and the population size estimates contradict one another – as Charles Kenny quite ably demonstrates in Getting Better, as life expectancies rise and more children reach adulthood, the general trend is to lower total fertility. The only way Ghana’s projection can be made to work is to assume massive demographic momentum that I am not sure will play out in the face of expected declines in infant mortality and the increased cost burden for prospective parents supporting older family members for much longer than they do today. In other words, this seems to me to be a rather dire overestimation of where Ghana is going to be in the future.
Now, this is just a quick cut at what appear to be the assumptions for one country, but I worry that this potential overestimation has a certain political utility. The Malthusian specter, however inaccurate it may be, remains a great motivator for aid and development spending. Further, presuming massive demographic momentum requires we assume that adequate reproductive health options are not in place in places like Ghana. Given that the monitoring of reproductive health, presumably to better direct development interventions, seems to be a large focus of UNDESA’s and other UN organizations’ mandate, they might have a bit of a built-in bias against a lower population number because such a number would presume significant progress on the reproductive health front, thus challenging the need for this particular service. In a wider sense, it seems to revive fears of a population bomb, albeit in this case limited to Africa. While I have no doubt that demography will be an important challenge to address in the future, I think the current numbers, even the low estimates, seem overstated.
Besides, any projection of any social process 90 years into the future probably has gigantic error bars that could encompass anything from negative growth to massive overgrowth . . . the problem here is that policy makers often fail to grasp this uncertainty, see the 100-year projection, freak out entirely and reorient the next 5 years worth of aid programming to address a problem that may not exist.
Fisheries . . . this is a development challenge
A while back, I had a blog post on a report for ActionAid, written by Alex Evans, on critical uncertainties for development between the present and 2020. One of the big uncertainties Alex identified were environmental shocks, though in that version of the report he limited these shocks to climate-driven environmental shocks. In my post, I suggested to Alex that he widen his scope, for environmental shocks might also include ecosystem collapse, such as in major global fisheries – such environmental shocks are not really related to climate change, but are still of great importance. The collapse of the Gulf of Guinea large marine ecosystem (largely due to commercial overfishing from places other than Africa) has devastated local fish hauls, lowering the availability of protein in the diets of coastal areas and driving enormous pressure on terrestrial fauna as these populations seek to make up for the lost protein. Alex was quite generous with my comments, and agreed with this observation wholeheartedly.
And then today, I stumbled on this – a simple visualization of Atlantic Fisheries in 1900 and 2000, by fish haul. The image is striking (click to expand):
Now, I have no access to the datasets used to construct this visualization, and therefore I can make no comments on its accuracy (the blog post on the Guardian site is not very illuminating). However, this map could be off by quite a bit in terms of how good hauls were in 1900, and how bad they are now, and the picture would still be very, very chilling. As I keep telling my students, all those new, “exotic” fish showing up in restaurants are not delicacies – they are just all that is left in these fisheries.
This is obviously a development problem, as it compromises livelihoods and food supplies. Yet I don’t see anyone addressing it directly, even aid organizations engaged with countries on the coast of the Gulf of Guinea, where this impact is most pronounced. And how long until even the rich really start to feel the pinch?
Go here to see more visualizations – including one of the reach of the Spanish fishing fleet that makes clear where the pressure on the Gulf of Guinea is coming from.
Academic Adaptation and "The New Communications Climate"
Andrew Revkin has a post up on Dot Earth that suggests some ways of rethinking scientific engagement with the press and the public. The post is something of a distillation of a more detailed piece in the WMO Bulletin. Revkin was kind enough to solicit my comments on the piece, as I have appeared in Dot Earth before in an effort to deal with this issue as it applies to the IPCC, and this post is something of a distillation of my initial rapid response.
First, I liked the message of these two pieces a lot, especially the push for a more holistic engagement with the public through different forms of media, including the press. As Revkin rightly states, we need to “recognize that the old model of drafting a press release and waiting for the phone to ring is not the path to efficacy and impact.” Someone please tell my university communications office.
A lot of the problem stems from our lack of engagement with professionals in the messaging and marketing world. As I said to the very gracious Rajendra Pachauri in an email exchange back when we had the whole “don’t talk to the media” controversy:
I am in no way denigrating your [PR] efforts. I am merely suggesting that there are people out there who spend their lives thinking about how to get messages out there, and control that message once it is out there. Just as we employ experts in our research and in these assessment reports precisely because they bring skills and training to the table that we lack, so too we must consider bringing in those with expertise in marketing and outreach.
I assume that a decent PR team would be thinking about multiple platforms of engagement, much as Revkin is suggesting. However, despite the release of a new IPCC communications strategy, I’m not convinced that the IPCC (or much of the global change community more broadly) yet understands how desperately we need to engage with professionals on this front. In some ways, there are probably good reasons for the lack of engagement with pros, or with the “new media.” For example, I’m not sure Twitter will help with managing climate change rumors/misinformation as it is released, if only because we are now too far behind the curve – things are so politicized that it is too late for “rapid response” to misinformation. I wish we’d been on this twenty years ago, though . . .
But this “behind the curve” mentality does not explain our lack of engagement. Instead, I think there are a few other things lurking here. For example, there is the issue of institutional politics. I love the idea of using new media/information and communication technologies for development (ICT4D) to gather and communicate information, but perhaps not in the ways Revkin suggests. I have a section later in Delivering Development that outlines how, using existing mobile tech in the developing world, we could both get better information about what is happening to the global poor (the point of my book is that, as I think I demonstrate in great detail, we actually have a very weak handle on what is going on in most parts of the developing world) and could empower the poor to take charge of efforts to address the various challenges, environmental, economic, political and social, that they face every day. It seems to me, though, that the latter outcome is a terrifying prospect for some in development organizations, as this would create a much more even playing field of information that might force these organizations to negotiate with and take seriously the demands of the people with whom they are working. Thus, I think we get a sort of ambiguity about ICT4D in development practice, where we seem thrilled by its potential, yet continue to ignore it in our actual programming. This is not a technical problem – after all, we have the tech, and if we want to do this, we can – it is a problem of institutional politics. I did not wade into a detailed description of the network I envision in the book because I meant to present it as a political challenge to a continued reticence on the part of many development organizations and practitioners to really engage the global poor (as opposed to tell them what they need and dump it on them). But my colleagues and I have a detailed proposal for just such a network . . . and I think we will make it real one day.
Another, perhaps more significant barrier to major institutional shifts with regard to outreach is the a chicken-and-egg situation of limited budgets and a dominant academic culture that does not understand media/public engagement or politics very well and sees no incentive for engagement. Revkin nicely hits on the funding problem as he moves past simply beating up on old-school models of public engagement:
As the IPCC prepares its Fifth Assessment Report, it does so with what, to my eye, appears to be an utterly inadequate budget for communicating its findings and responding in an agile way to nonstop public scrutiny facilitated by the Internet.
However, as much as I agree with this point (and I really, really agree), the problem here is not funding unto itself – it is the way in which a lack of funding erases an opportunity for cultural change that could have a positive feedback effect on the IPCC, global assessments, and academia more generally that radically alters all three. The bulk of climate science, as well as social impact studies, come from academia – which has a very particular culture of rewards. Virtually nobody in academia is trained to understand that they can get rewarded for being a public intellectual, for making one’s work accessible to a wide community – and if I am really honest, there are many places that actively discourage this engagement. But there is a culture change afoot in academia, at least among some of us, that could be leveraged right now – and this is where funding could trigger a positive feedback loop.
Funding matters because once you get a real outreach program going, productive public engagement would result in significant personal, intellectual and financial benefits for the participants that I believe could result in very rapid culture change. My twitter account has done more for the readership of my blog, and for my awareness of the concerns and conversations of the non-academic development world, than anything I have ever done before – this has been a remarkable personal and intellectual benefit of public engagement for me. As universities continue to retrench, faculty find themselves ever-more vulnerable to downsizing, temporary appointments, and a staggering increase in administrative workload (lots of tasks distributed among fewer and fewer full-time faculty). I fully expect that without some sort of serious reversal soon, I will retire thirty-odd years hence as an interesting and very rare historical artifact – a professor with tenure. Given these pressures, I have been arguing to my colleagues that we must engage with the public and with the media to build constituencies for what we do beyond our academic communities. My book and my blog are efforts to do just this – to become known beyond the academy such that I, as a public intellectual, have leverage over my university, and not the other way around. And I say this as someone who has been very successful in the traditional academic model. I recognize that my life will need to be lived on two tracks now – public and academic – if I really want to help create some of the changes in the world that I see as necessary.
But this is a path I started down on my own, for my own idiosyncratic reasons – to trigger a wider change, we cannot assume that my academic colleagues will easily shed the value systems in which they were intellectually raised, and to which they have been held for many, many years. Without funding to get outreach going, and demonstrate to this community that changing our model is not only worthwhile, but enormously valuable, I fear that such change will come far more slowly than the financial bulldozers knocking on the doors of universities and colleges across the country. If the IPCC could get such an effort going, demonstrate how public outreach improved the reach of its results, enhanced the visibility and engagement of its participants, and created a path toward the progressive politics necessary to address the challenge of climate change, it would be a powerful example for other assessments. Further, the participants in these assessments would return to their campuses with evidence for the efficacy and importance of such engagement . . . and many of these participants are senior members of their faculties, in a position to midwife major cultural changes in their institutions.
All this said, this culture change will not be birthed without significant pains. Some faculty and members of these assessments want nothing to do with the murky world of politics, and prefer to continue operating under the illusion that they just produce data and have no responsibility for how it is used. And certainly the assessments will fear “politicization” . . . to which I respond “too late.” The question is not if the findings of an assessment will be politicized, but whether or not those who best understand those findings will engage in these very consequential debates and argue for what they feel is the most rigorous interpretation of the data at hand. Failure to do so strikes me as dereliction of duty. On the other hand, just as faculty might come to see why public engagement is important for their careers and the work they do, universities will be gripped with contradictory impulses – a publicly-engaged faculty will serve as a great justification for faculty salaries, increased state appropriations, new facilities, etc. Then again, nobody likes to empower the labor, as it were . . .
In short, in thinking about public engagement and the IPCC, Revkin is dredging up a major issue related to all global assessments, and indeed the practices of academia. I think there is opportunity here – and I feel like we must seize this opportunity. We can either guide a process of change to a productive end, or ride change driven by others wherever it might take us. I prefer the former.
The Qualitative Research Challenge to RCT4D: Part 2
Well, the response to part one was great – really good comments, and a few great response posts. I appreciate the efforts of some of my economist colleagues/friends to clarify the terminology and purpose behind RCTs. All of this has been very productive for me – and hopefully for others engaged in this conversation.
First, a caveat: On the blog I tend to write quickly and with minimal editing – so I get a bit fast and loose at times – well, faster and looser than I intend. So, to this end, I did not mean to suggest that nobody was doing rigorous work in development research – in fact, the rest of my post clearly set out to refute that idea, at least in the qualitative sphere. But I see how Marc Bellemare might have read me that way. What I should have said was that there has always been work, both in research and implementation, where rigorous data collection and analysis were lacking. In fact, there is quite a lot of this work. I think we can all agree this is true . . . and I should have been clearer.
I have also learned that what qualitative social scientists/social theorists mean by theory, and what economists mean by theory, seems to be two different things. Lee defined theory as “formal mathematical modeling” in a comment on part 1 of this series of posts, which is emphatically not what a social theorist might mean. When I say theory, I am talking about a conjectural framing of a social totality such that complex causality can at least be contained, if not fully explained. This framing should have reference to some sort of empirical evidence, and therefore should be testable and refinable over time – perhaps through various sorts of ethnographic work, perhaps through formal mathematical modeling of the propositions at hand (I do a bit of both, actually). In other words, what I mean by theory (and what I focus on in my work) is the establishment of a causal architecture for observed social outcomes. I am all about the “why it worked” part of research, and far less about the “if it worked” questions – perhaps mostly because I have researched unintended “development interventions” (i.e. unplanned road construction, the establishment of a forest reserve that alters livelihoods resource access, etc.) that did not have a clear goal, a clear “it worked!” moment to identify. All I have been looking at are outcomes of particular events, and trying to establish the causes of those outcomes. Obviously, this can be translated to an RCT environment because we could control for the intervention and expected outcome, and then use my approaches to get at the “why did it work/not work” issues.
It has been very interesting to see the economists weigh in on what RCTs really do – they establish, as Marc puts it, “whether something works, not in how it works.” (See also Grant’s great comment on the first post). I don’t think that I would get a lot of argument from people if I noted that without causal mechanisms, we can’t be sure why “what worked” actually worked, and whether the causes of “what worked” are in any way generalizable or transportable. We might have some idea, but I would have low confidence in any research that ended at this point. This, of course, is why Marc, Lee, Ruth, Grant and any number of other folks see a need for collaboration between quant and qual – so that we can get the right people, with the right tools, looking at different aspects of a development intervention to rigorously establish the existence of an impact, and the establish an equally rigorous understanding of the causal processes by which that impact came to pass. Nothing terribly new here, I think. Except, of course, for my continued claim that the qualitative work I do see associated with RCT work is mostly awful, tending toward bad journalism (see my discussion of bad journalism and bad qualitative work in the first post).
But this discussion misses a much larger point about epistemology – what I intended to write in this second part of the series all along. I do not see the dichotomy between measuring “if something works” and establishing “why something worked” as analytically valid. Simply put, without some (at least hypothetical) framing of causality, we cannot rigorously frame research questions around either question. How can you know if something worked, if you are not sure how it was supposed to work in the first place? Qualitative research provides the interpretive framework for the data collected via RCT4D efforts – a necessary framework if we want RCT4D work to be rigorous. By separating qualitative work from the quant oriented RCT work, we are assuming that somehow we can pull data collection apart from the framing of the research question. We cannot – nobody is completely inductive, which means we all work from some sort of framing of causality. The danger is when we don’t acknowledge this simple point – under most RCT4D work, those framings are implicit and completely uninterrogated by the practitioners. Even where they come to the fore (Duflo’s 3 I s), they are not interrogated – they are assumed as framings for the rest of the analysis.
If we don’t have causal mechanisms, we cannot rigorously frame research questions to see if something is working – we are, as Marc says, “like the drunk looking for his car keys under the street lamp when he knows he lost them elsewhere, because the only place he can actually see is under the street lamp.” Only I would argue we are the drunk looking for his keys under a streetlamp, but he has no idea if they are there or not.
In short, I’m not beating up on RCT4D, nor am I advocating for more conversation – no, I am arguing that we need integration, teams with quant and qual skills that frame the research questions together, that develop tests together, that interpret the data together. This is the only way we will come to really understand the impact of our interventions, and how to more productively frame future efforts. Of course, I can say this because I already work in a mixed-methods world where my projects integrate the skills of GIScientists, land use modelers, climate modelers, biogeographers and qualitative social scientists – in short, I have a degree of comfort with this sort of collaboration. So, who wants to start putting together some seriously collaborative, integrated evaluations?
The Qualitative Research Challenge to RCT4D: Part 1
Those following this blog (or my twitter feed) know that I have some issues with RCT4D work. I’m actually working on a serious treatment of the issues I see in this work (i.e. journal article), but I am not above crowdsourcing some of my ideas to see how people respond. Also, as many of my readers know, I have a propensity for really long posts. I’m going to try to avoid that here by breaking this topic into two parts. So, this is part 1 of 2.
To me, RCT4D work is interesting because of its emphasis on rigorous data collection – certainly, this has long been a problem in development research, and I have no doubt that the data they are gathering is valid. However, part of the reason I feel confident in this data is because, as I raised in an earlier post, it is replicating findings from the qualitative literature . . . findings that are, in many cases, long-established with rigorously-gathered, verifiable data. More on that in part 2 of this series.
One of the things that worries me about the RCT4D movement is the (at least implicit, often overt) suggestion that other forms of development data collection lack rigor and validity. However, in the qualitative realm we spend a lot of time thinking about rigor and validity, and how we might achieve both – and there are tools we use to this end, ranging from discursive analysis to cross-checking interviews with focus groups and other forms of data. Certainly, these are different means of establishing rigor and validity, but they are still there.
Without rigor and validity, qualitative research falls into bad journalism. As I see it, good journalism captures a story or an important issue, and illustrates that issue through examples. These examples are not meant to rigorously explain the issue at hand, but to clarify it or ground it for the reader. When journalists attempt to move to explanation via these same few examples (as far too often columnists like Kristof and Friedman do), they start making unsubstantiated claims that generally fall apart under scrutiny. People mistake this sort of work for qualitative social science all the time, but it is not. Certainly there is some really bad social science out there that slips from illustration to explanation in just the manner I have described, but this is hardly the majority of the work found in the literature. Instead, rigorous qualitative social science recognizes the need to gather valid data, and therefore requires conducting dozens, if not hundreds, of interviews to establish understandings of the events and processes at hand.
This understanding of qualitative research stands in stark contrast to what is in evidence in the RCT4D movement. For all of the effort devoted to data collection under these efforts, there is stunningly little time and energy devoted to explanation of the patterns seen in the data. In short, RCT4D often reverts to bad journalism when it comes time for explanation. Patterns gleaned from meticulously gathered data are explained in an offhand manner. For example, in her (otherwise quite well-done) presentation to USAID yesterday, Esther Duflo suggested that some problematic development outcomes could be explained by a combination of “the three I s”: ideology, ignorance and inertia. This is a boggling oversimplification of why people do what they do – ideology is basically nondiagnostic (you need to define and interrogate it before you can do anything about it), and ignorance and inertia are (probably unintentionally) deeply patronizing assumptions about people living in the Global South that have been disproven time and again (my own work in Ghana has demonstrated that people operate with really fine-grained information about incomes and gender roles, and know exactly what they are doing when they act in a manner that limits their household incomes – see here, here and here). Development has claimed to be overcoming ignorance and inertia since . . . well, since we called it colonialism. Sorry, but that’s the truth.
Worse, this offhand approach to explanation is often “validated” through reference to a single qualitative case that may or may not be representative of the situation at hand – this is horribly ironic for an approach that is trying to move development research past the anecdotal. This is not merely external observation – I have heard from people working inside J-PAL projects that the overall program puts little effort into serious qualitative work, and has little understanding of what rigor and validity might mean in the context of qualitative methods or explanation. In short, the bulk of explanation for these interesting patterns of behavior that emerges from these studies resorts to uninterrogated assumptions about human behavior that do not hold up to empirical reality. What RCT4D has identified are patterns, not explanations – explanation requires a contextual understanding of the social.
Coming soon: Part 2 – Qualitative research and the interpretation of empirical data