Tag: academia

Wait, economists get to just guess and call it explanation?

An interesting review of Paul Collier’s The Bottom Billion and Wars, Guns and Votes by Yale Anthropologist Mike McGovern has gotten a little bit of attention recently in development circles, speaking as it does to ongoing debates about the role of statistical analysis, what counts as explanation, and where qualitative research fits into all of this.  I will take up McGovern’s good (but incomplete, in my opinion) review in another post.  Here, I needed to respond to a blog entry about this review.
On the Descriptive Statistics, Causal Inference and Social Science blog, Andrew Gelman discusses McGovern’s review.  While there is a lot going on in this post, one issue caught my attention in particular.  In his review, McGovern argues that “Much of the intellectual heavy lifting in these books is in fact done at the level of implication or commonsense guessing,” what Gelman (quoting Fung) calls “story time”, the “pivot from the quantitative finding to the speculative explanation.”  However, despite the seemingly dismissive term for this sort of explanation, in his blog post Gelman argues “story time can’t be avoided.” His point:

On one hand, there are real questions to be answered and real decisions to be made in development economics (and elsewhere), and researchers and policymakers can’t simply sit still and say they can’t do anything because the data aren’t fully persuasive. (Remember the first principle of decision analysis: Not making a decision is itself a decision.)

From the other direction, once you have an interesting quantitative finding,of course you want to understand it, and it makes sense to use all your storytelling skills here. The challenge is to go back and forth between the storytelling and the data. You find some interesting result (perhaps an observational data summary, perhaps an analysis of an experiment or natural experiment), this motivates a story, which in turn suggests some new hypotheses to be studied.

Now, on one hand I take his point – research is iterative, and answering one set of questions (or one set of new data) often raises new questions which can be interrogated.  But Gelman seems to presume that explanation only comes from more statistical analysis, without considering what I saw as McGovern’s subtle point: qualitative social scientists look at explanation, and do not revert to story time to do so (good luck getting published if you do).  We spend a hell of a lot of time fleshing out the causal processes behind our observations, including establishing rigor and validity for our data and conclusions, before we present stories.  This is not to say that our explanations are immediately complete or perfect, nor is it to suggest that our explanations do not raise new questions to pursue.  However, there is no excuse for the sort of “story time” analysis that McGovern is pointing out in Collier’s work – indeed, I would suggest that is why the practice is given a clearly derisive title.  That is just guessing, vaguely informed by data, often without even thinking through alternative explanations for the patterns at hand (let alone presenting those alternatives).
I agree with Gelman’s point, late in the post – this is not a failing of statistics, really.  It is a failure to use them intelligently, or to use appropriate frameworks to interpret statistical findings.  It would be nice, however, if we could have a discussion between quant and qual on how to avoid these outcomes before they happen . . . because story time is most certainly avoidable.

Posted on by Ed 3

If you are uncomfortable, you are probably doing it right (Part 2)

In part 1, I argued that most academics who study development and aid have a very weak understanding of the processes they critique and seek to influence . . . and the only real way to build that understanding is to engage more seriously with development agencies.  Why, then, have so few academics in the social sciences sought out such engagement – that is, why do so few academics work in development agencies as part of their training/research/practice?  I think it has something to do with an unachievable desire to alter development practice and outcomes without unsettling ourselves.  For example, many academics limit themselves to the critique of development practice to preserve some distance between themselves and the messy world of practice and policy.  However, limiting oneself to critique still invokes an ethics of engagement, for if these critiques come too late to be acted upon, or do not speak to the institutional context from which these practices spring, the end result will be writing accessible only by other academics that has little if any benefit to those with whom we work in the Global South.  This de facto extractive knowledge industry can hardly be seen as progressive, and its existence should upset us.
At the same time, holding ourselves apart from development practice out of a concern for being co-opted by (or used to legitimize) problematic political-economic agendas only makes sense if we treat development organizations as largely unchanging monoliths.  This is a terribly ironic failure for a body of critical scholarship that otherwise spends so much time identifying and celebrating difference.  Development agencies are not monoliths.  For example, within these agencies are individuals deeply concerned about the rights of those affected by new forest carbon programmes, who object to the framing of development objectives in terms of economic growth, and who lament and struggle against the historical amnesia that marks the cyclical re-emergence of problematic and failed development initiatives.  When we see development organizations as sites of contestation, unsettling questions arise.  What is the point of critically-informed scholarship if not to provide support to individuals in their struggles to reshape policy, budget and programming into something more productive?  What good will the most progressive, community-level effort come to if it can be plowed under by a single bad Country Development Cooperative Strategy (USAID) or Poverty Reduction Strategy Paper (World Bank)?  What is the point of studying development, if not to intervene?
We cannot alter development without unsettling ourselves, as development requires us to think about the ideas of change and progress, and our role in both.  I wrestle with this when I find myself arguing that the application of critical social theory to ‘development challenges’ can result in different and arguably more productive empirical understandings of events in the world (see here, here, here and here).  This struggle helps me evaluate of my own positionality, motivations and expectations for such interventions.  It is not a struggle that will come to a neat resolution.  If indeed the path of the critical development geographer is between the equally untenable poles of uncritical self-justifying judgement and self-promoting intellectual resource extraction, then it is a path that is constantly fraught with tension.  If you are unsettled, it means you are paying attention to this tension and trying to address it.  If you are uncomfortable, you are probably doing it right.

Posted on by Ed 8

If you are uncomfortable, you are probably doing it right (Part 1)

Back in April, I participated in a session on the role of geographers (and indeed academics more broadly) in development agencies.  Though many outside of academia do not seem to know this, engagement with development agencies by those of us working in geography, anthropology and sociology tends to provoke both strong feelings and some controversy.  Given geography’s and anthropology’s historical connection to colonialism, many academics fear that engagement with these agencies risks a return to these old relationships, where the work of academics serves to legitimize or even further neocolonial efforts.  I thought the session was outstanding – the discussion was probably the most spirited I’d seen at an AAG, but it never degenerated into name-calling or other unproductive behavior.
Due to the success of and interest in the session, the participants in my panel decided to put together a forum of brief position pieces to be published in the Transactions of the Institute of British Geographers, hopefully later this year (screaming fast by academic standards).  In my short piece, I took up the argument that we should be engaging with agencies more (probably not that surprising, considering where I work these days) – a position I supported in a distressingly well-read email exchange on a few big listservs this past fall (see a related blog post here).  Before I submitted it, I had to get it cleared by Legislative and Public Affairs (LPA), which led to several people reading it.  It was cleared without comment, which I believe only serves to support Bill Easterly’s claim (made in the context of the World Bank) that nobody really cares what we write in the academic journals, because they don’t think anyone reads them.
Along the way, though, my office director read it.  Or, more to the point, he read it three times, because, as he put it, it was “impenetrable.”  He did not say this dismissively, but instead to point out that the jargon in which I engaged in the piece (and I fully admit that my piece is very, very jargon-laden) made it nearly impossible to follow for the non-academic.  To his credit, he read it three times to get my point . . . how many people do you know who are willing to do that?
So, in the spirit of his intervention, I offer a translation of my piece, in two parts.  This is part 1.
Engagement with international development is fraught with tension.  On one side lies a belief in improvement that carries with it judgment of the lives of others.  At its worst, this judgment can become a justification for the lifestyles and foreign policy of “the developed” by placing both at the top of a pyramid of human progress to which everyone should aspire.  On the other side is the peril of an extractive intellectual industry.  When academic research and writing on development has no impact on policy and practice, it serves only to further the career of the researcher who gains from those s/he researches.  It is not possible for an academic to engage development and remain unsullied by one, the other, or both.  I see the job of the academic in development as walking between these extremes, balancing the risks of each. Therefore it is incumbent upon each of us to evaluate critically the path we walk between them.
It is very difficult for the contemporary academic to make such a critical evaluation.  Critical development studies are often based upon a surprisingly thin understanding of the object of research.  I can count on the fingers of one hand the development geographers who have worked in a development agency (receiving a contract from a development agency as a consultant or subcontractor does not count, as in that case one is only seeing the end product of a long process of policy building, budgeting, programming and contracting). Yet without an understanding of mundane bureaucratic moments such as budgeting, contracting and monitoring and evaluation it is simply impossible to understand why agencies do what they do, or reliably to identify points of intervention that might change practice in the world.
Though it was a book that brought me to critical development studies, Ferguson’s The Anti-Politics Machine is exemplary of this problem.  Ferguson’s analysis of the Canadian International Development Agency’s (CIDA) Thaba-Tseka project is constrained largely to the reports and field programmes that are the outputs of this complex process.  There is no doubt that he is correct about the ways in which CIDA’s representation of Lesotho and its challenges bore little resemblance to events on the ground.  However, without a link to the institutional practices and structures that are inextricably bound up with these (mis)representations, Ferguson’s explanation for development failure comes to rest on a vague sense that language/representations (largely reflected in documents related to development projects and agencies) shape action.  But this language, and these representations, are produced and reproduced in the often-byzantine interplay of policy, budget, programme and contracting that currently happens outside the scope of analysis for the bulk of academics.  Pointing out the problematic character of CIDA’s representations of Lesotho is not in itself a productive intervention – we must know when this construction was put into play, by whom, and to what end.  This information cannot be inferred from an organizational chart or a history of organizational actions.  Instead, it requires ethnographic attention in its own right.
A very large proportion of critical development studies rests on this sort of incomplete analysis, resulting in critiques and questions that often have limited relevance to the experience of development practice.  The mismatch of the products of such analysis with the experiences of those who occupy positions in development institutions is a source of the widening gulf between academic studies of development and the work of the development agencies we criticize and seek to influence.  This suggests that productive critical interventions require greater direct engagement with development agencies.
Next up, Part 2: Why does this failure of understanding prevent serious engagement?

Posted on by Ed 11

Academic Adaptation and "The New Communications Climate"

Andrew Revkin has a post up on Dot Earth that suggests some ways of rethinking scientific engagement with the press and the public.  The post is something of a distillation of a more detailed piece in the WMO Bulletin.  Revkin was kind enough to solicit my comments on the piece, as I have appeared in Dot Earth before in an effort to deal with this issue as it applies to the IPCC, and this post is something of a distillation of my initial rapid response.
First, I liked the message of these two pieces a lot, especially the push for a more holistic engagement with the public through different forms of media, including the press.  As Revkin rightly states, we need to “recognize that the old model of drafting a press release and waiting for the phone to ring is not the path to efficacy and impact.” Someone please tell my university communications office.
A lot of the problem stems from our lack of engagement with professionals in the messaging and marketing world.  As I said to the very gracious Rajendra Pachauri in an email exchange back when we had the whole “don’t talk to the media” controversy:

I am in no way denigrating your [PR] efforts. I am merely suggesting that there are people out there who spend their lives thinking about how to get messages out there, and control that message once it is out there. Just as we employ experts in our research and in these assessment reports precisely because they bring skills and training to the table that we lack, so too we must consider bringing in those with expertise in marketing and outreach.

I assume that a decent PR team would be thinking about multiple platforms of engagement, much as Revkin is suggesting.  However, despite the release of a new IPCC communications strategy, I’m not convinced that the IPCC (or much of the global change community more broadly) yet understands how desperately we need to engage with professionals on this front.  In some ways, there are probably good reasons for the lack of engagement with pros, or with the “new media.” For example, I’m not sure Twitter will help with managing climate change rumors/misinformation as it is released, if only because we are now too far behind the curve – things are so politicized that it is too late for “rapid response” to misinformation. I wish we’d been on this twenty years ago, though . . .
But this “behind the curve” mentality does not explain our lack of engagement.  Instead, I think there are a few other things lurking here.  For example, there is the issue of institutional politics. I love the idea of using new media/information and communication technologies for development (ICT4D) to gather and communicate information, but perhaps not in the ways Revkin suggests.  I have a section later in Delivering Development that outlines how, using existing mobile tech in the developing world, we could both get better information about what is happening to the global poor (the point of my book is that, as I think I demonstrate in great detail, we actually have a very weak handle on what is going on in most parts of the developing world) and could empower the poor to take charge of efforts to address the various challenges, environmental, economic, political and social, that they face every day.  It seems to me, though, that the latter outcome is a terrifying prospect for some in development organizations, as this would create a much more even playing field of information that might force these organizations to negotiate with and take seriously the demands of the people with whom they are working.  Thus, I think we get a sort of ambiguity about ICT4D in development practice, where we seem thrilled by its potential, yet continue to ignore it in our actual programming.  This is not a technical problem – after all, we have the tech, and if we want to do this, we can – it is a problem of institutional politics.  I did not wade into a detailed description of the network I envision in the book because I meant to present it as a political challenge to a continued reticence on the part of many development organizations and practitioners to really engage the global poor (as opposed to tell them what they need and dump it on them).  But my colleagues and I have a detailed proposal for just such a network . . . and I think we will make it real one day.
Another, perhaps more significant barrier to major institutional shifts with regard to outreach is the a chicken-and-egg situation of limited budgets and a dominant academic culture that does not understand media/public engagement or politics very well and sees no incentive for engagement.  Revkin nicely hits on the funding problem as he moves past simply beating up on old-school models of public engagement:

As the IPCC prepares its Fifth Assessment Report, it does so with what, to my eye, appears to be an utterly inadequate budget for communicating its findings and responding in an agile way to nonstop public scrutiny facilitated by the Internet.

However, as much as I agree with this point (and I really, really agree), the problem here is not funding unto itself – it is the way in which a lack of funding erases an opportunity for cultural change that could have a positive feedback effect on the IPCC, global assessments, and academia more generally that radically alters all three. The bulk of climate science, as well as social impact studies, come from academia – which has a very particular culture of rewards.  Virtually nobody in academia is trained to understand that they can get rewarded for being a public intellectual, for making one’s work accessible to a wide community – and if I am really honest, there are many places that actively discourage this engagement.  But there is a culture change afoot in academia, at least among some of us, that could be leveraged right now – and this is where funding could trigger a positive feedback loop.
Funding matters because once you get a real outreach program going, productive public engagement would result in significant personal, intellectual and financial benefits for the participants that I believe could result in very rapid culture change.  My twitter account has done more for the readership of my blog, and for my awareness of the concerns and conversations of the non-academic development world, than anything I have ever done before – this has been a remarkable personal and intellectual benefit of public engagement for me.  As universities continue to retrench, faculty find themselves ever-more vulnerable to downsizing, temporary appointments, and a staggering increase in administrative workload (lots of tasks distributed among fewer and fewer full-time faculty).  I fully expect that without some sort of serious reversal soon, I will retire thirty-odd years hence as an interesting and very rare historical artifact – a professor with tenure.  Given these pressures, I have been arguing to my colleagues that we must engage with the public and with the media to build constituencies for what we do beyond our academic communities.  My book and my blog are efforts to do just this – to become known beyond the academy such that I, as a public intellectual, have leverage over my university, and not the other way around.  And I say this as someone who has been very successful in the traditional academic model.  I recognize that my life will need to be lived on two tracks now – public and academic – if I really want to help create some of the changes in the world that I see as necessary.
But this is a path I started down on my own, for my own idiosyncratic reasons – to trigger a wider change, we cannot assume that my academic colleagues will easily shed the value systems in which they were intellectually raised, and to which they have been held for many, many years.  Without funding to get outreach going, and demonstrate to this community that changing our model is not only worthwhile, but enormously valuable, I fear that such change will come far more slowly than the financial bulldozers knocking on the doors of universities and colleges across the country.  If the IPCC could get such an effort going, demonstrate how public outreach improved the reach of its results, enhanced the visibility and engagement of its participants, and created a path toward the progressive politics necessary to address the challenge of climate change, it would be a powerful example for other assessments.  Further, the participants in these assessments would return to their campuses with evidence for the efficacy and importance of such engagement . . . and many of these participants are senior members of their faculties, in a position to midwife major cultural changes in their institutions.
All this said, this culture change will not be birthed without significant pains.  Some faculty and members of these assessments want nothing to do with the murky world of politics, and prefer to continue operating under the illusion that they just produce data and have no responsibility for how it is used.  And certainly the assessments will fear “politicization” . . . to which I respond “too late.”  The question is not if the findings of an assessment will be politicized, but whether or not those who best understand those findings will engage in these very consequential debates and argue for what they feel is the most rigorous interpretation of the data at hand.  Failure to do so strikes me as dereliction of duty.  On the other hand, just as faculty might come to see why public engagement is important for their careers and the work they do, universities will be gripped with contradictory impulses – a publicly-engaged faculty will serve as a great justification for faculty salaries, increased state appropriations, new facilities, etc.  Then again, nobody likes to empower the labor, as it were . . .
In short, in thinking about public engagement and the IPCC, Revkin is dredging up a major issue related to all global assessments, and indeed the practices of academia.  I think there is opportunity here – and I feel like we must seize this opportunity.  We can either guide a process of change to a productive end, or ride change driven by others wherever it might take us.  I prefer the former.

Posted on by Ed 7

And another thing . . .

Would folks who know precious little about development please stop telling everyone what the discipline of development looks like?  Seriously. Francis Fukuyama has a piece in the American Interest in which he decries the lack of what he calls “large perspective” work in the social sciences. Admittedly, I have some sympathy for his position here – like all academic disciplines, the social sciences generally reward narrow specialization, or at least that is what most of us are trained to believe.  I think there is another way to succeed in academia, a path I am taking – to write not only high quality, refereed research in one’s field(s), but also general-audiences works that gain a wider profile (that was the point of writing Delivering Development).  When you reach audiences beyond academia, you develop other lines of influence, other sources of funding . . . and generally give yourself some space in your home institution, as nobody wants to fire/lose the visible public intellectual.  Sadly, few of us choose the buck the system in this manner, and therefore become slaves to our journals and their relatively narrow audiences.
I also like Fukuyama’s clear argument about the goals of social science:

“The aspiration of social science to replicate the predictability and formality of certain natural sciences is, in the end, a hopeless endeavor. Human societies, as Friedrich Hayek, Karl Popper and others understood, are far too complex to model at an aggregate level.”

Yes, yes, a thousand times yes.  When we refuse to admit this, we empower the people who are willing to take problematic data and jam it through dicey quantitative tools to produce semi-coherent, super-shallow analyses that appear to present simple framings of the world and solutions to our problems while in fact they obscure any real understanding of what is going on, and what might be done.
But in between these two points, made at the beginning and end of the article, respectively, Fukuyama populates his piece with a number of statements about development that range from the problematic to the factually incorrect.  In the end, I am forced to conclude that he has little, if any, understanding of contemporary development in theory or practice.  Sadly, this did not keep him from making a number of sweeping, highly erroneous statements.  For example, at one point he makes the claim

Few scholars have sought to understand development as an inter-connected process with political, economic and social parts.

This claim exists to further his argument that development is plagued by siloed thinking that has led to intellectual incoherence and failed policy. While I might agree about development having problems with its intellectual coherence, he is totally wrong in this claim. It only holds up if one chooses to NOT use something as ubiquitous as Google Scholar (let alone Web of Science) to examine the literature of the past 20 years.  Anthropologists, geographers and sociologists have been doing just this sort of work, mostly at the community level, all along.  Often the lessons of this work are not aimed beyond the communities in which the work was undertaken, but there is a giant volume of work out there that has long taken this interconnection seriously.
Further, Fukuyama’s ignorance of the current state of the discipline and practice of development shows in his claim:

While paying lip service to the importance of institutions, most economists and field practitioners still see politics as at best an obstacle to the real work of development, which is improvement in incomes, health, education and the like, and not as an independent objective of development strategy. (Amartya Sen is an important exception to this generalization.) The democracy promotion agencies, for their part, spend relatively little time worrying about economic growth, social policy or public health, which in their view are goods often used by authoritarian regimes to buy off populations and prevent democratization.

While some economists still treat “the social” as maximizing behavior warped by a bunch of externalities, those that are any good concern themselves with politics (at scales from the state to the household).  Practitioners, perhaps more than anyone else, know that politics are hugely important to the work of development.  Sen has a wide purchase and following throughout development, including at my current employer.  And how does one then address the Democracy and Governance Office in my Bureau – they are, without question, a democracy promotion office . . . but their whole lives revolve around linking this to various other development efforts like economic growth or public health. When he claims that those who work for USAID “do not seek an understanding of the political context within which aid is used and abused” he’s simply factually incorrect. Basically, Fukuyama is just throwing out huge claims that have little or no anchor in the reality of contemporary development agencies or practice.
Fukuyama’s article was not really about development – it was about understanding social change.  However, in using development as his foil in this piece, Fukuyama has done a great disservice to the contemporary discipline – both in its good and bad aspects.  Like those who would give us useless universalizing generalizations and predictions from their social inquiries, Fukuyama’s (mis)reading of development makes it harder to see where the real problems are, and how we might address them.

Posted on by Ed 1