Polishing a turd? Another day, another index

UNDP and the Oxford Environment and Human Development Initiative recently announced the launch of the Multidimensional Poverty Index (MPI), the newest rapid poverty assessment tool.  This is the latest effort to expand the measurement of poverty beyond indicators of economic productivity, and is being hailed (at least by UNDP and OEHDI) as a significant advance in our efforts to understand the nature of poverty.  I’m not so sure . . .
We have tried to come up with quick measures (often referred to as indicators) of things like development, poverty and food insecurity for decades.  We chase after such indicators because, if they provide us with quick, cheap understandings of the human condition in particular places, they can guide policy and program design, thus maximizing the benefit of the aid money we spend around the world.   Since the mid-twentieth century, development thought has attached to various indicators of poverty and development.  For example, one of the earliest (and still prevalent) indicators of development is the Gross Domestic Product (GDP), which measures the value of all goods and services produced in a country in a given year.  GNP per capita is the number you get when you divide this value by the population of the country at hand, thus getting a measure of average per-person economic productivity.  The presumption here is that this average economic productivity reflects wages, and thus the ability of individuals to meet their material needs.  It certainly means something that the per capita GDP of the United States was $46350 in 2008 (the last year for which the World Bank has data), while Malawi’s per capita GDP was $288 in that same year (no, that is not a typo).   But what that means in terms of people’s real quality of life, their opportunities, etc. is not at all clear.  Clearly, Malawians are far less economically productive than Americans – but to address this issue, we have to understand why this is so.  Once we start to explore the different levels of economic productivity, we find that the causes of these differences are many, leading to other questions, such as why are so many Malawians engaged in subsistence farming, while Americans are engaged in the wage economy?  In short, per capita GDP is an interesting starting point for analysis, but it does not really capture the dynamics of poverty and human well-being in a manner that allows us to do anything about these situations.
To address this issue, other indicators and indexes (indices) that aggregate various indicators into a single value have emerged.  Perhaps the most famous is the Human Development Index, pioneered by UNDP’s Human Development Reports.  The HDI blends four indicators (life expectancy at birth, the adult literacy rate, the combined enrollment rate for primary, secondary, and tertiary schools, and a purchasing power parity adjusted measure of per capita GDP) to capture three different issues (health, education and income) which are then aggregated into a single score that runs from 0 (no human development) to 1 (presumably some sort of ideal human development).  This measure of well-being certainly moves beyond the purely economic, and probably does a better job of capturing the dynamics of poverty and well-being than any single measure, economic or otherwise, might.  But still, this is a limited index – there is no way to capture things like gender disparities that greatly impact people’s well-being and opportunities.
And now comes the MPI, the latest effort to get a development index right.  The MPI has quite a few more variables, and it has moved away from any reference to the economy in its measurement of poverty:
1. Health (each indicator weighted equally at 1/6)

  • Child Mortality: If any child has died in the family
  • Nutrition: If any adult or child in the family is malnourished

2. Education (each indicator weighted equally at 1/6 )

  • Years of Schooling If no household member has completed 5 years of schooling
  • Child Enrolment If any school-aged child is out of school in years 1 to 8

3. Standard of Living (each of the six indicators weighted equally at 1/18)

  • Electricity If household does not have electricity
  • Drinking water If does not meet MDG definitions, or is more than 30 mins walk
  • Sanitation If does not meet MDG definitions, or the toilet is shared
  • Flooring If the floor is dirt, sand, or dung
  • Cooking Fuel If they cook with wood, charcoal, or dung
  • Assets If do not own more than one of: radio, tv, telephone, bike, motorbike

There is a lot to like here – moving toward standard of living, and away from income, does a lot to make different situations comparable across countries and continents.  And shifting measures of health from life expectancy, which can be compromised by any number of issues in the life cycle, to child mortality and nutrition, which are highly correlated to health outcomes, is also a good idea.  But in the end, what will the MPI really add to our understanding of the dynamics of poverty and well-being that we could not have gleaned through the HDI – or through GDP, for that matter?  Put another way, I am worried that a lot of time and effort has gone into polishing a turd.
In my forthcoming book, I make an extended argument for doing away with these indicators altogether.  They are top-down efforts to organize and classify human experience in a manner that gives the illusion of actionable information, but none of the analytic purchase we actually need to do something in the world.  A close look at the MPI and its constituent indicators illustrates my point*.  Let’s examine Standard of Living – recall that I really like this category, and this reframing of this component of human well-being.  But what, exactly, do the indicators have to do with standard of living?  For example, why are radios, tvs, telephones, bikes, and motorbikes such critical assets in this index?  First, this presumes that these commodities are proxies for people’s standard of living, which is questionable at best.  Second, even if we accept that commodity ownership is an important part of the standard of living, why are we focused on these commodities?  For example, surely cattle ownership is far more important than any of these when evaluating people’s assets in East Africa.  And why does flooring matter so much?  Yes, it is possible that worms or other insects and animals could find their way into an earth-floored house, why not focus on roofing or wall materials (which are much more important in keeping out insects, and therefore dealing with issues like malaria)?
Why did the designers of this index choose these variables?  The answer, in part, lies in their explanation for their selection of variables “The ten indicators are almost the only set of indicators that could have been used to compare around 100 countries.” (p.13)  While you have to work with the data you have, availability is not a valid criteria for evaluating the usefulness of a particular measure.  In other words, if you are using an indicator variable as a proxy for a much larger process or issue, you have a responsibility to make sure that indicator actually says something meaningful about that process.  It is not at all clear to me that these variables have a meaningful link to the standard of living in many parts of the world.
To the credit of those who designed the MPI, they note that “one of the main lessons of this first exercise of estimating multidimensional poverty for developing countries is the urgent need to start collecting information on key internationally comparable indicators at the individual level” (p.13).  I’ve been part of an effort to rethink just how we identify and access this information, by building an information network that allows communities in the Global South to communicate with one another and with “experts” in the Global North – a bottom-up collection of data on the global state of human well-being.  Our estimates suggest that this approach would, in the relatively short term, become much more accurate and cost-effective for identifying and addressing the challenges that limit human well-being around the world than current top-down efforts, as embodied in large indices like the MPI.
If indices like the HDI, and now the MPI, tell us very little about the causes, and therefore the solutions, for the problems and challenges that the global poor deal with on a daily basis, they are not useful analytical tools – at best, they are a first step in a process of inquiry that identifies an interesting trend for future analysis.  So why are they still around?  At least in part because they are great PR vehicles – they make for interesting maps that ostensibly show how bad things are for so many people, and which justify continued development efforts to donors.  That is simply not good enough to justify the continued time and effort required to refine these indices.
*I’m not going to even get into the issue of weighting – basically, every individual variable listed above is weighted equally in this index.  So, infant child mortality rates have the same impact on the MPI score as using wood fuel, having a dirt floor, and television ownership.  Stop and think about that for a second.

Finally, a hopeful note on malaria?

Ah, malaria – I’m all too acquainted with this particular issue, having had it several times in the course of my fieldwork.  The first time is pretty miserable . . . but by about the fourth case, it is just a day feeling like you have the flu.  Of course, this presumes that you are relatively young and healthy – if not, malaria can be quite dangerous.
I am one of the “lucky” few who could not go near Mefloquine (brand name Larium to those of you who have taken anti-malarials) back when it was the “best choice” for preventing malaria.  I didn’t get malaria . . . but it did drive me toward a temporary bipolar situation and left me with residual vertigo that even now, 13 years later, I still feel at times.  So, after that experience, I simply stopped taking antimalarials entirely, and tried to deal with bugsprays and long pants as much as possible.  The relatively recent arrival of Malarone has made it possible for me to take effective antimalarials again, and when I am on short trips I do.  For the long term, though, you really shouldn’t be taking anti-malarials . . . they are really not good for you, and at some point they do become more problematic than malaria itself.
Given this situation, I have taken a rather acute interest in the efforts to battle malaria.  I’ve watched vaccines come and go.  I’ve seen the rage for bed nets as panacea consume everyone, even though they are quite compromised in their effectiveness by the fact that the anopheles mosquito likes to fly in the evening, when people are not yet in bed, and tends to fly very low to the ground, and thus below the level of many beds.  Hell, some crazy people have come up with a laser that can shoot mosquitos out of the air, thus preventing bites – the coolest, and most totally impractical solution for malaria I’ve ever seen (click here for a movie – really).  How, precisely, are people meant to power and maintain a LASER WHEN THEY HAVE NO ELECTRICITY?  And as I have watched all of these efforts, I have wished and hoped that someone could figure out a way to deal with this damn disease for the purely selfish reason that I am tired of getting it.
So, I was pretty excited to hear about a new development in this fight – an effort to genetically engineer mosquitos so that they cannot carry the parasite in the first place (LA Times, BBC, Tonic).  Malaria is obscenely difficult to kill, because it goes through a large number of stages in its life cycle, and each stage is vulnerable to treatment in different ways – thus, a treatment that works early in the infection cycle may not work on later stage infections – and worse, if there are parasites going through different stages at the same time (some have been gestating for longer than others), a treatment might only work on a fraction of the parasites in the bloodstream and liver at any given time.  But malaria has one weakness – it must have people to live in.  Without us, eventually there would be no malaria – we are the host, and mosquitos must collect it from our blood, before passing it to other people.  So, if the mosquito cannot act as the carrier, the parasite cannot move between hosts – and eventually the parasite dies out (the only real way to contract malaria is through mosquito bite*).  In other words, this just might work . . .
But there are serious caveats here.  First, we have to genetically modify mosquitos to do this.  Then we have to get the genetically modified versions to mate with unmodified versions, and for the genes that restrict malaria to be the ones that emerge in the offspring.  Since these genes would not convey any adaptive advantage to the mosquitos (the genetic modification actually causes them to die young, which strikes me as a significant genetic disadvantage), there is really no guarantee that this would happen – it could be that the modified mosquitos’ impact on the overall genetic pool is tiny – or huge.  I shudder a little at the proposed solution for this (From LA Times):

“connecting the gene to a piece of DNA that helps it spread by, for example, producing something that kills any mosquitoes that don’t contain the desired gene. Other research groups are working to develop such clever genetic tricks, but they are still years away from implementation.”

Once we start playing with a wider set of genes, I get worried.  As the article goes on to note, the effects of such modification are hard to predict in a single species, and since that species participates in a wider ecosystem, the impact on other species is equally hard to predict.  The last thing anyone wants are supermosquitos (a la superweeds and other superbugs that have resulted from previous genetic modification efforts).  So this is not a magic bullet, just a hopeful volley in what has been, and promises to be, a long battle.
*Funny side story: in 1999, I arrived back in Syracuse, NY just as a pretty bad case of malaria flared up.  Despite my protestations, several of my grad student colleagues bundled me off to the emergency room.  When the admissions person asked me what was wrong, I told her – rather simply – “I have malaria”.  The women stared at me for a second, and then said “should I be wearing a mask or something?”  I was pretty beaten up at that point, with an accordingly short temper, which explains my response: “Honey, unless you have a jar of anopheles mosquitos back there, I think we’re going to be alright.”