Open The Echo Chamber

So, a while back I decided to talk about how I negotiate peer review, semi-liveblogging my response to a revise and resubmit request from a pretty big development journal (see part 1, part 2 and part 3).  Well, I now have a response to my resubmission . . .

No.  To quote: “after much deliberation, the editors have reached a rather difficult decision. [The editors] feel that they cannot accept your revised paper.”

Yep, I have gone from revise and resubmit to outright reject.  This is . . . unusual, to be honest. More unusual, however, is the rationale for the rejection.  To quote from the decision:

What makes this difficult is that [the editors] recognize that you have in fact taken account of what the referees said, and have tried to accommodate their comments, but the editors feel that what has emerged from the revision process is not an appropriate paper for Development and Change.

Translation: you did what we asked, and addressed the referee comments, but in doing so you ended up with a paper that we think belongs at another journal.  Well, fair enough, this happens.  But why it is not appropriate is a little odd:

While they still believe that there is an interesting idea at the core of your paper, they don’t feel that the revisions have solved the initial problems, and they are not convinced that further rounds of revision would be any more successful. The intended contribution of the paper appears to be theoretical, but the paper hasn’t managed to work out that contribution in a way that will be accessible / comprehensible to our readers.

Soooo . . . I have an interesting paper, but the editors more or less think their readership can’t deal with the complexity of the argument.  [Note: I am disregarding the assessment that my revisons have not solved the initial problems, since they already have said that I took account of the referees’ issues – this is a contradiction I am just going to leave aside. That, and they did not show me any reviewer comments, so I have no idea what I did not resolve]  One of my colleagues has called this the oddest rejection he has ever seen.

Now, I want to be clear – the folks at the journal with whom I interacted throughout this process were very responsive and polite, and were kind even in their rejection (they were quite apologetic, actually).  I would submit to this journal again, though I admit to wondering exactly what aspect of my work might fit here, as I am confused by what they believe the capacity of their readers to be.

This, folks, is the nature of peer review – sometimes, you just have no idea what happened.  I am not privy to the internal conversations of the editorial board, and will not pretend to know exactly what happened here.  What makes this hard is that I did not receive any substantive comments on this second round of review, so I have no guidance at all on edits.  I am rereading the paper, adding a citation I had missed earlier, and making minor tweaks to the argument (the article I missed before actually strengthens the case for what I am doing in the manuscript).  I’ve sent it off to a trusted senior colleague to have a look, and to see where he thinks it might go next.  I will probably sound out the next editor in advance, just to make sure that s/he thinks the paper is appropriate before starting a long review process again . . .

Two years and counting, folks, since my initial submission.

Any editors out there interested?  Anybody?

Earlier this week, Linda Raftree pointed me to this article, which references another article that calls blogging without tenure “an extreme sport” because of the risks involved.  It is a little hard for me to comment on this specifically, as I did not start blogging until after I had tenure – not because I was afraid of blogging, but because it never occurred to me to blog before then (basically, my agent and my publisher pushed me to blog to promote my book).  I did plenty of “public sphere” writing, such as op-eds in The State (Columbia, SC).  Hell, right before I went up for tenure I published one titled “Governor’s energy report has no clothes.”  I walked into my chair’s office the day it was published, and he shook his head and said “not exactly keeping your head down, are you?”  The op-ed had no impact on my tenure at all.  In most cases, neither will blogging.

I think most academics are far too timid when it comes to public expression.  They fear reprisals against their careers, but rarely seem to be able to articulate where such reprisals might come from or how they might actually create harm.  I am sure there are indeed cases of highly dysfunctional situations where individual’s careers might be harmed by the public expression of their views on a given subject within their expertise, but such situations are volatile for many reasons and blogging is unlikely to ever be the cause of career problems.  In fact, I am convinced that there is far more upside to blogging than there might ever be a downside.  On the upside:

1) As I recently noted, my blog and twitter accounts appear to have done a great deal to spread my work around, and to get that work used (at least by other writers).  Find me a department that will complain about your rapidly rising citation counts.

2) You will develop a whole new community of colleagues, and they will bring new ideas and perspectives that you simply cannot get talking to people in your department, or even in your discipline.  These ideas and perspectives can be challenging, but if you can harness them, they can carry your thinking to new and innovative places.

3) When you develop a public persona, you can build a degree of freedom from problematic situations in your home institution.  You can cultivate a community in which there might be several people interested in giving you a job.  Further, universities love publicly-visible faculty, because they are easy to point to when someone asks what the faculty contribute to the larger society (and yes, this does get asked often).

4) You practice speaking in multiple registers: we all write academic articles, and if you are on the tenure track I hope you’ve figured that process out.  But do you know how to engage the person on the street?  Taxpayers fund a lot of research, and explaining to them why they should be happy they are funding yours is a worthwhile skill.  You can’t do that through a journal article, or in the language of your discipline.

On the downside:

1) Bill Easterly said it best: the blog is a hungry mouth.  It can be hard to keep up with posting, especially when you have a bunch of other stuff going on during the semester.

2) You will be exposed to griefers – the internet is a harsh place.  People will say nasty things about you and your ideas.  If you are fragile, do not try this at home.

Anyway, these are just my quick thoughts on blogging and academia, and I am sure my thoughts are incomplete and others will have something to add.  Indeed, you should check out Marc Bellemare’s recent post on things he has learned as an untenured blogger.  Speaking for myself, though, I have not regretted blogging at all, and aside from sometimes being exhausted after finishing a post, I have yet to see a serious drawback from doing so – but the benefits have been remarkable.

Following on my previous post, another thought that springs from personal experience and its convergence with someone’s research.  If you look at my Google Scholar profile, you will note that in 2011 my citation counts exploded (by social science standards, mind you – in the qualitative social sciences an article with 50 citations or more is pretty huge).  Now, part of this is probably a product of my academic maturation – a number of articles now getting attention have been around for 3-4 years, which is about how long it takes for things to work their way into the literature.  However, I’ve also seen a surge in a few older pieces that had previously plateaued in terms of citations.  This can’t be attributed to a new surge in interest in a particular topic, as these articles cross a range of development issues.  However, they all seem to be surging since I got on Twitter and joined the blogosphere.  Bascially, it seems a new circle (circles?) of interested folks now has access to my work and ideas, and the result is that my work is finding its way into a new set of venues/disciplines that it might otherwise not have reached.  It is hard to be sure about this, as my 18 months on the blog and 1 year on twitter are just at the edges of how long it takes to get an article written, submitted, accepted and published, but clearly something is happening here . . .

This seems to be borne out by some work done by Gunther Eysenbach examining the relationship between tweets (references to a paper on twitter) and the level of citation that paper eventually enjoyed.  Eysenbach found that “highly tweeted” papers tended to become highly cited papers, though the study was quite preliminary (h/t to Martin Fenner at Gobbledygook.  You can find links to Eysenbach’s paper and Martin’s thoughts on it here).  This makes sense to me – but it requires a bit more study.  I like what Fenner and his colleagues are trying to do now, capturing the type of reference made in the tweet (supporting/agreeing, discussing, disagreeing, etc.).  Frankly, references in general should be subject to such scrutiny.  As one of my colleagues once said, if citation counts are all that matter we should write the worst paper ever on a subject, jam it into some journal that did not know better, publicize it and wait for the piles of angry negative citations to pile in . . . only we just have to count the citations, not admit that we are being cited because people hate us!

The altmetrics movement is starting to take off in academia (see, for example, this very cool discussion) I have not yet seen any discussion, though, of what social media might do to journal prestige.  While there will always be flagship journals to which disciplines full of tenure-track faculty will bow, once tenure is achieved this sort of homage becomes less important.  Given what I am seeing with regard to my citations right now, my desire to have my work have impact beyond my discipline and the academy, and my concerns for the policing effect of peer review (which emerges most acutely in flagship journals – see my posts here and here), why should I struggle to get my work into a flagship journal when I can get a quick turnaround and publication in a smaller journal, still have the stamp of peer review on the piece, and then promote it via social media to a crowd more than willing to have a look?  If I (or anyone else) can drive citations through mild self-promotion via social media, does the journal it is published in really matter that much?  I wonder what sort of effect this might have on the structure of publishing now – will flagship journals have to become more nimble and responsive, or will they soldier on without changes?  Will smaller journals sense this opportunity and move into this gap?  Will my colleagues embrace the rising influence of social media on academic practice?

Does any of this matter?  Not really.  If the emerging studies on social media and citation are correct, and my trends are sustainable, then one day I will be one of the “important” folks with a lot of citations . . . and I will be training my students to engage in conventional and non-conventional ways.  I will not be the only one.  Those of us who engage with social media, and train our students to do so, will eventually win this race.  Change is coming to academia, but the nature and importance of that change remain up in the air . . .

Been a while . . . been busy.  And yes, I stole that post title from Ralph Nader . . .

As those who follow this blog know, one of my big concerns is with the walls that academia is building around itself through practices like the current incarnation of peer review in specialist journals. It’s not that I have a problem with peer review at all – I think it is an important tool through which we improve and vet academic work. Anything that survives peer review is by and large more reliable than an unvetted website (like this one, for example).

But the practice of peer review in contemporary academia has turned really problematic. Most respected journals are more expensive than ever, making access to them the near-sole province of academics with access to libraries willing to purchase such journals. The pressure to publish increases all the time, both in rising demands on individual researchers (my requirements for tenure were much tougher than most requirements from a generation before) and in terms of an ever-expanding academic community. The proliferation of published work that has emerged from these two trends has not really improved the quality of information or the pace of advances – there is still a lot of good work out there, but it is harder and harder to find in an ever-growing pile of average and even not-so-good work. And I have found peer review to often function as a means of policing new ideas, slowing the flow of innovative ideas into academia not because the ideas are unsupported, but because these ideas and findings run contrary to previously-accepted ideas upon which many reviewers might have done their work. This byzantine politics of peer review is not well-understood by those outside the academic tent, and does little to improve our public image.

So I am wondering where the tipping point is that might bring about something new. Social media is nice, but it is not peer-reviewed. I tend to think about it as advertizing that points me to useful content, but not as content itself (I have a post on this coming next). I still want peer-review, or something like it. So, a modest proposal: senior colleagues of mine in Geography – yes, those of you who are full professors at the top of the profession, who have nothing to lose from a change in the status quo at this point – who will get together and identify a couple of open-access, very low-cost journals and more or less pronounce them valid (probably in part by blessing them with a few of your own papers to start). Don’t pick the ones that want to charge $1500 in publishing fees – those are absurd. But pick something different . . .

This, I think, is all it would take to start a real movement in my discipline – admittedly, a small discipline, so maybe easier to move. Just making our publications open to all is a tiny first step, but an important one – once a wider community has access to our ideas, they can respond and prompt us for new ones. Collaborations can emerge that should have emerged long ago. Colleagues (and research subjects) in the Global South will be able to read what is written about their environments, economies and homes, improving our responsiveness to those with whom, and hopefully for whom, we work. First steps can be catalytic . . .

Remember the article on livelihoods I referenced in my last post?  The one that has been through 13-odd reviewers over the past 6 months?  Well, it came back this morning with that least satisfying of responses: revise and resubmit.  The reviewers disagreed on the strengths and weaknesses of the paper, but all of them wanted at least some small changes.

Now, this is hardly the first time I’ve had to deal with this.  Happens all the time.  But this review, like a number of others I have encountered in my career, highlights for me the need for strong editors at journals.  The email from the journal read, in part:

By the time of [the editorial board] meeting, we had received the reports of the external referees. You will find copies of their comments in the attached document. The editors feel that the referees have raised some important points and highlighted some shortcomings in the paper, which would need to be addressed through a round of revisions before we could proceed further. Please note that the editors are forwarding these comments as you might find them helpful; individual comments do not necessarily represent the views of the editors, or a consensus regarding the direction of any possible revision. [my emphasis].

Well, that’s just super.  It amounts to “here are some comments, which might or might not be relevant. Good luck!”  This gives the author nothing to work from, especially when some of the comments are contradictory.

So, a quick lesson for all of you budding academics out there.  Letters like this require a response asking for clarification.  I hammered out an email requesting a conversation with the editor.  Yes, that’s right, you can talk to the editors!  My first paragraph:

Thank you for forwarding the reviewer comments on my paper (Manuscript ID DECH-11-094 entitled “Livelihoods as Governmentality: Reframing the Logic of Livelihoods for Development”).  I appreciate the time and effort the reviewers put into the paper, and the consideration the editorial board has given my submission.  I would very much like to revise and resubmit this paper, but I need a bit of guidance if I am to do so in a productive manner.  Ideally, I would like to talk with the editor in charge of this submission to resolve some of my ideas for revision, and some larger concerns before moving forward.  I have summarized them, in a general sense, below:

(Note that this is polite – always be polite! It’s amazing how many people fail to do this.)

I then wrote 1500 words on what I proposed to do – with great specificity.  I hope to speak directly to the editor myself soon.  That way, I will know how s/he plans to treat the manuscript when they receive revisions, and how s/he views my suggestions – this is huge, as it will tell me what I can and cannot skip in the comments.

Oh, and one of the reviewers was a disaster – did not get the paper, wanted it to be the paper they would have written, etc.  I headed this person off with the following paragraph:

The comments of reviewer 1 are external critique – that is, they demand that the paper be something other than what it is, do not acknowledge the stated goals of the paper, or whether or not the paper achieved those goals.  The reviewer clearly did not engage with the core theory of the paper (for example in arguing the lit review was too confusing and should be cut down), instead demanding it be reframed as a gender analysis (which it is not) with relevance to policy (not the goal of this piece, which is more about foundational theory upon which policy statements might be constructed – policy relevance is also not a criteria for Development and Change).  The reviewer appears to be upset because the paper is not the gender and development paper s/he would have written (incidentally, I already wrote a paper closer to what they wanted and published it in World Development about three years ago – this paper is in part an effort to move past the limitations of that analysis), and instead of looking at the literature I am drawing on, they simply demanded I reframe and cite all the things they think are important.  Generally speaking, I try to take responsibility for moments when the reader becomes confused, treating this as a symptom of unclear writing or thinking on my part – I hope this is clear from my responses to the other reviewers’ comments.  However, the complete disengagement of this review with the paper as it was written, coupled with reviewer 3’s assessment of the paper as well-written, suggests to me that this reviewer’s concerns cannot be addressed without writing a completely different paper – and that paper would not make much of a contribution anymore (something they acknowledge). Suggested edits: none

Yep, you can make the case for the removal of reviewers from a revise and resubmit – especially if they will never approve the paper for what it is.

I’ll keep you posted on progress . . .