Hardly anybody [I know] reads…

Bill Easterly, one of the most accomplished and iconoclastic development economists and commentators, may have overreached a bit. He has picked a bone with the Global Forum for Health over their failure to win publications and citations in journals worthy of consideration at tenure review. Isn’t that a strange criterion for judging the success of the Global Forum for Health? They are insufficiently concerned with winning faculty jobs at US universities?

The evaluation has lots of other things to say about them, both positive (they have a lot of meetings!) and negative (“absence of an agreed results framework”).

So the World Bank seems to be following a disturbing trend. It is financing economic research publications that hardly anybody reads, and financing health awareness efforts that hardly anybody is aware of.

This also creates a new challenge for aid watchers – how can we hold accountable an aid agency we don’t know exists? What other dark matter in the aid world is awaiting discovery?

Global Forum for Health is a self-described advocacy organization. If anything, Bill Easterly should be taking the development economics profession to task for its solipsistic criteria for success. In manufacturing industries, the gold standard of achievement is not publications, it is patents. Unfortunately, the economics profession has no equivalent to the patent.

My problem with Easterly’s criticism is the standard for judgment: “…publications that hardly anybody reads.” An astute reader will insert brackets, so it reads “publications that hardly anybody [I know] reads.” Tenure review boards matter primarily to people whose jobs hinge on tenure decisions, and in places that read the same publications that Easterly does.

Academics overseas may have a very different idea of where to publish their research, and what research is credible, and what research is relevant to their careers. Tbe Economist recently stirred up a hornet’s nest with its report on the views of top economists on the profession, who accuse the discipline collectively of dogmatism and sloppy analysis (Barro, Buiter, DeLong, Eichengreen, Lucas, and Krugman). To the extent that publication in the top journals is both extremely time-consuming and something of a beauty contest, perhaps it isn’t reasonable to discount any and all research being done outside the top American journals.

IRB is incompatible with open access to data

Mathematician Arvind Narayanan at his blog 33 Bits writes a compelling post on the failure of efforts to protect the identities of individuals in public domain, scientific data sets. De-anonymization of data is not new in the field of electronic data privacy. This is the first discussion that I have read of the specific conflict between IRB approval and scientific peer review.

This thesis of this blog that the amount of entropy required to de-anonymize an individual — 33 bits — is low enough that it doesn’t offer meaningful protection in most circumstances. Obviously, the argument applies even more strongly to the anonymity of a well-defined group of people.

Let’s be clear: the paper is from 1994; who slept with whom in high school is not a huge deal a decade and a half later. However, the problem is systemic, and IRBs (Institutional Review Boards) keep blithely approving releases of data with such nominal de-identification applied. The re-identification of the institutional affiliation of an entire population of a study is of more concern from the privacy perspective than the de-anonymization of individual identities: it needs to be done only once, and affects hundreds or thousands of individuals.

Recently, a group of researchers from the Berkman Center released a dataset of Facebook profile information from an entire cohort (the class of 2009) of college students from “an anonymous, northeastern American university.” It was promptly de-anonymized by Michael Zimmer, who revealed that it was Harvard College:….