The big news around the ‘net today, as far as the life sciences are concerned, is the dramatic increase in the number of papers that are retracted, as documented (yet again) in this paper, and told in this NY Times piece by Carl Zimmer. (Check out some of the buzz here and here and here.) This story is primarily about the biomedical sciences, and so the question naturally arises, is the biomedical science community dysfunctional?
I’m going to say yes – but perhaps not in the way you think. As someone at a vulnerable career stage, whose future career path depends on the health of the biomedical community, I’ve experienced some of the problems in the community, and so I will offer you my opinion based on anecdotal evidence, for whatever it’s worth:
The biomedical community is dysfunctional because it has increasingly become a system based on a rigged lottery.
I’ll explain how the lottery works, and how I think it’s rigged. (For more on the lottery idea, see this report [PDF].)
Let’s start with the notable observation that the highest impact journals have the highest rate of retraction, particularly Science, Nature, Cell, and NEJM, which are really outliers compared to the other journals. These so-called top tier journals are a key part of the lottery, because publication in one of these journals is an overvalued currency when it comes to career advancement.
Getting into these journals itself is a lottery, as anyone except the journal editors will tell you. As just about any scientist who has submitted to them can attest, when you submit to these journals and get very good reviews, that only enters you into the lottery. Many papers are submitted to Nature and Science that a) receive good reviews, and b) make bold new claims, and only a small number of these can be published. So whether you get published in a top-tier journal also depends on other criteria – luck, the fame of your lab, whether you work is considered sexy (and not just groundbreaking) – i.e., it’s a rigged lottery.
A second aspect of the lottery is grant funding. Paylines are tight – too tight to distinguish between first-tier and second-tier proposals, and so somehow the first-tier proposals have to be put into funded and unfunded categories. And so again, other considerations come into play – your track record (which means the deck is stacked against younger investigators, despite the NIH’s efforts to counter this trend), and ‘feasibility’, i.e., whether you’ve basically completed the work with your preliminary data so that there is no possible reason the project could fail. Aside from leading to conservative, conventional science, this means that getting funded is a stochastic process. And so investigators submit more proposals, which means that the payline goes down even more, and you’ve got your classic positive feedback loop going.
A third part of the lottery is the competition for faculty positions. The number of positions hasn’t grown at the same rate as the number of postdocs, and so competition is tight. Which means that overburdened search committees, instead of picking candidates to interview based on a demonstrated ability to conceive of and successfully execute projects, pick candidates by counting – the total number of papers, and the total number of papers in Nature or Science. Sure, your recommendations and your research proposal matter, but less than you might think. (Weak recommendations and proposals will disqualify you, but very good ones won’t change the paper-counting game.) The best way to get yourself noticed is not to pick a new and intriguing problem and make progress – it’s to join the most famous lab you can and get on an already successful project that will enable you to crank out papers.
A similar story could be told about career advancement of junior faculty.
Basically, at every rung the career ladder you participate in the rigged lottery. The outcome is uncertain, and as the time to tenure and the grant writing burdens increase, the opportunity costs of this career path grow. Therefore people do whatever they can to rig the lottery in their favor. Rarely does this involve outright fraud or ethical violations, but it does lead to more overhyped claims, which are necessary in order to land yourself a slot in the overhyped so-called first-tier journals. And hence, the increase in retractions is a symptom of the disease.
This is a self-organizing process, and not a story about the moral decay of scientists. Sure, as the community grows, you’ll get more bad apples, but the vast majority of biomedical scientists are good, highly motivated people desperately fighting to sustain a career in a field they love, one filled with stimulating colleagues and exciting questions. Paradoxically, this very struggle is is taking its toll on the community, and stimulating colleagues and exciting questions get replaced by hyper-competitive players and safe lines of investigation.
It wasn’t always like this. Just about anyone I know who got their first faculty job in the 70’s or 80’s will say that it wasn’t so hard to get into Nature or Science until the 90’s, and that you didn’t need to worry about funding your lab with 3+ grants. It was demonstrably easier for a postdoc to get a job at a research university in the 80’s than it is now.
I don’t know what the solution is. As long as biology is hot, and as long as there are people who are passionate about making science their career, you’ll have people who really, really, really want to be biologists
The Finch and Pea 
One of the best explanations of this problem that I have read. My husband and I are both biologists. With 20 papers (but no postdoc ones in the Big 3 journals) and what everyone describes as a great research proposal (but he’s not in a famous lab), he hasn’t been able to get a faculty position. He has more papers than several, older, tenured faculty members in his department. It hurts.
What it takes to get a job these days is not only different, but it appears to be changing rapidly, because, in my limited experience anyway, the postdoc paper count of people hired ~10 years ago seems to be less than what I’ve seen in the CVs of job applicants who get interviews.