Sloppiness vs Reproducibility

I’m not a big fan of reproducibility projects. Shoddy papers shouldn’t be tolerated, but the truth is that sometimes rigorously done research isn’t reproducible — and when that happens, science gets interesting. It should go without saying that a peer-reviewed paper isn’t a guarantee of truth. If done properly, a paper is a record of a rigorous attempt to discover something about the world, no more, no less. What we believe about nature should reflect the accumulated evidence of many researchers and many papers, and that means the scientific literature should reflect our latest tentative, bleeding-edge thinking, even at the risk of being wrong. It’s counterproductive to hold up publication until some other lab reproduces your result, or to retract papers that don’t hold up, unless they had clear methodological flaws or artifacts that should have been caught in review.

Two recent articles capture what I think is the right attitude on reproducibility. First, as David Allison and his colleagues write, as a community of researchers, editors, and reviewers, we’re not doing as well as we should be when it comes to meeting high standards for best statistical and other methodological practices:

 In the course of assembling weekly lists of articles in our field, we began noticing more peer-reviewed articles containing what we call substantial or invalidating errors. These involve factual mistakes or veer substantially from clearly accepted procedures in ways that, if corrected, might alter a paper’s conclusions.

There is no excuse for this kind of sloppiness.

On the other hand, here is Columbia’s Stuart Firestein:

The failure to replicate a part or even the whole of an experiment is not sufficient for indictment of the initial inquiry or its researchers. Failure is part of science. Without failures there would be no great discoveries.

So yes, let’s clean up science by rooting out obvious “invalidating practices” that all too often plague papers in journals at all tiers. But let’s not be naive about how science works, and what the scientific literature is supposed to be. To paraphrase what  I wrote recently, if some of our studies don’t turn out to be wrong, than we’re not pushing hard enough at the boundaries of our knowledge.

GMO wild organisms: As if GMO crops weren’t controversial enough…

The big biotech controversy of last year was over the ethics of using CRISPR to edit human embryos – something which a team of Chinese scientists did last April. The possibility of designer babies led to a major scientific summit meeting, hosted by the National Academy of Sciences, during which the attendees concluded that “It would be irresponsible to proceed with any clinical use of germline editing” until safety concerns are allayed and society comes closer to an ethical consensus.

While the world was fretting about edited embryos, scientists introduced an even more ethically fraught biotechnology: gene drives, a tool to genetically modify organisms in the wild. Gene drives have the potential to do a lot of good, by controlling disease vectors like malaria-bearing mosquitos. But if you thought GMO crops were controversial, just wait to see how people react to GMO wild organisms.

I cover the new CRISPR-based gene drive technologies in my latest Pacific Standard column. Here’s the tl;dr version: Gene drives can do a lot of good, but because they are simple to make, and because their consequences aren’t confined by political borders, we’re going to have a hell of a time ensuring they’re used responsibly.

Not My Skill Set

There was a time when, to be a biologist, you either had to be able to draw well (or be able to may someone to do your drawing for you). Toilets also did not flush. It was a rough time. You can revisit some of that magic at The British Library’s Flickr page where there are literally oodles of images from old books provided free from copyright restrictions, because the books are old.


I don’t know what this critter is*, but I am glad that I am a lot bigger than it is. I think I’m bigger than it…

*It literally comes from a tome entitled The British Miscellany: or, coloured figures of new, rare, or little known animal subjects, etc. vol. I., vol. II

Actually it’s about scientific accuracy in zoo signage

Photo by Josh Witten (CC BY-NC-SA)

Science for the People: Where do Camels Belong?

sftp51dorvo2v9l-_sl250_This week on Science for the People, we’re discussing ecosystems, biodiversity, and whether or not “invasive” outside species are really as bad as they’re made out to be. We’ll spend the hour speaking to Dr. Ken Thompson, lecturer in the Department of Animal and Plant Sciences at the University of Sheffield, about his book Where Do Camels Belong? Why Invasive Species Aren’t All Bad.

Science for the People is now part of the Skepchick Network.

Don’t forget to support the Science for the People on Patreon to keep the sciencey goodness flowing toward your ear holes.

*Josh provides research help to Science for the People and is, therefore, completely biased.