Scientists and Dan Mangan both Want To Be Pleasantly Surprised, Not Expectedly Let Down

Dan Mangan is no stranger to using scientific references and metaphors to communicate larger meanings about life. His lovely track About as helpful as you can be without being any help at all uses a quote from science illustrator Charlie Harper to beautiful effect (and was a previous Song of the Week). The song that follows it directly on the album (2011’s Oh Fortune) is called How Darwinian and pivots on the lines I should know better by now/there’s only so much to go around. 

The A-side of his new 7″ Radicals, released today, works in the other direction. It’s an observation about life that actually has special significance for science. The song is heavy on guitars and crunchier and louder than usual for the normally melodic and almost symphonic singer-songwriter. After a few quiet verses, the song’s title begins as a quiet refrain and then swells into a noisy passionate plea: We Want To Be Pleasantly Surprised, Not Expectedly Let Down.

Yes, that is correct, expectedly let down. I really like how the phrase captures the ongoing tension in science between wanting and hoping for significant results  (that’s what most publishable and what brings recognition for doing new and exciting work) and knowing deep down that the most rigorous approach is to always assume the null result: to be expectedly let down.

Waiting to be pleasantly surprised is part of what leads to publication bias: only publishing the surprising and positive results, while ignoring studies that show no effect (or even negative effects) for interventions or drug treatments. It’s an especially troubling issue once those positive results are then collected into the meta-analyses. And yet a January 2012 study in BMJ found that only 32% of the meta-analyses they looked at discussed or investigated possible issues with publication bias.

It also contributes to the ongoing problem discouraging the publication of studies that replicate and check previous research. There is rarely anything pleasantly surprising about replication studies, despite how important they are. Ed Yong wrote an excellent examination of the replication problem in psychology for Nature that included the following:

One reason for the excess in positive results for psychology is an emphasis on “slightly freak-show-ish” results, says Chris Chambers, an experimental psychologist at Cardiff University, UK. “High-impact journals often regard psychology as a sort of parlour-trick area,” he says. Results need to be exciting, eye-catching, even implausible. Simmons says that the blame lies partly in the review process. “When we review papers, we’re often making authors prove that their findings are novel or interesting,” he says. “We’re not often making them prove that their findings are true.”

As Yong goes on to explain, there are many factors that contribute to these issues including problems with scientific publishing and funding models. The simplest explanation, though, is that scientists are people, and like Mangan notes we all want to be pleasantly surprised, not expectedly let down. It’s not a bad thing to keep in mind when reading new and exciting results or working to dig through and publish new research.

If you want to hear from more you can also check out What Happens Next, a new documentary about Dan Mangan, getting its Canadian national debut on CBC television October 20 and, for music fans everywhere, available streamed that afternoon from the CBCMusic site.

_______________________

Ahmed, I., Sutton, A. J., & Riley, R.D. (2012). Assessment of publication bias, selection bias, and unavailable data in meta-analyses using individual participant data: A database survey. BMJ, 344.

Author: mcshanahan

Science education researcher and writer

2 thoughts on “Scientists and Dan Mangan both Want To Be Pleasantly Surprised, Not Expectedly Let Down”

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s