the aura of peer review to everyone else

September 7, 2009

I came across an interesting article in Nature News about the controversy surrounding a paper published in PNAS (Proceedings in the National Academy of Sciences, a big deal journal) about genetically modified (GM) corn and its toxicity in streams. The article in Nature News discusses the controversy this PNAS paper ignited among other scientists in the GM field (those who tend to support its use and development). The “attackers” wrote an open letter rebutting the methods used in the original study, and the study’s author said she received a large amount of inflammatory (negative) feedback, more than the usual objective disagreement.

While I always think controversy and rebuttal letters are valuable to the scientific project of distilling true principles out of our otherwise vague world, I was interested to hear about the more emotional aspect of some of this response against the original author’s methods (and thus validity of the paper). What I see in these “pro-GM” scientists are people fighting for their field. They know that all it takes is one or two papers elucidating the negative effects of GM-crops to sway public support away from them, and away from their research (cf. monarch butterflies, Europe)

Scientific research mainly benefits but also suffers from the fact that when something is published in a respectable, peer-reviewed journal, people believe it. Scientists read a lot of papers within their field (or subfield). Each paper is usually not a definitive statement on something, it’s a bit of evidence for or against something. Just because one paper is published claiming X does not mean a scientist believes X. It takes a number of papers claiming X in a variety of different angles through a variety of different methods. For sure, a paper, especially one in a well known journal, lends a good deal of credence to claim X but by no means proves it.

And yet, the public, which includes scientists outside of their own subfields, gives so much more credence to individual papers. All it takes is one paper to influence important policy decisions that affect that field. The public does not read Methods sections, where most of the internal debate often stems.

I have been guilty myself, many times probably on this blog, of drawing conclusions from science-news headlines without thoroughly scouring the papers with the skepticism perhaps they deserve. After all, it’s impossibly to be an expert in every field. What’s important for us all to remember is that each paper is just a small bit (however convincing) of evidence for or against something. Very few papers can single-handedly take down or erect a scientific belief.

So the next time you find yourself citing a paper (from reading it directly or indirectly through the news) as evidence for something, remember to take it with a grain of salt. The world could do with a bit more scientific skepticism.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: