We like to think that science is objective, that its approach to knowledge derives from the lack of bias. Unfortunately, nothing could be further from the truth.

Despite the best of intentions, scientists confront and fight with bias all the time, both within themselves and  when interpreting the work of others.  Scientists have agendas just like anyone else, whether it’s a pet hypothesis or a cultural ideology, and it can be manifested in the way that the data are collected and interpreted.  So, how can the interested non-scientist find the truth amid the bias in the science they’re reading and hearing about?

First of all, one should appreciate the importance of peer review.  Before a scientific finding or idea is published in a scientific journal or funded by a federal agency like this country’s NIH or NSF, it is evaluated by other scientists who are experts in the same field (i.e., peers).  Reaching a consensus among 3 or more peers about the suitability of a manuscript for publication, or a grant proposal for funding,  generally assures that the influence of individual bias is minimized.  Of course, sometimes even reviewers can be biased, typically manifested as a rigid skepticism to new ideas that might supplant the status quo, no matter how good the science.

The larger problem, though, is that lots of science is presented in other ways, that don’t involve the rigorous kind of peer review required for journal publication and federal grant funding:  in books, magazine articles, interviews for newspaper articles and TV/radio broadcast, on websites, and in blogs like this (caveat emptor, dear readers).  In this case, the responsibility for review falls on publishers, editors, collaborators, and oneself, not necessarily a great recipe for objectivity.  Publishers and editors are usually no more expert on the topic than the writer in question, and often have a greater motive to publish an interesting story, whether or not from a purely objective perspective.  And, the recent brouhaha over the self-plagiarism and made-up quotes by science writer and blogger Jonah Lehrer demonstrate how important it is to have a skeptical readership.  As I said, caveat emptor.

A variation on this theme is when scientists pursue pre-publication publicity on their work.  In the past, this would often be a press release preceding or at best coinciding with the publication of an accepted paper.  Nowadays, scientists use their own web pages and blogs to promote their work, sometimes before the process of peer review has even been completed (e.g., see discussion of Richard Muller emphasizing that his work on climate change has been peer reviewed, even if it hasn’t been accepted for publication).  Shouldn’t we be waiting for the paper in question to actually be accepted for publication before we’re asked to assimilate it?

In addition, the internet has changed the nature of the game.  Everyone’s a critic, and can find some public venue for registering their mega kudos or boos.  Though I’ve noted above that having a large skeptical readership is often a good thing, unfortunately, the opportunity for unfettered criticism just encourages stridency, driven mostly by personal opinion.  We’ve lost any sense of civility, with commenters mostly yelling at, and past, one another.  Those outside of science don’t realize that the formal peer review process, although cloaked under the protection of anonymity to enable frank evaluations, is remarkably civil.  Of course, I’ve privately cursed many reviews that resulted in my manuscript or proposal being rejected (OK, publicly, too, sometimes), but the anger was directed at the decision itself, not the tone of the review, or any perceived bias, for that matter.

This brings me to the nature-nurture debate.  There are lots of examples of issues in science that have become controversial and complicated because of the biases of the protagonists:  climate change, evolution, genetically modified organisms, vaccines and autism, fat in the diet — you name it.  But none that I’m aware of have been at the center of public discourse as long or persistently as the argument over whether one’s genes or one’s environment have had a greater influence over one’s existence.  A prime example of this is measurement of IQ.

The science of IQ has been controversial almost from the first introduction of formal testing by Binet in the early 1900’s, and particularly regarding nature vs. nurture.  After all, it helped to usher in the early practice of eugenics, by providing quantitative evidence to confirm differences between ethnic groups and socioeconomic classes that those in the ruling classes presumed were genetically based.

Moreover, scientists have often taken sides in this debate, as represented in Richard Herrnstein and Charles Murray’s 1994 book, The Bell Curve, and Stephen J. Gould’s 1996 update to The Mismeasure of Man.  Appropriately, these particular scientists have also been roundly criticized for ideological bias in their treatment of scientific data.

Unfortunately, ideology in the name of science persists in the study of IQ and its determinants.  A 2001 book by Richard Lynn and Tatu Vanhanen (“IQ and the Wealth of Nations”) demonstrated that IQ varies with national wealth, specifically per capita GDP.  They interpreted this correlation to reflect racial, genetic differences between nations, and concluded that the correlation actually represented an immutable causal link between intelligence and national wealth, which could not be ameliorated.  The book has received strongly worded praise and criticism since its publication, but it’s remarkable that its essential point was based on a biased presumption about race and IQ, coupled with over-interpreting a mere correlation.

Recently, Ron Unz brought a fresh, more balanced perspective to the Lynn-Vanhanen data.   As highlighted by Joshua Rothman in the Boston Globe,  the Unz article is particularly interesting because it arrives at an unexpected conclusion.  After briefly chiding one of Lynn and Vanhanen’s critics for his ideological attacks on those who view national wealth as fixed and tied to the presumably genetically determined intelligence of a nation’s population, Unz eventually aligns himself with a similar view that IQ, with some exceptions, varies largely with cultural, not hereditary differences.

Unz pursued a re-analysis of  the Lynn-Vanhanen data, and found that the presumption of genetic determinism is unwarranted.  Focusing just on European countries and data from just the past half-century, avoiding the most objectionable data from sub-Saharan Africa, he compared common ethnic populations distributed to different countries, e.g., residents of Austria and Croatia, East and West Germany before unification, native Irish and Irish-American immigrants, etc.  Despite the presumed (and in some cases documented) genetic uniformity between these pairs, large differences in IQ were observed.  Unz interpreted these patterns to reflect instead a significant cultural, socioeconomic impact on IQ, and went on to note that IQ scores were particularly higher among urban populations.

However, in a separate article, he observes that IQ scores for East Asian populations are notably invariant as a function of relative wealth (again, per capita GDP), unlike European populations, suggesting perhaps that genetics may play a determinative role for some populations, after all.

I found Unz’s approach to this controversy to be balanced and measured, admirably so.  However, this also is just another phase in the interminable back and forth over nature v. nurture.  From a biological standpoint, there can only be nature AND nurture.  Environments alter gene expression, and individuals of a particular genetic disposition alter their environments, or exhibit what’s called phenotypic plasticity in using alternative strategies to side-step genetic predisposition to solve new environmental challenges.

So, genes and environments are hopelessly intertwined, at all stages of development.  And, in this particular case, I would argue that populations hardly represent homogeneous gene pools, nor do cultural groups associated with populations represent homogeneous environments in any straightforward way.  I see no reason to presume that an urban population is different from a rural population based solely on cultural, environmental variables, or that Austrians and Croats are genetically similar as populations, one of the several pairings that Unz used in his analysis.

Besides, at this point, what does it matter?  What does IQ really tell us about people that is interesting and useful?  Because Unz sees evidence that IQ is higher among urban than rural populations, perhaps the “strongly abstract and analytical thinking” required by the IQ test is more critical to city dwellers.

Fine, but what are we going to do with that?  Should we be advising people to move to the country because they earn a low IQ score?

Facetiousness aside, I worry that, at this level of analysis, the only possible outcome is a gross overgeneralization about a trend that is impossible to apply to real world problems at a much more local, even individual level.

As Unz points out, perhaps the most interesting outcome of this story is the relative plasticity of data interpretation.  Careful data collection is not enough.  It takes an objective interpretation of the data to identify what the data represent, and how they fit into both current knowledge and hypothesized new models attempting to extend that knowledge, with an open mind about what things mean.

Certainly, Unz presents an interesting example of how scientists can work with the same data but come to very different conclusions. But should we trust in HIS conclusion?  After all, The American Conservative is not a peer-reviewed journal, Unz is not a social scientist (physics was his original sandbox), and he is not merely a writer for the magazine — he’s the PUBLISHER.  For me, the tone of his article is persuasive, and the observations are thought-provoking.  But I’d also like to see how his key points converge with formally reviewed and published studies.  And, as I’ve noted, my own training in biology makes me skeptical of ANY generalizations about a solitary, exclusive influence for genes or environment on human existence.  

Parsing the truths and half-truths in public science can be a big challenge for the public and for science journalists trying to identify a reliable take-home message in a new scientific finding.  Frankly, it’s a challenge to scientists as well.  

Complicating matters on this particular issue — nature vs. nurture — is that it has attracted so many ideological partisans, who, like many current U.S. politicians, refuse to look beyond narrow party-line perspectives that have little to do with the science, but everything to do with a political agenda.  

As Unz puts it, “Factual reality runs second to ideological tribalism.”  I suppose I should count myself as lucky that, in my own field of olfactory neurobiology, not much is ever likely to reach such fever pitch.  

So, how can the interested non-scientist find the truth amid the bias in the science they’re reading and hearing about?  Frankly, with a bit of hard work:  i) look for converging findings from different reports; ii) avoid writing that exhibits strident, “tribalistic” language; iii) give greater weight to analyses originating in peer-reviewed journals, where accessible; and iv) favor a balanced representation of what is currently known as the foundation for any new findings, even when the ultimate finding clearly supports one aspect over another.  First and last, be skeptical.  

Related articles (mostly via Zemanta)

• The Latest Findings On Intelligence (danielwillingham.com)
• Nature or nurture? It may depend on where you live (eurekalert.org)
• Intelligence, Again (aleksandreia.com)
• Ron Unz — Writings and Perspectives (www.ronunz.org)
• In Praise of Peer Review – A Personal Perspective (scholarlykitchen.sspnet.org)
• Peer Review: The Nuts and Bolts (www.senseaboutscience.org)
• Peer review is dead, long live blog review (www.abc.net.au)