Faith and Statistics

This week I read a recently published study [1] on the correlation between a nation's religiosity and "quantifiable societal health." The study itself is quite interesting, revealing that major indicators of societal health do not improve as religiosity increases. Perhaps equally interesting are the internet responses to this study, ranging from blogs to religious news organizations to academic editorials. The internet responses reveal repeated (and exasperating) failures to properly understand and comprehend statistical information. I want to present some of these responses in this post. They represent extremely common fallacies and rhetorical abuses which occur in discussions of all kinds of topics, and have (over time) influenced me to give up trying to discuss anything with anyone.

First, a review of the study's claims and results. The study's author, Gregory S. Paul, analyzed data collected by several large-scale, multi-national surveys. Only wealthy western nations (and Japan) are considered in the study because of their close social and economic similarities. As Paul puts it, the twentieth century was a "global societal experiment" and the time is now ideal for collection and analysis of comparative data across the western world. Paul's objective is to quantitatively test claims such as that made by Tom DeLay, that many social ills result from children being taught "that they are nothing but glorified apes who have evolutionized out of some primordial soup of mud." Paul is testing the notion that Atheism hurts and Religion helps.

"Religiosity" is assessed through six nation-wide measures (apparently self-identified by survey respondents): strong belief in God, amount of time spent in prayer, regularity of church attendence, literal belief in the Bible, and belief in evolution. The study also considered the prevalence of atheists and agnositics as an opposite scale for religiosity (presumably also reflecting self-identification). Clearly, this definition primarily means Christian religiosity, and emphasizes fundamentalism in some measures. Several of the measures apply equally well to quantifying the presence of faithful Jews and Muslims, and "everything else" is quantified by singling out atheists and agnostics (generically spiritual people, wiccans, and whatever else are not likely to call themselves "atheists").

"Quantifiable societal health" is measured through the national rates of homicide, youth suicide, life-expectancy, teenage pregnancy, teenage abortions, teenage births, gonorrhea infection, syphilis infection, and under-five mortality.

The results: Most of the results show fairly clear trends among western nations: those with more religiosity fair worse. In some cases, most of the nations appear clustered together with the US as a conspicuous outlier. This seems to be the case with homicide and suicide rates. In these cases, the trends become more clear when plotted against belief in evolution or number of atheists. Clear trends include: evolutionism is associated with atheism; atheism is associated with reduced homicide, suicide, infant mortality, STDs, teen pregnancies, and abortions. Atheism also seems to be associated with increased life expectancy.

Meanwhile, increased prayer is strongly correlated with increased infant mortality, Biblical literalism is associated with increased teen pregnancy, and "absolute belief in God" is strongly correlated with increased abortions. These trends are clear even when the US is ignored as an outlier.

Conclusions: The results don't allow us to say too much about causation, but we can draw a few carefully worded conclusions. Christianity has been going strong for a long time in the US, and has had decades to fix our social ills. It doesn't seem likely that American religiosity is a consequence of high crime and teen pregnancies. Similarly, religiosity has been dropping throughout Europe for decades, and it never really existed in Japan (not the way it does in Europe and the US). In those countries, atheism and evolutionary theory have had plenty of time to screw things up. They haven't. We can quite safely conclude that

• High levels of religiosity do not solve social problems, and
• Rising levels of atheism and evolutionism do not exacerbate social problems.

Responses:

Critical responses abound on the web. They come from "objective" religious news organizations [2], liberal Christians trying to justify their existences [3], evangelical propagandists [4], conservative jingoists [5], skeptical/liberal bloggers [6-7], academic magazines [8], and legitimate new organizations [9]. The Internet Infidels even chimed in with a half-interested page of posts best summarized as "meh" [10].

Although the study never says otherwise, most respondants are kind enough to remind us that "correlation does not equal causation" [2-3,6-7,10]. I might also add that dolphins live in the ocean, lest we forget. Sadly, Ruth Gledhill [9] did appear to conflate correlation with causation in a Times Online article. Blogger "PunditGuy" [5] was quick to put her in her place, saying "The story Ruth... strung together is the crapiest of crap. As a matter of fact, it’s so chock-fulla-crap that future crap stringers will be hard pressed to knock ‘ol Ruth off of the crap heap." It's nice to see how the internet creates a diverse marketplace of carefully reasoned civil discourse.

Most respondants also embraced the most common and irritating statistical mistakes: appeals to outliers, anecdotes and special cases as counter-examples, straw-man arguments, and attacks on the researcher's ulterior motives. Another common error is the appeal to "other factors" which may account for a phenomenon, which distracts from correlative results but doesn't actually refute them. Examples of these fallacies might look like this:

• "Some dolphins live in rivers. Therefore dolphins do not live in the ocean."
• "I once saw a dolphin that lived at the West Edmonton Mall, very far from the ocean. Your study can't account for that, and is therefore deeply flawed."
• "If every dolphin was a fresh-water dolphin, then dolphins wouldn't live in the ocean, and the study would be totally wrong."
• "This researcher has a clear agenda to poke holes in the 'fresh-water-dolphins' theory, and the results must therefore be suspect."
• "Other factors, such as salty water, may be more relevant in explaining dolphin habitat."

It may seem patronising to trot out these ridiculous fallacies, but people -- almost everybody, even smart people -- make these mistakes constantly and they drive me nuts.

Outliers and Anecdotes.

Zenit.org describes itself as "an international news organization" whose mission is "to provide objective coverage of events, documents and issues emanating from or concerning the Catholic Church." It probably shouldn't be a surprise that their commentary on the Paul study [2] is less than clear-minded. "If religion is linked to social problems," Zenit asks, "then Ireland should be improving socially, since it is now growing more secular." Zenit seems to have noticed that statistical correlations do not hold absolutely for all places at all times.

Zenit also gives us an anecdote from "highly secular Britain," where "the press publishes a constant stream of articles lamenting the ever-higher rates of infections from sexually transmitted diseases." While the British may perceive an alarmingly high rate of STDs, quantitatively the British levels are still much lower than in the US. Sheesh.

Not content with the simple fallacies above, Zenit also serves up this complicated multi-error contraption:

Other differences between Europe and the United States, such as the fact that one is a society composed of immigrants, may well be more relevant in explaining social trends. Or perhaps it is due to the higher levels of economic inequality, or the relative newness of American society compared to Europe.

First we see a straw-man: it is implied that the study only "compares" the US with Europe. In fact, the study quantifies data separately for several European countries, the US, and Japan. The reported trends occur within Europe not just in comparisons between the US and Europe. Zenit also gives us a non-sequitor with the gibberish about inequality and "social newness." These may be parallel factors, but they say nothing against the correlations revealed in Paul's study.

A similar (though rhetorically complex) fallacy is offered by Eric Lyons of Apologetics Press, who says [4]:

The God of the Bible cannot logically be blamed because “theists” or “Christians” forsake His commands and do that which is right in their own eyes (cf. Judges 17:6). Furthermore, simply because the more atheistic, pro-evolution democracies do not permit their godless philosophy of life to produce the true fruits of the “survival of the fittest” mentality, but rather choose to live according to moral guidelines similar to those found in the Bible (e.g., not murdering, stealing, lying, etc.), does not mean that alleged low rates of crime, murder, etc. is the fruit of true atheistic thought. In short, unrighteousness, whether it stems from atheism or a corrupted form of Christianity, produces bitter fruit that will eventually bring about the wrath of God.

Lyons seems to be offering a global conspiracy theory in which atheists deliberately act Christ-like in order to fool the world into embracing atheism. Lyons also seems to think that this is a "logical" thing for atheists to be doing. There is nothing so difficult as refuting an absurd delusion. In order to refute it, you must first make sense of it. I'm not sure that is even possible with this paragraph.

Hypothetical Agendas

Bob, a liberal Christian blogger [3], wants badly to justify his existence, which doesn't square well with Paul's finding that societies don't successfully mix religiosity with belief in evolution. Bob says,

Paul dismisses the idea that one can believe in both Christ and evolution. Hmmm. Ever feel like you don’t exist? Or that your congregation, your denomination, your entire segment of the Christian population is invisible? Of course one can be religious and accept evolution. But this tips us off to a bias in Paul’s study.

Two of his criteria for measuring religiosity are the percentage of the population that accepts evolution, and a literal belief in the Bible. Belief in creationism and biblical inerrancy is largely a characteristic of American-style fundamentalist Christianity. So Paul really isn’t measuring the level of religiosity, but of conservative Christianity, which is a set of beliefs largely originating in the US. Small wonder the US is the outlier.

I think I answered Bob's complaints adequately well in the introduction to this essay.

Bob also says, "We have no data to infer that religion causes murders, nor that murders cause religion. Speculation about the causality involved here shines more light on the writer’s worldview and mental frame than it does on the state of religion or social dysfunction." That's a slick move: it isn't the data, it's the world-view. Quite post-modern.

Bob offers a more palatable interpretation:

I’m proud to say that bloggers have been more perceptive regarding this question of which is the cause and which the effect. Richard Mathis at Jesus Was a Bleeding-Heart Liberal has a post titled very succinctly “Why Jesus hung out with Drunks and Whores”. Perhaps communities beset by pressing social ills feel their own powerlessness and need for God more than others.

This interpretation fits extremely well with the social justice emphasis of progressive Christianity, but doesn't seem likely. It is true that our poor-health indicators are quite high in the US, but are the majority of religious people really "beset" with those ills? I don't think that homicide and teen pregnancy rates are sufficient (under any model) to account for the high degree of religiosity in the US.

Eric Lyons [4] tells us that Paul's study is meaningless because America is full of fake Christians:

Most Americans believe in a higher power, which they may call “God,” but for many this is not the God of the Bible. They simply believe in a “convenient” creator, who allows them to do whatever feels good. They reject the Bible as revelation from God, and choose to live according to their own rules (which can lead to a dysfunctional society if those “rules” are contrary to biblical mandates). A great percentage of the remaining theists in America who call themselves Christians have perverted Christianity to the extent that somehow (among other things) having sexual relations outside of a scriptural marriage and killing innocent, unborn babies is acceptable. This type of theism is no better than atheism, and its fruit will be just as bitter.

Lyons concludes that "A country comprised of true Christians would be mostly void of such things as sexually transmitted diseases, murder, thievery..." and everything else. If all Christians behaved perfectly, then their behavior would be perfect; a pretty hypothetical situation, but impossible and pointless.

Blogger Iain Jackson [7] wants to know why there are so many suicides in Australia. He also suggests that gun laws might have something to do with the differences. And he says,

I'm just saying that this study needed to be put together with, perhaps, a tad more rigor and a bit less apparent agenda. (Said agenda, judging from the bibliography, to play whack-a-mole with the concept of "intelligent design". While ID can certainly be whacked with wild abandon on its merits, I'm not at all sure this article will be a good one to do it with. And, in any event, merits of the article aside, Europeans and some Asians will look at it and say, "Yep, that's what's wrong with the US, all right."

Once again, the article isn't just a comparison of the US with Europe. The trends also hold within Europe in comparisons between European countries. The attack on Paul's motives is strange; the claims of Intelligent Design are referenced to motivate the study. This does not mean that Paul set out with an agenda to discredit Intelligent Design. Even if Paul had such an agenda, it would make no difference to the study's data or results.

Data Points

Bob of I am a Christian Too says "Many of the charts in Paul’s study show a clustering of the European and Asian countries with the US off by itself in a corner. The sample size is not large enough to show a true trend" [3]. Iain Jackson [7] asks for more "rigor."

But Lee Salisbury notes [6] that "Mr. Paul’s report is based on a decade long, cross-national collaboration on social science surveys of 38 nations and 23,000 interviews." This would seem to be the largest, most rigorous study ever conducted on this subject. Saying there are too few data points is like saying "too few Western countries participated in the 20th century."

Academic Scorn

One of the more interesting and eye-crossing critiques comes from Hally Hall-I Chu, writing in The Revealer (New York University) [8]. Chu beats around a number of bushes. First she downplays the significance and uniqueness of Paul's findings, saying "Such studies are not uncommon." Second she assigns Paul a motive: "Paul’s aim is to debunk the common assumption that religion exists for the good of the society."

Third, Chu reminds us that it is too soon to make a judgement call. Then she makes a judgement call: "The paper, published this month, is too recent for a judgment call on its validity. But factual contents aside, it is a study imbued with gross generalizations and questionable premises." Which of Paul's generalizations are gross, and which premises questionable? Chu never tells us.

This is where things get interesting: Chu digresses at some length about "Orientalism":

Paul is working within an academic climate that has undergone great changes in the last few decades. In 1978, Edward Said published his seminal work, Orientalism, which brought to light the problematic approaches that social science scholars had been using: forced construction of the non-West, exoticism of the Other, and assumption of cultural homogeneity regardless of class, education level, and gender.

Said was by no means the only voice of dissent against the older scholars' Orientalist views regarding the non-West. Other academics had preceded Said in calling for changes in scholarship concerning the Other. There was already a general trend starting in the mid-1900s toward not viewing cultures as monolithic entities. Said’s book merely pushed the effort into the academic mainstream.

In light of this long-time effort to strengthen the accuracy of cultural and religious studies, the blanket assumptions that Gregory Paul employs in his study are nothing less than a step backward for the social science disciplines.

Wow, what a smack down! I just wish Chu had identified one specific actual problem with Paul's study, explained it, and offered a plan to solve it in the next decade-long multi-national cooperative 23,000 participant study.

Chu's article winds and twists in a few other directions, but the real substance of her complaint is "the study's lack of a more well-rounded definition of 'religiosity.'" On this subject she says:

Paul’s view of America’s religious demographic is incomplete. After reading the study, one is inclined to believe that America is a Christian nation, religious organizations exist only to worsen social problems, and all religious people in America adhere to the Christian faith.

...Is skepticism the only alternative to Christianity? Where would America’s Muslims, Pagans, and practitioners of Falun Gong fall under Paul’s study?

Paul does mention in passing that there are variations in religiosity. But the statistics that he cites in support of his core arguments all underscore a Christian-only view of American religiosity. Different kinds of religious people might believe religion is good for society; but it is statistics from the Christian faithful of the Midwest and the South that Paul uses to debunk this notion.

As I noted in the introduction to this essay, Paul's study is designed primarily to test claims made by conservative Christians about conservative Christian beliefs. The study is also not invalid with respect to numerous other forms of religiosity, captured by (if nothing else) the prevalence of self-identified "atheists" and "agnostics" which represent the degree to which "everything else" is absent.

Conclusions

An essay of this magnitude demands some conclusions. From the above examples I must conclude that a well-reasoned, quality discussion on such a subtle philosophical and scientific topic is like a needle in a haystack. So many are quick to produce "rebuttals," to express their disapproval. If a solid, well-argued critique exists, it is not likely to be found amidst the chatter of journalists and bloggers and evangelists. Maybe such a critique will show up eventually at my University library, but wouldn't it be something to find it in conversation, in real-time? Sadly, I usually find only fallacies of all types. Fallacies and a hostile unwillingness to receive correction. The ability to think logically and carefully must be the rarest of all human talents.

References:

• [1] Gregory S. Paul, "Cross-National Correlations of Quantifiable Societal Health with Popular Religiosity and Secularism in the Prosperous Democracies," Journal of Religion and Society, vol. 7, 2005.
• [2] Zenit News Agency, "Religion: Harmful for Society? New Study Says Yes, But Its Argument Shows Flaws." Zenit.org, Oct. 15, 2005.
• [3] Bob, "Religion and Societal Health," I am a Christian Too (blog), Oct. 2, 2005.
• [4] Eric Lyons, M.Min., "Theism or Atheism: Whose Fruit is Sweeter?" Apologetics Press :: In the News, 2005.
• [5] Bill Nienhuis, "Today's Winner," Pundit Guy (blog), Sept. 27, 2005.
• [6] Lee Salisbury, "Religion May Be Dangerous to Our Health," Dissident Voice (blog), Oct. 11, 2005.
• [7] Iain Jackson, "What about this God person, anyway?" Grim Amusements (blog), Oct. 2, 2005.
• [8] Hally Hall-I Chu, "We Are the Others," The Revealer, Center for Religion and Media at New York University, 2005.
• [9] Ruth Gledhill, "Societies worse off 'when they have God on their side'," The Times Online, Sept. 27, 2005.
• [10] Internet Infidels online discussion thread.