19 replies to this topic

[Okrahomer]

I was criticized in the recently-closed "Revisit Loomiss..." thread for challenging the following assertion:

“In reality, there are only around 4-6 million people on this planet who consider themselves LDS.”

My suggestion was to test it by making a simple extrapolation based on one of the most recent U.S. religious self-identification surveys.  My simple extrapolation yielded 5 – 6 million self-identified Mormons in the U.S. alone.  In response, “someone” characterized my math skills as “phony” and “bogus”; while the survey I chose (the 2007 Pew U.S. Religious Landscape Survey) was dismissed as flawed due to “oversampling.”  A link to a Joanna Brooks’ article (below) was provided to prove the “oversampling” charge.

First Point:

The idea of extrapolating estimated numbers from Pew and other well respected religious self-identification surveys is exactly what researchers, journalists and organizations like the U.S. Census Bureau do.  For instance the U.S. census used ARIS data to make this 2008 estimate for adult populations only:

http://www.census.go...les/12s0075.pdf

And Joanna Brooks used ARIS data to do the same thing here:

http://www.religiond..._not_adding_up/

And then there is this 2004 extrapolation across all age groups (children, teens and adults) which utilized one self-identification survey (ARIS) for adults and another (the UNC National Study of Youth and Religion) for teens and children:

“Some groups (e.g., Episcopal, Congregational, Judaism) have fewer children proportionate to their total population, and some groups (e.g., Catholic, Pentecostal, Latter-day Saints) have higher proportionate numbers of children, in which case the un-modified extrapolation to the total population would yield an undercount. For example, in 2004, 2.5% of American teenagers said they were Latter-day Saints (National Study of Youth and Religion, University of North Carolina at Chapel Hill; sample size: 3,370 nationwide), a figure higher than the 1.3% of American adults who identified themselves as Latter-day Saints in 2001.  If this 2.5% figure were extrapolated to the total population, it would yield a figure of 7,334,574 total Latter-day Saints (children, teens and adults) in the U.S. for 2004 - a number nearly 2 million higher than counted in official membership records. This difference in survey results between adult and teen populations indicates that this group's U.S. membership skews young. Similar differences (pushing a group's actual numbers higher than extrapolated if it skews young and lower than extrapolated if it skews old) would be expected for all groups.”  (http://www.lb9.uscou...r_relGroups.pdf)


Second Point:

I e-mailed Erin O’Connell, Associate Director for Communications at the Pew Research Center’s Forum on Religion and Public Life.  I asked if they were aware of the Joanna Brooks criticism.  (Note:  The “oversampling” criticism actually originated with ARIS researchers Richard D. Phillips and Ryan T. Cragun.)  Erin replied almost immediately and indicated that they were in fact aware of the Phillips/Cragun criticism and believed it to be “unwarranted.”  Erin suggested a short telephone conversation with a Pew researcher who could explain in more detail.  I called the researcher and enjoyed a very interesting and enlightening 30-minute conversation.  Following are some of his more relevant observations:

The Phillips/Cragun “oversampling” criticism (http://www.mormonsocialscience.org/) was actually directed at the 2011 Pew Religious Devotion survey titled “Mormons in America” which can be found here:  (http://www.Pewforum....ve-summary.aspx).  The criticism was not directed at the 2007 U.S. Religious Landscape survey (which was my source), which is found here:  http://religions.Pewforum.org/.  “Mormons in America” focused on measures of “religious devotion” and was specific to Mormons; while “Landscape” focused on “self-identification” and was specific to the entire U.S. population.  Apparently, “someone” conflated the two.

Phillips & Cragun have yet to express their criticism of “Mormons in  America” in any academic journal or appropriate forum; therefore, while Pew is aware of the criticism, it has been unable to respond officially; nevertheless, the researcher was more than happy and quite prepared to tell me over the phone why he is convinced the criticism is “unwarranted” and also “puzzling.”

For “Mormons in America”, the researcher explained that Pew did in fact utilize an “oversample” technique in certain Utah counties with higher-than-average Mormon populations; however, the data from these counties was also correspondingly “weighted” to discount for the oversampling.  This is in complete accord with long-established statistical techniques utilized within the Social Sciences.  The researcher also pointed out that results from “Mormons in America” (where “over-sampling” was utilized) were virtually identical to results from prior “Religious Devotion” surveys Pew has done where the oversampling/weighting technique was not employed.

With regard to the 2007 “Landscape” survey (which was the basis of my extrapolation), the researcher made clear that absolutely no “oversampling” technique had been employed.  He also pointed out that Pew (as well as other surveying organizations) regularly asks about religious preferences for the demographic portion of nearly every survey—even when the purpose of the survey is not about religion.  This data is “tracked” and accumulated annually:  the percentage of U.S. adults self-identifying as “Mormon” has been tracking at 1.6% to 1.7% for the past few years; but, the figure tracked at 1.9% throughout all of 2011; and it has been tracking at 1.6% so far in 2012.  In fact, the tracking percentages are not all that different—usually within the margins of error—from one reputable survey to the next.  Pew’s adult Mormon percentage tracked at 1.6% in 2007; ARIS tracked it at 1.4%; while both Pew and Gallup tracked it at 1.9% in 2011.

Finally, the Pew researcher thought it important to point out that the ARIS approach is not above criticism.  For example, ARIS reports a large percentage of U.S. adults who simply identify themselves as “Christian.”  Since ARIS does not utilize a “secondary” line of questions (like Pew and Gallup do), this percentage is significantly inflated.  A large number of respondents who initially identify themselves as “Christian” will subsequently identify themselves more specifically—as “Baptist” or “Mormon” for instance—if they are given the choice.

Please feel free to draw your own conclusions--I certainly have.

Edited by Okrahomer, 25 April 2012 - 05:14 PM.

[calmoriah]

Well done, am always impressed when people go to the source when wanting more info rather than just speculate or complain about lacks.

[Okrahomer]

Thanks, Cal.  I was surprised by how quickly the people at Pew responded as well as how totally open to a conversation they were.

[wenglund]

Nice investigative work!

Thanks, -Wade Englund-

[Okrahomer]

wenglund, on 25 April 2012 - 06:42 PM, said:

Nice investigative work!

Thanks, -Wade Englund-

Thank you.  One would like to believe people like Joanna Brooks would double-check important information like the Phillips//Cragun accusation, before repeating it.  The people at Pew are very impressive. The researcher I spoke with was genuinely interested in re-examining the data--even while I was on the phone with him--to reconfirm that they had not made an error and in fact did "get it right.".  It was a pleasure to speak with him.

[Storm Rider]

Wasn't the Pew analysis based solely on the USA rather than the world?

[Okrahomer]

Storm Rider, on 25 April 2012 - 11:15 PM, said:

Wasn't the Pew analysis based solely on the USA rather than the world?

Yes, that's right.  But the assertion of "only 4 - 6 million people on the entire planet" is rendered invalid solely on the basis of U.S. extrapolation--there at least that many self-identified Mormons in the U.S. alone.

Edited by Okrahomer, 26 April 2012 - 03:46 AM.

[Xander]

Okrahomer, on 26 April 2012 - 03:33 AM, said:

Yes, that's right.  But the assertion of "only 4 - 6 million people on the entire planet" is rendered invalid solely on the basis of U.S. extrapolation--there at least that many self-identified Mormons in the U.S. alone.

This is absolute bunk. You'd have us believe, contrary to what even the Church has said, that virtually every member on record in the United States (6 million) considers him/herself a Mormon. This is absolutely ludicrous, and I don't know how you can keep misusing this Pew survey as evidence. And I highly doubt anyone at Pew would be so naive as to think this little survey would support the conclusions you're grasping at. The fact is the vast majority of Mormons "on record" do not consider themselves Mormon. LDS statistician David Stewart has been closely examining these numbers for years. He is the owner of cumorah.com and has stated that, "While the Church makes no claims about member activity rates and no official reports of LDS activity rates are published, the Encyclopedia of Mormonism notes, 'Attendance at sacrament meeting varies substantially. Canada, the South Pacific, and the United States average between 40 percent and 50 percent. Europe and Africa average about 35 percent. Asia and Latin America have weekly attendance rates of about 25 percent.'

He also notes that, "European LDS activity rates appear to have fallen well below the older 35% figure cited in the Encyclopedia of Mormonism."

And, "the average missionary in 1989 brought 9.1 people into the church, while in 2000 the average missionary brought 4.6 people into the church. When one accounts for actual activity and retention rates, with the great majority of LDS convert growth occurring in Latin America and other areas with low retention and only 20-25% of convert growth occurring in North America, it can be determined that of the 4.6 persons baptized by the average missionary each year, approximately 1.3 will remain active.

In his article, "How Many Members Are There Really?", (Dialogue: A Journal of Mormon Thought, 38:2:53-78, Summer 2005) David Knowlton demonstrates why there is really no reason to believe more than 25% of the members in Mexico and Chile actually consider themselves LDS. National census in Brazil reveals the same type of thing, roughly 20-24% of the claimed members actually consider themselves LDS. The Chilean Census was actually a little below 20%! The Church claims to have three million members just in these three countries alone. So that's well over two million people right there the Church has no business claiming to be LDS, and this from just three countries. And before you start rationalizing how these national census's mean nothing, Stewart shoots down that piece of wishful thinking before it gets off the ground:

Quote

Census data are among the most robust of any research data because of the methodology of querying every individual in a country rather than relying upon sampled data. Most researchers could only dream of such methodological rigor and statistical power.

Nonetheless, the vast discrepancy between official LDS membership claims and self-identified religious preferences on national censuses have led some to challenge the reliability of census data. LDS Public Affairs spokesman Dale Bills stated, "There may be any number of reasons for the discrepancy, including personal preferences of some citizens regarding disclosure of their religious affiliation". The data do not bear out such claims. Of the 169,872,856 individuals queried in the 2000 Brazilian census, only 383,953 people, or 0.226%, declined to answer the question on religious affiliation. Fewer than 1% of respondents on the Mexican census (732,630 out of 84,794,454 respondents above age 5) failed to specify a religious preference. Other Latin American censuses demonstrate similarly excellent response rates that any researcher would envy. Furthermore, LDS leaders made strong official requests from the pulpit for local members to declare the LDS faith as their faith of preference in virtually every nation where the census included religious affiliation questions. It is therefore unreasonable to claim that large numbers of believing Latter-day Saints refused to identify their religious preference. Such rationalizations only obscure the obvious reality that the overwhelming majority of those who did not declare the LDS faith to be their faith of preference, in fact, do not consider it to be their faith of preference.



Further corroboration of the accuracy of census data in reporting respondents' religious preferences can be found in the consistent correlation between self-identified religious preferences and official membership for the Jehovah's Witness and Seventh-day Adventist organizations, both of which have member definitions tied to activity. The number of individuals identifying the Jehovah's Witness organization as their faith of preference on Latin American censuses weighs in at between 175% and 206% of official membership figures in these countries, representing both baptized adult members and a large number of affiliates. More individuals identified themselves as Seventh-day Adventists than are officially claimed in each country. The LDS Church enjoys a relatively positive reputation in these nations, and so it is unlikely that Mormons would be less likely than Adventists or Jehovah's Witnesses to express their true religious preferences. The consistently low correlation between LDS membership claims and self-identified census data across many nations, the high correlation between membership and census data for other denominations, and the close correspondence between census data and other research on member self-identification and participation all suggest that census data are reliable.



The comparison between 103,735 self-identified Latter-day Saints reported on the Chilean census and the 57,000 Chileans attending LDS meetings each Sunday suggests that far from short-changing the strength of the Church, census data generously overestimate the quantity of participating, committed international members. This is because national censuses report self-identified religious preferences which do not imply church activity. Without exception, the number of individuals attending church weekly is always less than the number identifying themselves as members of the LDS Church or any other faith on a national census.



The evidence overwhelmingly suggests that census data accurately represent respondents' true religious preferences. Beyond accounting for the exclusion of religious data on young children in some censuses, it is exceedingly difficult to find tenable explanations other than member disengagement for the vast discrepancies between self-identified religious affiliation and church-reported membership claims. Fractional rates of self-identification document that most individuals outside of North America officially claimed on LDS membership rolls do not consider themselves members of the LDS Church, demonstrating that the challenge of inactivity runs far deeper than inadequate socialization, economic hardship, or transportation problems.

There is every indication that these extremely low retention rates represent the norm throughout Latin America. This means the Church continues to say there are four million extra members just south of the border, which in reality don't exist - except on paper.

You can disagree without being rude.   Do NOT begin by slamming someone else's real life research "bunk" if you want to stay in the thread.  All you are doing is throwing out other data in a very disrespectful way without ever addressing the data in the OP.

[Buzzard]

While I am no statistician, and thus have little to contribute to the "meat" of this thread, it seems self-evident that using weekly sacrament meeting attendance is a pretty flawed way to judge church membership numbers.  On any given week, our ward averages 50-55% attendance.  But that does not account for members who have to work, who may be out of town or sick, or simply were tired.  Other families come some of the time, but not every Sunday. You could hardly say that someone who attends even every other month does not consider themselves LDS. Then I have been in the homes of rock solid inactives. People who have not been inside a chapel for years.  But they cheerfully greet the home teachers, even have the BOM out on the end table.  So figuring out who is and who is not really a member would involve a lot of fuzzy math.  Obviously some of the 14 million and change do not consider themselves members any more.  But 4-6 million self-identified members worldwide seems a figure plucked from a anti-'s pipe dream.

[blooit]

Slight tangent, but Adherents.com stated that LDS statistics were much more accurate than most membership records of other churches. This was about 5 years ago IIRC. I'll try to link when I have a minute.

[Okrahomer]

Xander, on 26 April 2012 - 06:13 AM, said:

You'd have us believe, contrary to what even the Church has said, that virtually every member on record in the United States (6 million) considers him/herself a Mormon.

I wasn't aware the Church had taken a position on any of this.  But since my extrapolation of Pew data yields between 5 - 6 million self-identified Mormons in the U.S., how exactly does that differ from the Church's official figures of around 6.1 million?  Are you saying the Church disputes its own numbers?

[The Nehor]

About 7 billion. Oh.....Mormons....I read morons. Carry on.

[Okrahomer]

Xander, on 26 April 2012 - 06:13 AM, said:

LDS statistician David Stewart has been closely examining these numbers for years. He is the owner of cumorah.com and has stated that, "While the Church makes no claims about member activity rates and no official reports of LDS activity rates are published, the Encyclopedia of Mormonism notes, 'Attendance at sacrament meeting varies substantially. Canada, the South Pacific, and the United States average between 40 percent and 50 percent. Europe and Africa average about 35 percent. Asia and Latin America have weekly attendance rates of about 25 percent.'

If you recall, I used David Stewart's worst case "20%" figure to extrapolate for the International segment of my "total" estimate.  But I think you are confusing "activity rates" with "self-identification."  If someone did not attend Sacrament Meeting last week, it does not mean they no longer consider themselves to be LDS. I thought self-identification was what you had in mind when you said:

In reality, there are only around 4-6 million people on this planet who consider themselves LDS.”

[Okrahomer]

Xander, on 26 April 2012 - 06:13 AM, said:

This is absolute bunk.

It depends on your definition of "bunk".  How would you describe your citation of Joanna Brooks to support your argument?  As it turns out, she did not do a very good job for you.

First:  She extrapolated 4.4 million self-identified U.S. Adult LDS.  Assuming you consider teenagers and children to be "people on this planet", your own witness (Joanna Brooks) disproved your assertion.

Second:  The "oversampling" criticism (as cited in the article) was directed at the "Mormons in America" survey, NOT the "Landscape" survey.

[Okrahomer]

Xander, on 26 April 2012 - 06:13 AM, said:

And I highly doubt anyone at Pew would be so naive as to think this little survey would support the conclusions you're grasping at.

I did not ask the Pew researcher about my extrapolation, so in a way, you are correct about this.  He did say that correctly interpreting the data required careful understanding and application of the margin of error.

My conversation with the folks at Pew was much more focused on your accusation of "oversampling."  As I have explained above, your accusation was not only "unwarranted", it was also directed at the wrong survey.

Edited by Okrahomer, 26 April 2012 - 06:12 PM.

[Okrahomer]

The Nehor, on 26 April 2012 - 03:09 PM, said:

About 7 billion. Oh.....Mormons....I read morons. Carry on.

[KevinG]

The Nehor, on 26 April 2012 - 03:09 PM, said:

About 7 billion. Oh.....Mormons....I read morons. Carry on.

The two subsets do cross in places.

[KevinG]

Are there Mormons that work for PEW research?  If so we will have to discount all of their findings due to bias.  At least that's what I have learned from some of the brilliant and scholarly critics of Mormonism on this board.

Edited by KevinG, 26 April 2012 - 07:04 PM.

[Analytics]

Hi Okrahomar,

Nice post.

For my part, I don't have a good handle on how many self-identifying Mormons there are in the states.  The range of 1.4% to 1.9% that is coming out of the surveys is quite large.  It would seem that using a survey to get info about such a small percentage of the overall population is going to be especially subject to bias.  One thing that would be interesting is to consider the "self-identifying ratio," defined as the self-identifying members in a population divided by the church's reported membership.  In Utah, the number is something like 5/7--as I recall, about 50% of Utahns self-identify as members, while the church claims 70% of the state is on the rolls. If that 5/7 ratio is applicable to the United States as a whole, then if we asked all 6.1 million U.S. Mormons what their religion is, only about 4.5 million would self-identify.

Be that as it may, there is one part of what you said that needs clarifying:

Okrahomer, on 25 April 2012 - 05:03 PM, said:

I e-mailed Erin O’Connell, Associate Director for Communications at the Pew Research Center’s Forum on Religion and Public Life.  I asked if they were aware of the Joanna Brooks criticism.  (Note:  The “oversampling” criticism actually originated with ARIS researchers Richard D. Phillips and Ryan T. Cragun.)  Erin replied almost immediately and indicated that they were in fact aware of the Phillips/Cragun criticism and believed it to be “unwarranted.”  Erin suggested a short telephone conversation with a Pew researcher who could explain in more detail.  I called the researcher and enjoyed a very interesting and enlightening 30-minute conversation.  Following are some of his more relevant observations:

The Phillips/Cragun “oversampling” criticism (http://www.mormonsocialscience.org/) was actually directed at the 2011 Pew Religious Devotion survey titled “Mormons in America” which can be found here:  (http://www.Pewforum....ve-summary.aspx).  The criticism was not directed at the 2007 U.S. Religious Landscape survey (which was my source), which is found here:  http://religions.Pewforum.org/.  “Mormons in America” focused on measures of “religious devotion” and was specific to Mormons; while “Landscape” focused on “self-identification” and was specific to the entire U.S. population.  Apparently, “someone” conflated the two.

Just to be clear, the "someone" you are alluding to is somebody other than Joanna Brooks, right?  In the link to her blog, she talks about the oversampling, but her criticism is in regards to activity rates among self-identified members.  The "Mormons in America" survey claims 65% of self-identifying Mormons regularly go to the temple.  Brooks says, "Those are eye-popping numbers that don’t quite match up to what most Mormons experience week-to-week in their congregations."  She blames over-sampling for this.

Does that jive with your expeirence?  Of everybody in your ward--innactive and active--that would self-identify as members, do you think 65% not only have temple recommends, but also go to the temple "regularly"?  Do you think that of the 5-6 million Mormons in America, 65% regularly attend the temple?  That's what Pew claims.

[Okrahomer]

Analytics, on 26 April 2012 - 07:53 PM, said:

For my part, I don't have a good handle on how many self-identifying Mormons there are in the states.  The range of 1.4% to 1.9% that is coming out of the surveys is quite large.  It would seem that using a survey to get info about such a small percentage of the overall population is going to be especially subject to bias.  One thing that would be interesting is to consider the "self-identifying ratio," defined as the self-identifying members in a population divided by the church's reported membership.  In Utah, the number is something like 5/7--as I recall, about 50% of Utahns self-identify as members, while the church claims 70% of the state is on the rolls. If that 5/7 ratio is applicable to the United States as a whole, then if we asked all 6.1 million U.S. Mormons what their religion is, only about 4.5 million would self-identify.

Pew, ARIS, and Gallup survey adults only (18 years and older), so the 1.4 % - 1.9% applies specifically to the U.S. adult population.  You have to make a secondary extrapolation for teens and children, because the LDS Church includes them in membership figures.  The only reputable survey of teens in the U.S. of which I am aware is the University of North Carolina study (previously cited), which indicates a higher percentage of U.S. teens (2.5%) self-identify as Mormon vs. the 1.4% - 1.9% of adults.  I believe this closes quite a bit of the gap.  Your 4.5 million is almost identical to Joanna Brooks’ 4.4 million—all adults.

Analytics, on 26 April 2012 - 07:53 PM, said:

Just to be clear, the "someone" you are alluding to is somebody other than Joanna Brooks, right?

I was alluding to Xander.  He linked to the Brooks’ article to support his allegation that the 2007 Pew U.S. Religious Landscape survey should be dismissed due to “oversampling.”  What I found out from Pew, is that Brooks (a la Phillips & Cragun) was actually criticizing the 2011 Pew “Mormons in America” survey; not the 2007 “Landscape.”  The 2007 Landscape survey was the source of my extrapolation.  Pew confirmed there was absolutely no oversampling in the 2007 survey.

Analytics, on 26 April 2012 - 07:53 PM, said:

In the link to her blog, she talks about the oversampling, but her criticism is in regards to activity rates among self-identified members.  The "Mormons in America" survey claims 65% of self-identifying Mormons regularly go to the temple.  Brooks says, "Those are eye-popping numbers that don’t quite match up to what most Mormons experience week-to-week in their congregations."  She blames over-sampling for this.

I do not entirely discount her skepticism; however, I think it unfair that she did not allow Pew to respond to the accusation before repeating it.  The Pew researcher I spoke with confirmed that they had in fact utilized an “oversampling/weighting” technique for the 2011 "Mormons in America" survey—but only for this survey.  He was (is) convinced they made all the correct adjustments to "get it right."  Again:  They did not utilize “oversampling/weighting” for the 2007 Landscape survey which was the basis of my U.S. extrapolation.