More Junk Data On Lds Apostasy Rates

[canard78]

Recently some new "apostasy" data has been doing the rounds. I wanted to give my perspective on it and see what others here think.

http://www.fullerconsideration.com/membershipmethodology.php

In the above data the author claims that activity has declined consistenly over the last 100 years. In other words, the church might be growing overall numbers of members but the levels of inactivity/apostasy are growing in both number and % of the total.

Some of his data is reliable - especially the stuff he's taken from LDS primary sources. So, for example, convert baptisms are reliable.

The problems start with the children of record modelling.

E.G.: Convert baptisms from children of record. He bases this on "current US age distribution." That's fine for any current US converts. But it's not for any US historical data and not for any non-US convert families. He's missing variables so his model will be flawed on that measure.

He makes the same false assumptions when estimating birth rates. He inputs current Utah/US birth rates.

The reason children of record data really matters is because that's his source for the data we're really interested in (the data we don't have any reported access to): activity rate and apostasy rate.

He makes the assumption that new children of record are only reported to the church by active members. Therefore, he assumes, we can model a hypothetical activity rate by looking at children of record factored against birth rates.

Simple example:

100 adult members total.

Birth rate: 5 per hundred adults.

New children of record this year: 5

He therefore assumes 100% activity.

That's because 100 people had 5 children between them and all 5 were reported to the church. That means he assumes that the 10 people (5 couples) who produced the children were active when they reported the birth to the church and he also assumes that those 10 active people are representative of the other 90 in the group for whom we have no data and no indicator of activity. He's using a 10% sample of the total to make assumptions.

If, in the following year the numbers look like this, he can then estimate the activity rate again:

100 adult members total.

Birth rate is still: 5 per hundred adults.

New children of record this year: 3

He therefore assumes 60% activity.

The same total number of people have reported the birth of fewer children. He can therefore assume that those people still had the unreported children (given the birth rate is stable).

The problem with this approach is it makes far too many assumptions.

Even if he had a consistent set of data over time, he's still selected a sample (new parents) that's not representative of all other members.

New parents aren't representative of all Mormons, but the model assumes they are. Is the activity rate among young adults or empty nesters higher or lower than young families?

He's also not looking at a consistent set of people. If in year one, 5 couples produced 5 children and then in year two 5 couples produced 3 children then what has changed? It's likely that they are different couples from within the base of 100 people. If we accept that 3 children reported and an assumed 2 children unreported really does indicate inactivity what does that tell us? Were those two unreported births to couples who were inactive in year 1 and 2? Or do they go inactive (as he suggests) in year two. We only have a data point for them in year 2 as they had no children to report (or not) in year 1.

Finally, even if new parents were a reasonable sample to run a model against, you need a consistent and representative data set, which he doesn't have.

The church's methodology for reporting births has varied hugely over the last 100+ years. In fact, it seems to be one of the most changeable data set he uses and yet it's the most critical one to make his inactivity trend work. He's also feeding birth rate data into a global and historic data set but using US birth rate data to fill gaps.

You can't model with confidence with unrepresentative data and lots of methodology changes.

His model looks credible because he's reached an end point of 36% current activity. This is a generally accepted figure based on several sources (including LDS). Even that is probably wrong. 36% is probably average attendance at sacrament meeting. Many members don't attend every week. So 36% attendance could be 50% or 40% or 60% activity... we don't really know.

He creates a model to fit an accepted/credible end point and then applies that model to the historic data. When it looks unreasonable (there's an unrealistic drop in activity a few years) he goes back to the model and tweaks it until it fits his narrative.

Based on where he shared this data first and based on the tone he shared it with, there's a clear agenda of wanting to show the church in a long and slow demise. If you have an assumed outcome and set narrative then it's fairly easy to create a model that fits that narrative.

I'm not saying he has been intentionally deceptive. It's just a simply reality of research (whether statistical or otherwise) that your assumptions will influence your conclusions.

Edited May 27, 2015 by canard78

[Mystery Meat]

Recently some new "apostasy" data has been doing the rounds. I wanted to give my perspective on it and see what others here think.

http://www.fullerconsideration.com/membershipmethodology.php

In the above data the author claims that activity has declined consistenly over the last 100 years. In other words, the church might be growing overall numbers of members but the levels of inactivity/apostasy are growing in both number and % of the total.

Some of his data is reliable - especially the stuff he's taken from LDS primary sources. So, for example, convert baptisms are reliable.

The problems start with the children of record modelling.

E.G.: Convert baptisms from children of record. He bases this on "current US age distribution." That's fine for any current US converts. But it's not for any US historical data and not for any non-US convert families. He's missing variables so his model will be flawed on that measure.

He makes the same false assumptions when estimating birth rates. He inputs current Utah/US birth rates.

The reason children of record data really matters is because that's his source for the data we're really interested in (the data we don't have any reported access to): activity rate and apostasy rate.

He makes the assumption that new children of record are only reported to the church by active members. Therefore, he assumes, we can model a hypothetical activity rate by looking at children of record factored against birth rates.

Simple example:

100 adult members total.

Birth rate: 5 per hundred adults.

New children of record this year: 5

He therefore assumes 100% activity.

That's because 100 people had 5 children between them and all 5 were reported to the church. That means he assumes that the 10 people (5 couples) who produced the children were active when they reported the birth to the church and he also assumes that those 10 active people are representative of the other 90 in the group for whom we have no data and no indicator of activity. He's using a 10% sample of the total to make assumptions.

If, in the following year the numbers look like this, he can then estimate the activity rate again:

100 adult members total.

Birth rate is still: 5 per hundred adults.

New children of record this year: 3

He therefore assumes 60% activity.

The same total number of people have reported the birth of fewer children. He can therefore assume that those people still had the unreported children (given the birth rate is stable).

The problem with this approach is it makes far too many assumptions.

Even if he had a consistent set of data over time, he's still selected a sample (new parents) that's not representative of all other members.

New parents aren't representative of all Mormons, but the model assumes they are. Is the activity rate among young adults or empty nesters higher or lower than young families?

He's also not looking at a consistent set of people. If in year one, 5 couples produced 5 children and then in year two 5 couples produced 3 children then what has changed? It's likely that they are different couples from within the base of 100 people. If we accept that 3 children reported and an assumed 2 children unreported really does indicate inactivity what does that tell us? Were those two unreported births to couples who were inactive in year 1 and 2? Or do they go inactive (as he suggests) in year two. We only have a data point for them in year 2 as they had no children to report (or not) in year 1.

Finally, even if new parents were a reasonable sample to run a model against, you need a consistent and representative data set, which he doesn't have.

The church's methodology for reporting births has varied hugely over the last 100+ years. In fact, it seems to be one of the most changeable data set he uses and yet it's the most critical one to make his inactivity trend work. He's also feeding birth rate data into a global and historic data set but using US birth rate data to fill gaps.

You can't model with confidence with unrepresentative data and lots of methodology changes.

His model looks credible because he's reached an end point of 36% current activity. This is a generally accepted figure based on several sources (including LDS). Even that is probably wrong. 36% is probably average attendance at sacrament meeting. Many members don't attend every week. So 36% attendance could be 50% or 40% or 60% activity... we don't really know.

He creates a model to fit an accepted/credible end point and then applies that model to the historic data. When it looks unreasonable (there's an unrealistic drop in activity a few years) he goes back to the model and tweaks it until it fits his narrative.

Based on where he shared this data first and based on the tone he shared it with, there's a clear agenda of wanting to show the church in a long and slow demise. If you have an assumed outcome and set narrative then it's fairly easy to create a model that fits that narrative.

I'm not saying he has been intentionally deceptive. It's just a simply reality of research (whether statistical or otherwise) that your assumptions will influence your conclusions.

 

I mentioned this recently in another thread, but someone I know has seen the numbers and his report to me was that the Church is at historic highs (at least for the modern church) in terms of activity rates. 

 

FWIW.

[BookofMormonLuvr]

I mentioned this recently in another thread, but someone I know has seen the numbers and his report to me was that the Church is at historic highs (at least for the modern church) in terms of activity rates. 

 

FWIW.

If the traditional activity rate is 35% and it is at 37%- is that a historic high?

 

[Mystery Meat]

If the traditional activity rate is 35% and it is at 37%- is that a historic high?

 

Sure. 2% would be quite a jump in pure numbers.

 

 

ETA: 2% would represent roughly 300,000 previously baptized members that weren't attending, that are now.

Edited May 27, 2015 by lvjd66

[cinepro]

The stupid thing about all this is that the Church has the numbers.  The only reason the report in the OP has to guess and estimate is because the Church chooses not to make them public.

[lithpblithh]

When I first saw it I had a hard time with this graph:

 

http://www.fullerconsideration.com/images/Fig_4b_Activity_Rate.PNG

 

90%-100% activity rate during 1920-1935 and mid 1950s? Yeah right.

[rockpond]

Recently some new "apostasy" data has been doing the rounds. I wanted to give my perspective on it and see what others here think.

http://www.fullerconsideration.com/membershipmethodology.php

In the above data the author claims that activity has declined consistenly over the last 100 years. In other words, the church might be growing overall numbers of members but the levels of inactivity/apostasy are growing in both number and % of the total.

Some of his data is reliable - especially the stuff he's taken from LDS primary sources. So, for example, convert baptisms are reliable.

The problems start with the children of record modelling.

E.G.: Convert baptisms from children of record. He bases this on "current US age distribution." That's fine for any current US converts. But it's not for any US historical data and not for any non-US convert families. He's missing variables so his model will be flawed on that measure.

He makes the same false assumptions when estimating birth rates. He inputs current Utah/US birth rates.

The reason children of record data really matters is because that's his source for the data we're really interested in (the data we don't have any reported access to): activity rate and apostasy rate.

He makes the assumption that new children of record are only reported to the church by active members. Therefore, he assumes, we can model a hypothetical activity rate by looking at children of record factored against birth rates.

Simple example:

100 adult members total.

Birth rate: 5 per hundred adults.

New children of record this year: 5

He therefore assumes 100% activity.

That's because 100 people had 5 children between them and all 5 were reported to the church. That means he assumes that the 10 people (5 couples) who produced the children were active when they reported the birth to the church and he also assumes that those 10 active people are representative of the other 90 in the group for whom we have no data and no indicator of activity. He's using a 10% sample of the total to make assumptions.

If, in the following year the numbers look like this, he can then estimate the activity rate again:

100 adult members total.

Birth rate is still: 5 per hundred adults.

New children of record this year: 3

He therefore assumes 60% activity.

The same total number of people have reported the birth of fewer children. He can therefore assume that those people still had the unreported children (given the birth rate is stable).

The problem with this approach is it makes far too many assumptions.

Even if he had a consistent set of data over time, he's still selected a sample (new parents) that's not representative of all other members.

New parents aren't representative of all Mormons, but the model assumes they are. Is the activity rate among young adults or empty nesters higher or lower than young families?

He's also not looking at a consistent set of people. If in year one, 5 couples produced 5 children and then in year two 5 couples produced 3 children then what has changed? It's likely that they are different couples from within the base of 100 people. If we accept that 3 children reported and an assumed 2 children unreported really does indicate inactivity what does that tell us? Were those two unreported births to couples who were inactive in year 1 and 2? Or do they go inactive (as he suggests) in year two. We only have a data point for them in year 2 as they had no children to report (or not) in year 1.

Finally, even if new parents were a reasonable sample to run a model against, you need a consistent and representative data set, which he doesn't have.

The church's methodology for reporting births has varied hugely over the last 100+ years. In fact, it seems to be one of the most changeable data set he uses and yet it's the most critical one to make his inactivity trend work. He's also feeding birth rate data into a global and historic data set but using US birth rate data to fill gaps.

You can't model with confidence with unrepresentative data and lots of methodology changes.

His model looks credible because he's reached an end point of 36% current activity. This is a generally accepted figure based on several sources (including LDS). Even that is probably wrong. 36% is probably average attendance at sacrament meeting. Many members don't attend every week. So 36% attendance could be 50% or 40% or 60% activity... we don't really know.

He creates a model to fit an accepted/credible end point and then applies that model to the historic data. When it looks unreasonable (there's an unrealistic drop in activity a few years) he goes back to the model and tweaks it until it fits his narrative.

Based on where he shared this data first and based on the tone he shared it with, there's a clear agenda of wanting to show the church in a long and slow demise. If you have an assumed outcome and set narrative then it's fairly easy to create a model that fits that narrative.

I'm not saying he has been intentionally deceptive. It's just a simply reality of research (whether statistical or otherwise) that your assumptions will influence your conclusions.

 

I agree... too many assumptions.

 

I saw the report posted on Facebook.  It would interest me if, for example, he had one unknown variable which he replaced with an assumption and modeled a hypothetical result.

 

But, he's got multiple unknown variables for which he has stacked assumptions on top of assumptions.  As a statistician, I can see the fun in what he's doing but I don't believe it has much value in terms of worthwhile data.

 

To make matters worse, he's starting with the data that the church publishes in gen con which already appears to have problems (e.g. our super low mortality rate).

[The Nehor]

The stupid thing about all this is that the Church has the numbers.  The only reason the report in the OP has to guess and estimate is because the Church chooses not to make them public.

 

Maybe it is keeping people out of trouble.

[canard78]

I mentioned this recently in another thread, but someone I know has seen the numbers and his report to me was that the Church is at historic highs (at least for the modern church) in terms of activity rates. 

 

FWIW.

 

Do you mean they've seen the official numbers? If he means the data in the OP then then the good news in this data modelling should be taken with as big a pinch of salt as the bad.

 

According to his figures, activity was at 31.2% in 2014, down .1% vs 2013.

[Mystery Meat]

Do you mean they've seen the official numbers? If he means the data in the OP then then the good news in this data modelling should be taken with as big a pinch of salt as the bad.

 

According to his figures, activity was at 31.2% in 2014, down .1% vs 2013.

 

Yes, the official numbers.