Jump to content

Solomon Spalding (for the millionth time)


Uncle Dale

Recommended Posts

During the last several months I've frequently been asked to comment

upon the content and findings published by Matt Jockers' team of Stanford

computer analysts in their 2008 Literary and Linguistic Computing

paper, entitled: "Reassessing Authorship of the Book of Mormon"

2008LLC4.jpg

Not being a professional statistician, I've resisted saying much

about this publication, but have long hosted a web-page devoted

to the subject, here:

http://solomonspalding.com/Lib/Jock2008.htm

I am informed that my reproduction of the Jockers paper must be

drastically reduced in content, in order to comply with copyright

restrictions. In the next week I'll delete much of my on-line file.

If anybody wishes to retain a copy of this html version of

the full Jockers paper, I suggest that they download the file

in the next few days, before I have to remove it from the web.

I've also been asked to reproduce G. Bruce Schaalje's recent

rebuttal of the Jockers paper; but since I have no permission to

create such a web-edition, I've confined my references to his

material to a few short comments, here:

http://solomonspalding.com/Lib/Jock2008.htm#comments

If anybody notices obvious errors or missing important information,

please let me know, and I'll make the necessary changes.

Thank you,

Uncle Dale

Link to comment

Dale,

I think, from my layman's perspective, you have summarized the Jockers studies pretty well, and included the criticisms that have been brought to bear.

In your summary of the Schaalje paper, which I think will be published in the near future if I understand Bruce correctly, you summarized the pca plots but did not include a summary of the enhancements that Bruce made to the Jockers and the results that Bruce obtained when performing tests on the Federalist papers.

Thanks,

Glenn

Link to comment

Dale,

I think, from my layman's perspective, you have summarized the Jockers studies pretty well, and included the criticisms that have been brought to bear.

In your summary of the Schaalje paper, which I think will be published in the near future if I understand Bruce correctly, you summarized the pca plots but did not include a summary of the enhancements that Bruce made to the Jockers and the results that Bruce obtained when performing tests on the Federalist papers.

Thanks,

Glenn

If you'd like to type up a couple of paragraphs on that subject,

I'd be happy to add them into the comments section.

Archive.org had a copy of Bruce's paper on-line, but it seems

to have disappeared -- so I don't currently have access to the text.

UD

Link to comment

If you'd like to type up a couple of paragraphs on that subject,

I'd be happy to add them into the comments section.

Archive.org had a copy of Bruce's paper on-line, but it seems

to have disappeared -- so I don't currently have access to the text.

UD

Dale, here is the data. I am laying it out pretty much as Bruce did with no commentary.

Author Candidates were Joseph Smith, Sidney Rigdon (1831-1846), Sidney Rigdon (1863-1873), Solomon Spalding, Oliver Cowdery, and Parley P. Pratt using the NSC methods from the Jockers study. The tests were run on the fifty-one papers known to be authored by Alexander Hamilton without Hamilton included as a possible author.

Results:

Sidney Rigdon - 28 of 51 with posterior probabilities ranging as high as 0.9999

Parley P. Pratt - 12

Olivery Cowdery - 11

Joseph Smith Jr. - 0

Solomon Spalding - 0

Bruce added a "latent author" variable to test for the possibility that none of the candidate authors were actually the true author of the text. The math is beyond me but to quote from Bruce's paper "We propose a latent author with a distribution of literary features just barely

consistent with the new text."

Using the latent author method and the same candidate author set, two of the papers were attributed to Rigdon and the rest to the latent author.

A third experiment was run, this time using Hamilton as an author candidate, using twenty-five of the 51 papers as a training text for Hamilton and using the remaining twenty-six texts for the actual tests.

Hamilton was identified as the author of all of those texts.

Please note that this information was taken from a draft preprint of the paper. Bruce has polished it up because he said that "there has been some movement" on the publication of the paper, so we would expect some differences in the final product. Bruce did not indicate that there were any changes to the conclusions and data.

From what I can understand of the graphs that Bruce has produced on his paper, the two texts that Rigdon scored on against the latent author were included in the twenty-six final texts using Hamilton as an additional candidate.

That is the only conjecture that I will make.

Thanks,

Glenn

Link to comment

Dale, here is the data. I am laying it out pretty much as Bruce did with no commentary.

Author Candidates were Joseph Smith, Sidney Rigdon (1831-1846), Sidney Rigdon (1863-1873), Solomon Spalding, Oliver Cowdery, and Parley P. Pratt using the NSC methods from the Jockers study. The tests were run on the fifty-one papers known to be authored by Alexander Hamilton without Hamilton included as a possible author.

Results:

Sidney Rigdon - 28 of 51 with posterior probabilities ranging as high as 0.9999

Parley P. Pratt - 12

Olivery Cowdery - 11

Joseph Smith Jr. - 0

Solomon Spalding - 0

Bruce added a "latent author" variable to test for the possibility that none of the candidate authors were actually the true author of the text. The math is beyond me but to quote from Bruce's paper "We propose a latent author with a distribution of literary features just barely

consistent with the new text."

Using the latent author method and the same candidate author set, two of the papers were attributed to Rigdon and the rest to the latent author.

A third experiment was run, this time using Hamilton as an author candidate, using twenty-five of the 51 papers as a training text for Hamilton and using the remaining twenty-six texts for the actual tests.

Hamilton was identified as the author of all of those texts.

Please note that this information was taken from a draft preprint of the paper. Bruce has polished it up because he said that "there has been some movement" on the publication of the paper, so we would expect some differences in the final product. Bruce did not indicate that there were any changes to the conclusions and data.

From what I can understand of the graphs that Bruce has produced on his paper, the two texts that Rigdon scored on against the latent author were included in the twenty-six final texts using Hamilton as an additional candidate.

That is the only conjecture that I will make.

Thanks,

Glenn

Looks good, Glenn --

I've added those comments here:

http://solomonspalding.com/Lib/Jock2008.htm#comments5

I've also begun to shorten the web-page contents in order

to comply with copyright restrictions; so several pages

have already been removed.

UD

Link to comment

Looks good, Glenn --

I've added those comments here:

http://solomonspalding.com/Lib/Jock2008.htm#comments5

I've also begun to shorten the web-page contents in order

to comply with copyright restrictions; so several pages

have already been removed.

UD

I think that is as far as we can go right now until Bruce's paper is published and the experts weigh in on it. That should be interesting and hopefully there will be some informed consensus.

Glenn

Link to comment

Using the latent author method and the same candidate author set, two of the papers were attributed to Rigdon and the rest to the latent author.

A third experiment was run, this time using Hamilton as an author candidate, using twenty-five of the 51 papers as a training text for Hamilton and using the remaining twenty-six texts for the actual tests.

Hamilton was identified as the author of all of those texts.

Thanks,

Glenn

So this means that Hamiliton could have written the book of mormon? Was he still alive at tbat date? He could have done it if he were still alive. By why would he include JS in the plot? JS was just a little beneath him in intellect. :P

Link to comment

So this means that Hamiliton could have written the book of mormon? Was he still alive at tbat date? He could have done it if he were still alive. By why would he include JS in the plot? JS was just a little beneath him in intellect. :P

I assume that you are writing a bit tongue in cheek here?

The texts had nothing to do with the Book of Mormon. They were all Federalist Papers known to have been written by Hamilton. This was a test of the Jockers NSC methods applied to the Federalist papers.

The Federalist papers have been used by different people to test different authorship attribution techniques because the provenance of most of the papers are known, i.e. authored by either James madison or Alexander Hamilton.

Glenn

Link to comment

...

The Federalist papers have been used by different people to test different authorship

attribution techniques because the provenance of most of the papers are known,

i.e. authored by either James Madison or Alexander Hamilton.

Glenn

But there is an extended purpose in Bruce's citing this example, and that is

because Jockers himself has tested the NSC classification methodology against

Federalist papers authorship and that study has been received by statisticians

and historical scholars as valid.

The leap in application Jockers has attempted, is to bring NSC methods over

to a set of texts which has disputed authorship, and the set of known authors

is also disputed. He justifies his experimental application of these methods

in the case of Book of Mormon chapter attribution, based upon at least three

points -- #1: The method accurately attributes Isaiah and Malachi to the BoM's

Isaiah and Malachi (albeit it with a few false positives) -- #2: The method

accurately excludes known non-BoM authors (Barlow and Longfellow) from the

attributions (albeit it with a few false positives) -- #3: The method produces

authorship attributions for Spalding and Rigdon, which in some cases are so

strong as to be statistically impossible, if credited to coincidence. To these

three points might be added one more: #4: The NSC results generally correspond

to independent "Delta" methods also applied to the Book of Mormon chapters.

Whether or not Jockers' transfer of a computerized methodology recognized as

applicable to the Federalist Papers, over to study of the Book of Mormon,

will be supported by independent, disinterested experts, remains to be seen.

But we might also consider what would happen if Joseph Smith were added to

the mix -- and his 1830 BoM "Preface" were added to the computerized testing.

Jockers has updated his study recently, by adding Smith's word-print to the

testing, but he did not add in Smith's 1830 "Preface." If that text was indeed

composed by Smith, then he would be among the author-candidates for the BoM

as a whole, and should be "testable."

An accurate, valid application of NSC classification methods to the BoM,

using word-prints for Smith, Isaiah and Malachi, should produce results

showing that Smith wrote the 1830 "Preface;" that Isaiah and Malachi wrote

the BoM chapters independently attributable to them --- and that none of

the three authors wrote any of the remaining BoM chapters.

That is, if the Mormons are right, in saying that Smith only wrote the "Preface."

But there is a possible "fly in the ointment" with my latter suggestion.

Perhaps NSC classification testing is only appropriate when ALL of the

author-candidates are among the list tested for authorship by that method.

If this is the case, then simply applying the word-prints of Smith, Isaiah

and Malachi to the BoM would be useless ----- unless we also tested the

text for the word-prints of Lehi, Nephi, Mormon, Moroni, Ether, Helaman, etc.

I'd be curious to hear what the LDS statisticians have to say about this.

What if the actual author-candidate list for the Federalist Papers is

one day shown to include Thomas Jefferson? What if we found out, for

certain, that Jefferson composed two of the papers? Would such a discovery

render past NSC testing invalid -- because one of the actual authors had

not been included in the study?

Or, so long as SOME of the actual writers' word-prints are included in

the computerized testing, is the NSC method robust and flexible enough

to turn out "none of the above" percentage results for texts NOT

composed by those SOME?

Before Jockers can be fully relied upon, these are the sorts of

issues that the experts will have to resolve. And I doubt very seriously

that the controversy will be conclusively decided in the pages of

BYU Studies or in Maxwell Institute publications. The issue will

be resolved by an eventual scholarly consensus in the peer-reviewed,

non-sectarian professional literature; or it will not be resolved at all.

UD

Link to comment

But there is an extended purpose in Bruce's citing this example, and that is

because Jockers himself has tested the NSC classification methodology against

Federalist papers authorship and that study has been received by statisticians

and historical scholars as valid.

The leap in application Jockers has attempted, is to bring NSC methods over

to a set of texts which has disputed authorship, and the set of known authors

is also disputed. He justifies his experimental application of these methods

in the case of Book of Mormon chapter attribution, based upon at least three

points -- #1: The method accurately attributes Isaiah and Malachi to the BoM's

Isaiah and Malachi (albeit it with a few false positives) -- #2: The method

accurately excludes known non-BoM authors (Barlow and Longfellow) from the

attributions (albeit it with a few false positives) -- #3: The method produces

authorship attributions for Spalding and Rigdon, which in some cases are so

strong as to be statistically impossible, if credited to coincidence. To these

three points might be added one more: #4: The NSC results generally correspond

to independent "Delta" methods also applied to the Book of Mormon chapters.

I do not know what methodology that Jockers et al used in their Federalist papers experiments. Did they use the same candidate set that they did for the Book of Mormon? Did they also use a candidate set that excluded the known authors but included the Book of Mormon candidate set?

I do not understand the "statistically impossible" attributions assertion when all of the attribution results are relative to the the other included candidates.

When Bruce excluded Hamilton as an author candidate for the 51 papers he is known to have authored, the authorship attribution results were similar to some of the results that the Jockers study obtained. The exception was for Spalding. He had zero texts attributed to him.

Rigdon had the most attributed to him, 28, "with posterior probabilities as high as .9999.

However, when Hamilton was added to the mix, none of the Federalist papers were attributed to Rigdon, or anyone else but Hamilton, which would seem to indicate that it is important to have the real author included in any candidate set.

That is why Bruce made a few enhancements to the Jockers NSC methodology to ensure the results will not be skewed to the point of being unreliable if the actual author of a text is not included. There will probably be some enhancements to his "latent author" algorithm, but it shows some promise.

I am going to quote a few sentences from Bruce's draft detailing some of the problems that Bruce saw with the Jockers study. And remember, this is from the draft, not the final edition that will be published.

"While NSC and related methodologies have much to offer to stylometric analysis, the recent use of NSC in a study of Book of Mormon authorship (Jockers et al. 2008) is problematic. The training data set was assumed to contain writings of the true author, goodness-of-fit of the procedure was not checked, posterior authorship probabilities of the test texts did not reflect text size, and multiplicity was ignored when interpreting the results. These issues would not usually be a problem in genomics investigations using NSC, but they are important issues in most stylometric problems."

Whether or not Jockers' transfer of a computerized methodology recognized as

applicable to the Federalist Papers, over to study of the Book of Mormon,

will be supported by independent, disinterested experts, remains to be seen.

I hope that it will not be another year before Bruce's paper is published. Then we can let the professionals duke it out and see the lay of the land after the dust settles.

Glenn

Link to comment

...

I hope that it will not be another year before Bruce's paper is published. Then we can

let the professionals duke it out and see the lay of the land after the dust settles.

Glenn

I don't have any special information on this subject, other than what

everybody else has heard. I do think, however, that not EVERY SINGLE

writer of a controversial body of literature need be known in advance,

before that text can be tested for probable authorship. I think that if

Smith did indeed write the "Preface" to the 1830 BoM (and no other part)

then there is probably some way of demonstrating that -- even if we do

not add Moroni, Nephi, etc., to the list of author-candidates.

As for the future -- some other statisticians are already citing the

2008 Jockers paper as providing some degree of reliable method/conclusions.

Until we see some more publications, that's about all we have to go on

in the professional literature right now.

UD

Link to comment

If anybody notices obvious errors or missing important information,

please let me know, and I'll make the necessary changes.

Thank you,

Uncle Dale

Well, I don't know about that Uncle Dale. But what I do know is that your former mormon turned catholic friend contacted you for some information and he posted it on a different board. Some catholics are now celebrating that the book of mormon is a fraud and the study proves it. Do you think that they are partying just a little too early? Interested to know.

Link to comment

Well, I don't know about that Uncle Dale. But what I do know is that your former mormon turned catholic friend contacted you for some information and he posted it on a different board. Some catholics are now celebrating that the book of mormon is a fraud and the study proves it. Do you think that they are partying just a little too early? Interested to know.

Actually, the lady you refer to comes from several generations of

Catholics, some of whom were Christians before the time of

George Washington (or so I'm told). She has spent some time near

Nauvoo, and knows something about Mormons, since one of her ancestors

was in the anti-Mormon Hancock Co. Militia (if I recall correctly).

I've never known a Mormon who decided to follow after the Bishop of Rome --

though I'm sure there have been such apostates, somewhere and somewhen.

I see the Jockers' report as something like a fellow reaching his hand

out the window, and telling people that it's raining outside.

Another person comes along and disagrees, saying that the moisture

thus felt came from some other source and it isn't raining at all.

Yet another person comes along and professes that sticking a hand

out a window to check for rain is not a reliable method, etc. etc.

Perhaps it is raining -- perhaps it is not raining. If enough people

actually take the trouble to go outside and check for themselves,

then perhaps we can reach a consensus on the controversy.

In Jockers' case, some additional capable people need to check out

his methodology and his results. Until that happens, the best we can

say is that he remains unchallenged in the professional forum where

he published his report. And, in fact, is being cited as a reliable

source by other scientists in the field.

For those readers who are convinced by Jockers' arguments, the lack

of any professional rebuttal, after many months, speaks for itself.

Uncle Dale

.

Link to comment

For those readers who are convinced by Jockers' arguments, the lack

of any professional rebuttal, after many months, speaks for itself.

Uncle Dale

.

I don't think that most people care one way or another...at least not those who don't have a stack in the issue. Of course, we have Dan telling us that a rebuttal is on its way. I would think that the lds will be the only ones offering a rebuttal. And I am sure that it will be a good one. Not that it will make a difference.

Link to comment

...we have Dan telling us that a rebuttal is on its way.

...

Dan Vogel?

I would think that he would wait until Jockers publishes his

updated Bom authorship report, including Joseph Smith's word-print.

Or, perhaps you mean Dan Vogel's supporter, Chris Smith? His

rebuttal is on-line here

Or, perhaps you mean Dan Peterson is going to publish something

by a Mormon author in a Maxwell Institute journal?

That will be practically meaningless and will receive no professional

attention outside of the LDS cloister.

I can write a paper on the true origin of L. Ron Hubbard's stuff,

but if the only rebuttal it ever receives comes from Scientology

itself, the rebuttal will be practically meaningless.

I can write a paper on the true origin of Rev. Sun Moon's sermons,

but if the only rebuttal it ever receives comes from the Unification

Church itself, the rebuttal will be practically meaningless. Even an

article in the "Washington Times" would be practically meaningless.

If LDS statisticians intend on joining a professional dialogue about

Jockers' work, they'll have to do so in the non-sectarian professional

literature, and go through proper peer-review, without Maxwell Institute

editors holding their hands.

UD

Link to comment
Uncle Dale, on 28 August 2010 - 02:32 PM, said:

For those readers who are convinced by Jockers' arguments, the lack

of any professional rebuttal, after many months, speaks for itself.

Uncle Dale

If LDS statisticians intend on joining a professional dialogue about

Jockers' work, they'll have to do so in the non-sectarian professional

literature, and go through proper peer-review, without Maxwell Institute

editors holding their hands.

UD

Dale,

I am sure you are aware that Bruce Schaalje has presented a paper for publication critiquing the Jockers study. It was presented to the Literary and Linguistic Computing Magazine in April of 2009. It has yet to be accepted for publication, although Bruce did say a couple of weeks ago that there is now some movement on it. I have not been informed as to why the delay nor when we can expect to see the actual publication.

Just to note, Bruce has authored and coauthored other publications which have been had favorable peer reviews, so it is not as if he is a tyro in statistics or publishing.

In other words, there has been a professional rebuttal. Those who are convinced by the Jockers study arguments have not even attempted to engage the points that Bruce made. Some of the math is beyond most of us, but the fact that when Bruce ran the unmodified NSC algorithms on the Federalist papers known to be authored by Hamilton without including Hamilton as an author candidate, Rigdon came up as the probable author for 28 of those 51 papers with some very high posterior probabilities. However, when Hamilton was included as an author candidate, he was correctly chosen every time.

Those facts speak for themselves.

Glenn

Link to comment

...

there has been a professional rebuttal

...

And, when it finally reaches the literature, it will be the beginning

of a discussion -- perhaps even a conversation and a dialogue.

Nobody other than Mormons will read a response in the Maxwell Institute

publications. Sorry about that, but it's true.

If the response is meant primarily for faith promotion, then I suppose

it's OK. If it is meant as a means to engage other scholars in a

scientific inquiry, then there will be a problem.

Where I see the real problem arising for Mormons, is if Jockers' new

use of NSC classification for texts with contested authorship lists

is picked up and applied by other scientists in other authorship studies.

Today there is only Jockers to deal with. In a year or two we may see

half a dozen statisticians using Jockers' methods for purposes entirely

unrelated to the LDS "restoration scriptures."

A rebuttal paper in BYU Studies (which is probably more widely

read than Maxwell Institute articles), may generate a bit of interest,

so long as its contents are focused upon LDS-related topics. But who

on earth will read a Provo journal, when it comes to second-generation

topics relating to Jockers' methods, but NOT to Mormon topics.

If the LDS scholars are going to nip this problem in the bud, the place

to do that is out in the general public, far away from BYU and from Utah.

I wonder if anybody has given that any thought yet?

UD

Link to comment

If the LDS scholars are going to nip this problem in the bud, the place

to do that is out in the general public, far away from BYU and from Utah.

I wonder if anybody has given that any thought yet?

UD

I'm sort of scratching my head here. I thought that I pointed out in the post you just snipped from, that the problem is being addressed, and will be in the same venue that the Jockers Study was published.

As I mentioned before, once that happens and the dust settles, we will see better how things lie.

Glenn

Link to comment

I'm sort of scratching my head here. I thought that I pointed out in the post you just snipped from, that the problem is being addressed, and will be in the same venue that the Jockers Study was published.

As I mentioned before, once that happens and the dust settles, we will see better how things lie.

Glenn

You can check with Bruce to find out for certain,

but I think that the "movement" in his paper's

eventual publication has not come from the reviewers

at the computing journal. Last I heard, he was also

submitting it for publication in Provo, and had

submitted a more arcane mathematical paper to a

stats journal, which is not the pre-pub we've read.

As you say -- we shall see.

UD

Link to comment
Readers of these comments should keep in mind that the 2008 Jockers paper identifies Solomon Spalding, Sidney Rigdon, Oliver Cowdery and Parley P. Pratt as contributors to the Book of Mormon, only in terms of relative probability. If more author-candidates were appended to Jockers' list, the new additions would likely score at least a few "hits," and thus reduce the probability percentages already obtained for the original author-candidates. If the "true writers" of the Book of Mormon have been somehow left off of Jockers' list, then of course they would score no probability percentages at all. It is also possible that no "true writers" of the book were included among Jockers' selection of 19th century writers, and that no matter what probability percentages were assigned in the 2008 paper, all those authorship figures are meaningless in the real world.

So...back to square one for modern authorship proponents. Especially since we're dealing with a translation, why would it be suprising to find a "true" modern author anyway? That would not speak to false claims on the part of JS & Co.

Link to comment

I just hope that the rebuttal comes soon. Of course, it will not matter for many critics who wish the lds church to fall. But this study is being paraded around on critic sites as an end all end all for the lds faith. The rebuttal should come sooner than later.

Link to comment

I just hope that the rebuttal comes soon. Of course, it will not matter for many critics who wish the lds church to fall. But this study is being paraded around on critic sites as an end all end all for the lds faith. The rebuttal should come sooner than later.

The wait is worth it. Once it's out there, this will all have seemed a very temporary issue.

Ben M.

Link to comment
why me, on 29 August 2010 - 09:58 AM, said:

I just hope that the rebuttal comes soon. Of course, it will not matter for many critics who wish the lds church to fall. But this study is being paraded around on critic sites as an end all end all for the lds faith. The rebuttal should come sooner than later.

The wait is worth it. Once it's out there, this will all have seemed a very temporary issue.

Ben M.

Is the rebuttal the one that Bruce Schaalje presented for publication. The one that the Literary and Linguistic Computing periodical has been dragging its feet on? Or is there another in addition?

Thanks,

Glenn

Link to comment

The wait is worth it. Once it's out there, this will all have seemed a very temporary issue.

Ben M.

I am glad to hear it. But I am afraid the damage has been done. With the Internet the critics will keep dragging out this study to prove their point. And of course, there will be no rebuttal included when they pull it out.

If it is a good rebuttal, I think that FAIR should put it in their wiki and in their regular website and Dale should include it in his website.

Link to comment

Archived

This topic is now archived and is closed to further replies.

Guest
This topic is now closed to further replies.
  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...