Giving ChatGPT the "write the BoM challenge"

[LoudmouthMormon]

I wish I could say this was my idea, but someone beat me to it.  Pretty cool though.

Request:

Quote

Write a book with a minimum of 250,000 words about a society which spans 1,000 year history. The culture, politics, and religion must gradually alter with generations of main characters being replaced with successive generations over time. The culture & history must include scientific advancements, interactions with deity, archaeological discoveries, wars between nations, political structure, and a unique monetary system which is based in precious metals that would be practical in a market economy. The book must also have at least 100 proper nouns which have not been commonly used in the modern world for the past 100 years.

 

ChatGPT's answer:

Quote

I am sorry, as an AI language model, I am capable of generating high-quality texts, but writing a book with a minimum of 250,000 words is beyond my capacity. It would take significant time and effort to create such a complex work that requires attention to detail and coherence over a thousand-year timeline. However, I can provide you with a brief outline of the story to give you an idea of what it might look like.

Title: The Shifting Sands of Time: A Thousand-Year Odyssey

Prologue: The First Age (1-100 years)

The story begins in a primitive society, where the people live in small tribes and worship a pantheon of gods. The tribes are constantly at war with each other over territory, resources, and cultural differences. The protagonist, a young warrior named Kellor, emerges as a leader of his tribe and seeks to unite the warring factions under a common banner. He makes alliances with neighboring tribes and persuades them to follow his vision of a more civilized society.

Chapter 1: The Second Age (101-200 years)

Kellor establishes a confederation of tribes and becomes the first king of a united people. He introduces a code of laws, a system of taxation, and a standing army to maintain order and defend against external threats. He also encourages scientific inquiry and sponsors research into agriculture, metallurgy, and medicine. However, Kellor's reforms are not universally accepted, and there are still factions that resist his authority.

Chapter 2: The Third Age (201-300 years)

Kellor's successors continue to build upon his legacy and expand the kingdom's borders through diplomacy and conquest. They establish trade routes with other civilizations and adopt new technologies from abroad. The kingdom becomes a center of commerce and culture, attracting scholars, artists, and artisans from all over the world. The capital city, once a small village, grows into a metropolis, with grand palaces, temples, and public works.

Chapter 3: The Fourth Age (301-400 years)

As the kingdom grows in wealth and power, it becomes increasingly stratified, with a ruling elite and a vast underclass of peasants and slaves. The religious institutions become corrupt and exploitative, using their influence to extract tribute from the people and suppress dissent. A new movement arises, advocating for social justice and equality, and challenging the legitimacy of the monarchy. The movement gains momentum, and there are violent clashes between the rebels and the royal army.

Chapter 4: The Fifth Age (401-500 years)

The rebellion succeeds in overthrowing the monarchy and establishing a republic. The new government is based on the principles of democracy and individual rights, and it promises to eradicate poverty and oppression. However, the republic is soon plagued by factionalism, corruption, and ineffectiveness. The economy suffers, and there are famines, riots, and civil unrest. The republic collapses under the weight of its own contradictions, and a new era of chaos and anarchy begins.

Chapter 5: The Sixth Age (501-600 years)

Out of the ruins of the republic emerges a charismatic leader, who claims to have divine inspiration and a vision of a new world order. He rallies the people behind him, promising to restore order, morality, and prosperity. He establishes a theocracy, with himself as the supreme ruler and the religious hierarchy as his loyal servants. The theocracy is highly centralized and authoritarian, but it is also efficient and effective, capable of mobilizing resources and suppressing dissent. The leader becomes a cult figure, worshipped as a god by his followers.

Chapter 6: The Seventh Age (601-700 years)

The theocracy reaches its zenith, with a vast empire spanning continents and oceans. The empire is highly militarized, with a

[network error]

 

[InCognitus]

This is a lot like the Solomon Spaulding and View of the Hebrews theory for the origin of the Book of Mormon with the same results:   [network error]

[pogi]

3 hours ago, LoudmouthMormon said:

I wish I could say this was my idea, but someone beat me to it.  Pretty cool though.

Request:

 

ChatGPT's answer:

 

Is this supposed to be evidence that a human couldn't create such a novel? 

Don't get me wrong, I think the Book of Mormon is inspired, but if you asked it to create any book with 250,000 words it is incapable, yet we have thousands of complex novels created by humans that accomplish that task easy enough.   It is still infant technology.  It'll get there. 

[LoudmouthMormon]

16 minutes ago, pogi said:

Is this supposed to be evidence that a human couldn't create such a novel? 

This is supposed to be an old discarded LDS apologetic, given new life through modern technology.  It's retro kitsch with a layer of ironic meta.  You know, stuff millennials like.  

[smac97]

8 minutes ago, pogi said:

Is this supposed to be evidence that a human couldn't create such a novel? 

I think it is evidence which acknowledges the length, complexity, details, internal coherence, etc. of the text of the Book of Mormon.

I think it is evidence less about "could or could not" relative to Joseph Smith, as that's a bit too binary.  Instead, it may be evidence relevant to "more plausible or less plausible" as to 19th-century origins, particularly given A) Joseph's education and writing skills, B) the information available in Joseph Smith's time, C) the short timeframe of the production of the translated text, D) the seemingly total absence of any evidence of collusion (despite ample incentives for Joseph's contemporaries and subsequent critics to find such evidence), and E) the absence of any coherent counter-explanation for the origins of the text.

8 minutes ago, pogi said:

Don't get me wrong, I think the Book of Mormon is inspired, but if you asked it to create any book with 250,000 words it is incapable, yet we have thousands of complex novels created by humans that accomplish that task easy enough.

I don't think this is an apt description of the "lots of people could do what Joseph did" explanation relative to the creation of the text.  See, e.g., John Welch's "The Miraculous Translation of the Book of Mormon."  An excerpt:

Quote

Numerous approaches can and should be taken in approaching the Book of Mormon. This complex book has been read and scrutinized in many ways: textually, doctrinally, historically, comparatively, literarily, legally, statistically, geographically, philosophically, practically, biographically, intellectually, prayerfully, and spiritually—to name some of the most obvious. The richness of this book inevitably invites several questions: How was this book written? Where did it come from? Joseph Smith testified that he translated the Book of Mormon miraculously, by the gift and power of God. Is that testimony credible?

The following set of over two hundred documents assembles data pertinent to that ultimate question. In particular, from these contemporaneous historical records, this study seeks to determine, as precisely as possible, when the Book of Mormon was translated and how long it took to complete this impressive task. The overwhelming accumulation of the consistent historical details provided by eyewitness participants and local observers leads to the solid conclusion that the Book of Mormon was translated in a very short period of time. Inside of three astonishingly compressed months, Joseph Smith produced the Book of Mormon. Its text simply emerged as it fell from his lips, line after line, recorded by his attentive scribe. The rapidity of the translation left no time for steps normally taken in producing translations.
...
In sum, it is shown that nearly all the 590 pages printed in the 1830 edition of the Book of Mormon were translated, dictated, and written within an extremely short and intensely busy period of time, from April 7 to the last week of June 1829. Virtually no excess time existed during those three months for Joseph Smith to plan, to ponder about, to research, to hunt for sources, to organize data, to draft, to revise, or to polish the pages of the original manuscript of this book. Although Joseph became aware of and began contemplating this assignment in September 1823, and while he translated the 116 pages containing the book of Lehi from April 12, 1828, to June 14, 1828, which were sadly lost that summer, once Joseph and Oliver set to work on April 7, 1829, the pages of the Book of Mormon flowed forth in rapid succession. The text of the Book of Mormon was dictated one time through, essentially in final form. This was done despite significant interruptions and distractions. Such a feat, in and of itself, constitutes a considerable achievement, given the length, quality, and complexity of the Book of Mormon alone.

Reproducibility and replicability "are often cited as hallmarks of good science."  We are not really situated to "reproduce" what Joseph did, as we don't have the Plates, the "gift and power of God," and so on.

But it seems that we do have the ability to reproduce/replicate "naturalistic" alternative theories as to the origins of the text of the Book of Mormon.  And yet, nobody has been able to do it.  To produce a comparably complex and lengthy and internally consistent text "within an extremely short and intensely busy period of time" of about two months.  To do so verbally ("Its text simply emerged as it fell from his lips, line after line, recorded by his attentive scribe").  To do so with no substantive backtracking, re-wording, revisions, etc.

8 minutes ago, pogi said:

It is still infant technology.  It'll get there. 

You may well be right.  See here: In January, ChatGPT Failed The Bar. In March, GPT-4 Exceeds The Nationwide Student Average

Quote

In 2011, Apple introduced Siri. This voice recognition system was designed as an ever-present digital assistant, that could help you with anything, anytime, anywhere. In 2014, Amazon introduced Alexa, which was designed to serve a similar purpose. Nearly a decade later, neither product has ever reached its potential. They are mostly niche tools that are used for very discrete purposes. Today's New York Times explains how Siri, Alexa, as well as Google Assistant lost the A.I. race to tools like GPT. Now, we have another notch in the belt of OpenAI's groundbreaking technology.

Yesterday, OpenAI released GPT-4. To demonstrate how powerful this tool is, the company allowed a number of experts to take the system for a spin. In the legal corner were Daniel Martin Katz, Mike Bommarito, Shang Gao, and Pablo Arredondo. In January 2023, Katz and Bommarito studied whether GPT-3.5 could pass the bar. At that time, the AI tech achieved an overall accuracy rate of about 50%.

In their paper, the authors concluded that GPT-4 may pass the bar "within the next 0-18 months." The low-end of their estimate proved to be accurate.

Fast-forward to today. Beware the Ides of March. Katz, Bommarito, Gao, and Arredondo posted a new paper to SSRN, titled "GPT-4 Passes the Bar Exam." Here is the abstract:

In this paper, we experimentally evaluate the zero-shot performance of a preliminary version of GPT-4 against prior generations of GPT on the entire Uniform Bar Examination (UBE), including not only the multiple-choice Multistate Bar Examination (MBE), but also the open-ended Multistate Essay Exam (MEE) and Multistate Performance Test (MPT) components. On the MBE, GPT-4 significantly outperforms both human test-takers and prior models, demonstrating a 26% increase over ChatGPT and beating humans in five of seven subject areas. On the MEE and MPT, which have not previously been evaluated by scholars, GPT-4 scores an average of 4.2/6.0 as compared to much lower scores for ChatGPT. Graded across the UBE components, in the manner in which a human tast-taker would be, GPT-4 scores approximately 297 points, significantly in excess of the passing threshold for all UBE jurisdictions. These findings document not just the rapid and remarkable advance of large language model performance generally, but also the potential for such models to support the delivery of legal services in society.

Figure 1 puts this revolution in stark contrast:

Two months ago, an earlier version of GPT was at the 50% mark. Now, GPT-4 exceeded the 75% mark, and exceeds the student average performance nationwide. GPT-4 would place in the 90th percentile of bar takers nationwide!

And GPT scored well across the board. Evidence is north of 85%, and GPT-4 scored nearly 70% in ConLaw!

We should all think very carefully how this tool will affect the future of legal services, and what we are teaching to our students.

Wow.  Wow.

More here:

Quote

• ABA Journal, Latest Version of ChatGPT Aces Bar Exam With Score Nearing 90th Percentile
• Josh Blackman (South Texas), In January, ChatGPT Failed The Bar. In March, GPT-4 Exceeds The Nationwide Student Average
• Casetext Press Release, CoCounsel Builds on the Power of GPT-4, the AI That Outperformed Real Bar Candidates
• Chicago-Kent Press Release, GPT-4 Passes the Bar Exam
• Reuters, Bar Exam Score Shows AI Can Keep Up With 'Human Lawyers,' Researchers Say

A brief description of ChatGPT:

Quote

ChatGPT is a member of the generative pre-trained transformer (GPT) family of language models. It was fine-tuned (an approach to transfer learning^[6]) over an improved version of OpenAI's GPT-3 known as "GPT 3.5".^[7] The fine-tuning process leveraged both supervised learning as well as reinforcement learning in a process called reinforcement learning from human feedback (RLHF).^[8][9] Both approaches used human trainers to improve the model's performance. In the case of supervised learning, the model was provided with conversations in which the trainers played both sides: the user and the AI assistant. In the reinforcement learning step, human trainers first ranked responses that the model had created in a previous conversation.^[10] These rankings were used to create 'reward models' that the model was further fine-tuned on using several iterations of Proximal Policy Optimization (PPO).^[8][11] Proximal Policy Optimization algorithms present a cost-effective benefit to trust region policy optimization algorithms; they negate many of the computationally expensive operations with faster performance.^[12][13] The models were trained in collaboration with Microsoft on their Azure supercomputing infrastructure, using Nvidia GPUs, "supercomputer developed for OpenAI is a single system with more than 285,000 CPU cores, 10,000 GPUs and 400 gigabits per second of network connectivity for each GPU server".^[14]

In addition, OpenAI continues to gather data from ChatGPT users that could be used to further train and fine-tune ChatGPT. Users can upvote or downvote responses they receive from ChatGPT and fill out a text field with additional feedback.^[15][16]

"The models were trained in collaboration with Microsoft on their Azure supercomputing infrastructure, using Nvidia GPUs, 'supercomputer developed for OpenAI is a single system with more than 285,000 CPU cores, 10,000 GPUs and 400 gigabits per second of network connectivity for each GPU server.'"

If and when we finally get to the point where a computer can generate a comparable text, then what?  If we look at what Joseph Smith managed to accomplish in around two months in 1829, and if we finally get a comparable text written 200 years later by an AI trained using "a single system with more than 285,000 CPU cores, 10,000 GPUs and 400 gigabits per second of network connectivity for each GPU server," then might that not be evidence that what Joseph did was pretty remarkable?  Perhaps even beyond his 19th-century capabilities?  Again from Jack Welch:

Quote

The translation of the Book of Mormon, completed by Joseph Smith in June 1829, was an amazing feat. By any standard, this 588-page holy book is extraordinary. Isaiah’s words, “a marvelous work and a wonder” (Isa. 29:14), which can be translated from the Hebrew more literally as “a miraculous work and a miracle,” readily describe the coming forth of this key restoration text.

Divine manifestations of several kinds directed the rapid course of the translation. Through angelic ministrations, the gift of powers to translate, the guidance of visions, and in many other ways, the hand of God was evident in the truly astounding work of bringing forth the ancient Nephite record. 

Thanks,

-Smac

[pogi]

29 minutes ago, smac97 said:

I think it is evidence which acknowledges the length, complexity, details, internal coherence, etc. of the text of the Book of Mormon.

I can accept that.

29 minutes ago, smac97 said:

I don't think this is an apt description of the "lots of people could do what Joseph did" explanation relative to the creation of the text. 

I'm not sure who you are quoting here, but I  never said or implied that. 

[smac97]

3 minutes ago, pogi said:

Quote

I think it is evidence which acknowledges the length, complexity, details, internal coherence, etc. of the text of the Book of Mormon.

I can accept that.

Quote

Quote

Don't get me wrong, I think the Book of Mormon is inspired, but if you asked it to create any book with 250,000 words it is incapable, yet we have thousands of complex novels created by humans that accomplish that task easy enough.

I don't think this is an apt description of the "lots of people could do what Joseph did" explanation relative to the creation of the text. 

I'm not sure who you are quoting here, but I  never said or implied that. 

I took your statement ("we have thousands of complex novels created by humans that accomplish that task easy enough") as referencing the "lots of people could do what Joseph did" explanations which have been bandied about over the years.  As I was apparently in error, I apologize.

Thanks,

-Smac

[pogi]

14 minutes ago, smac97 said:

I took your statement ("we have thousands of complex novels created by humans that accomplish that task easy enough") as referencing the "lots of people could do what Joseph did" explanations which have been bandied about over the years.  As I was apparently in error, I apologize.

Thanks,

-Smac

Thanks for apologizing. 

I stated that I believe it is inspired.   I don't think it could have been written it without the inspiration of God.   

What I was pointing out is that ChatgPT can't even replicate a novel, "any" novel of that length, which thousands of people can and have done.  So, it doesn't say much in that regards. 

Edited March 17 by pogi

[smac97]

3 minutes ago, pogi said:

Quote

I took your statement ("we have thousands of complex novels created by humans that accomplish that task easy enough") as referencing the "lots of people could do what Joseph did" explanations which have been bandied about over the years.  As I was apparently in error, I apologize.

Thanks for apologizing. 

I stated that I believe it is inspired.   I don't think it could have been written it without the inspiration of God.   

I saw that.  I was not attributing the "lots of people could do what Joseph did" line of reasoning to you, or I did not intend to.  Hence my apology.

3 minutes ago, pogi said:

What I was pointing out is that ChatgPT can't even replicate a novel, "any" novel of that length, which thousands of people can and have done.  So, it doesn't say much in that regards. 

The rate of improvement in ChatGPT appears to nigh unto exponential.  From my previous post:

Each of the above bars represents 1 annual quarter, or three months.  For ChatGPT to go from scoring <10% on the MBE to ~75% in 7 quarters, or less than two years, is pretty astonishing.

So ChatGPT may not be able to write a novel now, but it may be able to do so in pretty short order.

Thanks,

-Smac

[pogi]

16 minutes ago, smac97 said:

So ChatGPT may not be able to write a novel now, but it may be able to do so in pretty short order.

I don't doubt it one bit.  I also don't doubt that it will be able to write a scriptural type book equal in complexity to the BoM.   That won't disprove the Book of Mormon though.  That is why I don't see much point in this.  It just seems like a superficial way to give confidence in the Book of Mormon.  Whether ChatGPT can or cannot create something equivalent in complexity to the Book of Mormon says nothing about the inspired source of the Book of Mormon. 

[smac97]

10 minutes ago, pogi said:

Quote

So ChatGPT may not be able to write a novel now, but it may be able to do so in pretty short order.

I don't doubt it one bit.  I also don't doubt that it will be able to write a scriptural type book equal in complexity to the BoM.   That won't disprove the Book of Mormon though. 

I don't think the point is to "disprove the Book of Mormon," but rather to note that if this happens, just how much effort and technological resources are necessary to match what Joseph Smith accomplished nearly 200 years ago.

10 minutes ago, pogi said:

That is why I don't see much point in this.

I see the point, but I don't see it as being that persuasive of one.

10 minutes ago, pogi said:

It just seems like a superficial way to give confidence in the Book of Mormon. 

It an arrow in the quiver, one of many arguments.  That's all.

10 minutes ago, pogi said:

Whether ChatGPT can or cannot create something equivalent in complexity to the Book of Mormon says nothing about the inspired source of the Book of Mormon. 

I think it might end up saying something about that.  How probative and persuasive that ends up being, though, is very much an "eye of the beholder" thing.

Thanks,

-Smac

[The Nehor]

ChatGPT is a mirage of a person. It is an interesting exercise but it is limited. It doesn’t understand what it is doing. It is a huge cache of data and an algorithm on how to use that data created in a very weird way.

No one understands everything that goes into their decision making including the creators. It is brute force engineering where we keep evolving the algorithm and just keeping the ones that get it right and spinning off new ones. It can pass tests because we fed it tests and only kept the algorithms that did the best.

For fun try to ask ChatGPT to write a report on a book it has never read before. It writes like a High School student who hasn’t read the material complete with trying to use generic comments that apply to everyone. It also gets facts wrong a lot of the time because the data it was using got them wrong and it can’t tell the difference at all.

[LoudmouthMormon]

4 hours ago, smac97 said:

Wow.  Wow.

My father was a printer/proofreader.  He set type and layout for the Salt Lake Tribune and Deseret News.  His father before him was a printer as well.  It was a respected profession going all the way back to Gutenberg days.  Then the entire profession dried up within 5 years after they invented the spell check button and the home printer. 

Makes me wonder what professions are going to be obsoleted in the next 10 years because of this stuff.  Once LawyerGPT V3 is downloadable for ten bucks on sale, totally capable of doing 80% of what lawyers do, better than the average lawyer, what will your profession look like?

[Tacenda]

2 hours ago, LoudmouthMormon said:

My father was a printer/proofreader.  He set type and layout for the Salt Lake Tribune and Deseret News.  His father before him was a printer as well.  It was a respected profession going all the way back to Gutenberg days.  Then the entire profession dried up within 5 years after they invented the spell check button and the home printer. 

Makes me wonder what professions are going to be obsoleted in the next 10 years because of this stuff.  Once LawyerGPT V3 is downloadable for ten bucks on sale, totally capable of doing 80% of what lawyers do, better than the average lawyer, what will your profession look like?

Life is certainly getting more interesting the older I get, wow!

[The Nehor]

We obsoleted much of human physical labor with the Industrial Revolution. If this continues the need for human brains will likewise diminish. It will change differently since there will still be things humans are required for. I might use a lawyer bot to take care of paperwork but I doubt it would take the lawyer out of the courtroom. We might be able to get rid of 80% of doctors when bots learn enough to diagnose problems better than humans can.

We will then either create new industries that require human brains specifically (and create demand for their product somehow) or will have to adapt society since full employment will be impossible or possibly both.

[LoudmouthMormon]

12 hours ago, The Nehor said:

We obsoleted much of human physical labor with the Industrial Revolution. If this continues the need for human brains will likewise diminish.

 

And furthermore, everyone should go watch Idiocracy.  It's the dumbest, most poorly made flick that will give you years of nightmares.

[Kenngo1969]

Hey!  I could use a Coke IV!  Where can I get one of those?!*  

   

*Do I have to go to the nearest ER?  "2L Coke IV, stat! "

[Calm]

56 minutes ago, Kenngo1969 said:

Hey!  I could use a Coke IV!  Where can I get one of those?!*  

   

*Do I have to go to the nearest ER?  "2L Coke IV, stat! "

Because I am in that kind of mood…

A Coke IV is stupid unless you don’t like the taste because it robs you of the pleasure of the sensation of sweetness and carbonation (though I would go for ginger beer for my carbonation fix, yech Coke).  And if you don’t like the taste of Coke or the sensation of bubbles, then why not just eat something sugary that you like (cream puffs would be my go to sugar fix) and pop caffeine pills?

[The Nehor]

20 hours ago, LoudmouthMormon said:

 

And furthermore, everyone should go watch Idiocracy.  It's the dumbest, most poorly made flick that will give you years of nightmares.

I like the movie but the reverse eugenics of the film was just weird. The idea of devolution of intelligence is nonsensical.

 

[LoudmouthMormon]

Honestly, XKCD has eased my worries on numerous occasions.  I'm glad to see it.