Science is Broken -- peer review and replication

[cdowis]

Advocates of the existing scientific research paradigm usually smugly declare that while some published conclusions are surely false, the scientific method has "self-correcting mechanisms" that ensure that, eventually, the truth will prevail. Unfortunately for all of us, Wilson makes a convincing argument that those self-correcting mechanisms are broken.

For starters, there's a "replication crisis" in science. This is particularly true in the field of experimental psychology, where far too many prestigious psychology studies simply can't be reliably replicated. But it's not just psychology. In 2011, the pharmaceutical company Bayer looked at 67 blockbuster drug discovery research findings published in prestigious journals, and found that three-fourths of them weren't right. Another study of cancer research found that only 11 percent of preclinical cancer research could be reproduced. Even in physics, supposedly the hardest and most reliable of all sciences, Wilson points out that "two of the most vaunted physics results of the past few years — the announced discovery of both cosmic inflation and gravitational waves at the BICEP2 experiment in Antarctica, and the supposed discovery of superluminal neutrinos at the Swiss-Italian border — have now been retracted, with far less fanfare than when they were first published."

What explains this? In some cases, human error. Much of the research world exploded in rage and mockery when it was found out that a highly popularized finding by the economists Ken Rogoff and Carmen Reinhardt linking higher public debt to lower growth was due to an Excel error. Steven Levitt, of Freakonomics fame, largely built his career on a paper arguing that abortion led to lower crime rates 20 years later because the aborted babies were disproportionately future criminals. Two economists went through the painstaking work of recoding Levitt's statistical analysis — and found a basic arithmetic error.
<snip>

+++++++++++++++++++++++++++++++++
The peer review process doesn't work. Most observers of science guffaw at the so-called "Sokal affair," where a physicist named Alan Sokal submitted a gibberish paper to an obscure social studies journal, which accepted it. Less famous is a similar hoodwinking of the very prestigious British Medical Journal, to which a paper with eight major errors was submitted. Not a single one of the 221 scientists who reviewed the paper caught all the errors in it, and only 30 percent of reviewers recommended that the paper be rejected. Amazingly, the reviewers who were warned that they were in a study and that the paper might have problems with it found no more flaws than the ones who were in the dark.

This is serious. In the preclinical cancer study mentioned above, the authors note that "some non-reproducible preclinical papers had spawned an entire field, with hundreds of secondary publications that expanded on elements of the original observation, but did not actually seek to confirm or falsify its fundamental basis."

http://theweek.com/articles/618141/big-science-broken

Last summer, the Open Science Collaboration announced that it had tried to replicate one hundred published psychology experiments sampled from three of the most prestigious journals in the field. Scientific claims rest on the idea that experiments repeated under nearly identical conditions ought to yield approximately the same results, but until very recently, very few had bothered to check in a systematic way whether this was actually the case. The OSC was the biggest attempt yet to check a field’s results, and the most shocking. In many cases, they had used original experimental materials, and sometimes even performed the experiments under the guidance of the original researchers. Of the studies that had originally reported positive results, an astonishing 65 percent failed to show statistical significance on replication, and many of the remainder showed greatly reduced effect sizes.

Their findings made the news, and quickly became a club with which to bash the social sciences. But the problem isn’t just with psychology. There’s an ­unspoken rule in the pharmaceutical industry that half of all academic biomedical research will ultimately prove false, and in 2011 a group of researchers at Bayer decided to test it. Looking at sixty-seven recent drug discovery projects based on preclinical cancer biology research, they found that in more than 75 percent of cases the published data did not match up with their in-house attempts to replicate. These were not studies published in fly-by-night oncology journals, but blockbuster research featured in Science, Nature, Cell, and the like. The Bayer researchers were drowning in bad studies, and it was to this, in part, that they attributed the mysteriously declining yields of drug pipelines. Perhaps so many of these new drugs fail to have an effect because the basic research on which their development was based isn’t valid.

When a study fails to replicate, there are two possible interpretations. The first is that, unbeknownst to the investigators, there was a real difference in experimental setup between the original investigation and the failed replication. These are colloquially referred to as “wallpaper effects,” the joke being that the experiment was affected by the color of the wallpaper in the room. This is the happiest possible explanation for failure to reproduce: It means that both experiments have revealed facts about the universe, and we now have the opportunity to learn what the difference was between them and to incorporate a new and subtler distinction into our theories.

The other interpretation is that the original finding was false. Unfortunately, an ingenious statistical argument shows that this second interpretation is far more likely.............

http://www.firstthings.com/article/2016/05/scientific-regress

 

 

 

Edited April 19, 2016 by cdowis

[cdowis]

Here is an excellent article looking at the entire issue of peer review from the Journal of the Royal Society of Medicine

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1420798/

[ksfisher]

36 minutes ago, cdowis said:

 

For starters, there's a "replication crisis" in science. This is particularly true in the field of experimental psychology, where far too many prestigious psychology studies simply can't be reliably replicated.

 

Not really central to what you're saying, but of interest anyway, the Center for Open Science study which made headline proclaiming this "replication crisis" has itself been discredited.

 

https://www.washingtonpost.com/news/speaking-of-science/wp/2016/03/03/errors-riddled-2015-study-showing-replication-crisis-in-psychology-research-scientists-say/

 

[mfbukowski]

Science works- when it works.

But the worship of science is one of the great fallacies of our times.  Future generations will look back and see us as a century or two of tremendous discovery in certain areas while ignoring what is important in life- family, having a meaningful life of joy and happiness, and a satisfactory understanding of our place in the universe.

Nihilism never was happiness, nor can it ever be happiness.

Science gives us no meanings- no significance except what is defined as "statistical significance".  Scientism is nihilism.  Logical positivism is dead even according to AJ Ayer, one of its founders.  Logical positivism is the belief that for a proposition to be "true" it must be scientifically verifiable.

Quote

 

By the late 1960s, logical positivism had clearly run its course.^[41] Interviewed in the late 1970s, A J Ayer supposed that "the most important" defect "was that nearly all of it was false".^[42][43] Although logical positivism tends to be recalled as a pillar of scientism,^[44]Carl Hempel was key in establishing the philosophy subdiscipline philosophy of science^[13] where Thomas Kuhn and Karl Popper brought in the era postpositivism.^[39]John Passmore found logical positivism to be "dead, or as dead as a philosophical movement ever becomes".^[42]

Logical positivism's fall reopened debate over the metaphysical merit of scientific theory, whether it can offer knowledge of the world beyond human experience (scientific realism) versus whether it is but a human tool to predict human experience (instrumentalism).^[45][46]

 

https://en.wikipedia.org/wiki/Logical_positivism

Science, like any other human discipline is often controlled by human emotions and politics about what subjects get funding and who can or will become a superstar, or how to get ahead in the pecking order.

It works fine when it works.  It's like anything else.

I tend toward instrumentalism, with a wikipedia link above.

Edited April 19, 2016 by mfbukowski

[cdowis]

del

Edited April 19, 2016 by cdowis

[mfbukowski]

2 minutes ago, cdowis said:

CFR, from a peer reviewed professional journal.

I don't know if you are joking or not.

[mfbukowski]

I am just going to post this about my understanding of Nietzsche and nihilism- because he saw this precise problem coming, over a hundred years ago

A lot of religious people do not like Nietzsche because he like to say things to shock people and so he is misunderstood.  For example his famous dictum that "God is dead" was more an observation about Christianity at the time he wrote it than anything else.   No question he did not like German Protestantism- which he saw as "Christianity", but in my opinion, from a Mormon understanding, MUCH (not all) of what he says has merit.  I personally agree with him that that the "God" of pre-restoration Christianity indeed was "dead" if he had ever even been alive- the god of substance theology and the trinity was based on pagan Greek notions.  Joseph restored the real God in my opinion, and today, to humanity, God lives again.

Nietzsche decries nihilism- the belief in nothing.  That is seen today in those who still believe that scientific truth is the only "truth", it is a truth without meaning.

He is not a nihilist at all- he believes, as do we also, that mankind should create it's own world an give meaning to "matter unorganized", though of course he does not use those terms.  We of course believe that an exalted "Human God" does exactly that.

For once, wikipedia does a pretty good job on summarizing his views and I will quote that article at length.  I think it is germane to this discussion because it shows what happens when one only believes in "objective truth" devoid of meaning- as those who believe in scientism and positivism do.  In fact Nietzsche can be seen as an "anti-positivist"

So read this summary with a grain of salt, and see it as if you are reading something pro-LDS- do not close off your mind because it is that "evil Nietzsche guy".  And doing so, relate it to the present day and see present day circumstances in what is said.

Quote

 

Nihilism is often associated with the German philosopher Friedrich Nietzsche, who provided a detailed diagnosis of nihilism as a widespread phenomenon of Western culture. Though the notion appears frequently throughout Nietzsche's work, he uses the term in a variety of ways, with different meanings and connotations, all negative^{[citation needed]}. Karen Carr describes Nietzsche's characterization of nihilism "as a condition of tension, as a disproportion between what we want to value (or need) and how the world appears to operate."^[24] When we find out that the world does not possess the objective value or meaning that we want it to have or have long since believed it to have, we find ourselves in a crisis.^[25] Nietzsche asserts that with the decline of Christianity and the rise of physiological decadence,^{[clarification needed]} nihilism is in fact characteristic of the modern age,^[26] though he implies that the rise of nihilism is still incomplete and that it has yet to be overcome.^[27] Though the problem of nihilism becomes especially explicit in Nietzsche's notebooks (published posthumously), it is mentioned repeatedly in his published works and is closely connected to many of the problems mentioned there.

Nietzsche characterized nihilism as emptying the world and especially human existence of meaning, purpose, comprehensible truth, or essential value. This observation stems in part from Nietzsche's perspectivism, or his notion that "knowledge" is always by someone of some thing: it is always bound by perspective, and it is never mere fact.^[28] Rather, there are interpretations through which we understand the world and give it meaning. Interpreting is something we can not go without; in fact, it is something we need. One way of interpreting the world is through morality, as one of the fundamental ways that people make sense of the world, especially in regard to their own thoughts and actions. Nietzsche distinguishes a morality that is strong or healthy, meaning that the person in question is aware that he constructs it himself, from weak morality, where the interpretation is projected on to something external. Regardless of its strength, morality presents us with meaning, whether this is created or 'implanted,' which helps us get through life.^[29]

Nietzsche discusses Christianity, one of the major topics in his work, at length in the context of the problem of nihilism in his notebooks, in a chapter entitled "European Nihilism".^[30] Here he states that the Christian moral doctrine provides people with intrinsic value, belief in God (which justifies the evil in the world) and a basis for objective knowledge. In this sense, in constructing a world where objective knowledge is possible, Christianity is an antidote against a primal form of nihilism, against the despair of meaninglessness. However, it is exactly the element of truthfulness in Christian doctrine that is its undoing: in its drive towards truth, Christianity eventually finds itself to be a construct, which leads to its own dissolution. It is therefore that Nietzsche states that we have outgrown Christianity "not because we lived too far from it, rather because we lived too close".^[31] As such, the self-dissolution of Christianity constitutes yet another form of nihilism. Because Christianity was an interpretation that posited itself as the interpretation, Nietzsche states that this dissolution leads beyond skepticism to a distrust of all meaning.^[32][33]

 

So again, apply what is said to pre-restoration Christianity.

Pre- restoration Christianity provides a belief in objective truth which we now see in atheists like Dawkins who use that belief in objective truth to discredit belief in God.

Science comes in and takes over that position - that the only objective truth is scientific truth- and therefore proves- allegedly- that God does not exist, and the Christian world swallows the argument and we have people leaving churches in droves.

Understand that.  Christianity does not have an open canon- there are TRUTHS revealed once- in the Bible- for all eternity.  Ex nihilo creation.  "TRVTH" inscribed in rock on the top of a Greek pagan temple.

So now science accepts objective truth- but it cannot find meaning.  There is no verifiable way to experiment about God, so God must not exist.  So now you have positivism and Dawkins and scientism

Nietzsche's point- "translated" using LDS jargon is that "truth" is contextual and is created from matter unorganized by the Human Mind.   If we think of God as an exalted "Human" and that each of us as humans, create our own little "worlds" this makes sense. 

We create meaning in our own lives, through study and testimony- direct personal revelation.  We have an open canon and are flexible enough to change with the times as needed because we have a prophet who is living- truth is not dead and etched in stone but part of an interactive process with our Father- our "Human" God with whom we co-create reality as we understand it.  He reveals more to us as we need it.

But the key here is the creation of meaning in our lives.  THIS is the key similarity here with Nietzsche.

So yes, Nietzsche said a lot of crazy things about the German Protestantism of his time.   Some of it was wrong- some not so much "wrong".  But on these points he was right on the money

And the main point is yes- scientisim and its attendant nihilism does not create morality nor does it create happiness.

And now it is becoming clear that it doesn't even necessarily describe "objective reality" but the "truth" which will create the most income for the institution doing the research.

My two cents.

[thesometimesaint]

Science by its very nature is not a search for truth. It is a search for an explanation for an observation, as such it is always tentative. It is simply the best explanation for some observation we have at the present time.

[cdowis]

8 minutes ago, thesometimesaint said:

Science by its very nature is not a search for truth. It is a search for an explanation for an observation, as such it is always tentative. It is simply the best explanation for some observation we have at the present time.

There is a saying, "Teaching school is great, except for the students."

Science is also terrific, except for the scientists.  They are human, and influenced as any mortal == proud, stubborn (even in the face of facts), naive, motivated by money (grants) and reputation (awards), recognition, willing to cheat and lie to achieve their goals, even attempting to prove a political agenda, etc etc.  They are just folks like the rest of us, but have impressive academic credentials.

Next time  someone asks, what do the experts think, let's see this in a peer-reviewed journal, etc, just remember what you have learned here.

[thesometimesaint]

Agreed, but what is your alternative hypothesis?

"But to affirm that the Sun is really fixed in the center of the heavens and that the Earth revolves very swiftly around the Sun is a dangerous thing, not only irritating the theologians and philosophers, but injuring our holy faith and making the sacred scriptures false."

Robert Bellarmine, Cardinal of the Roman Catholic Church and the foremost Vatican theologian of the seventeeth century.

[ERayR]

1 hour ago, thesometimesaint said:

Science by its very nature is not a search for truth. It is a search for an explanation for an observation, as such it is always tentative. It is simply the best explanation for some observation we have at the present time.

The best explanation from which POV?

[ERayR]

47 minutes ago, thesometimesaint said:

Agreed, but what is your alternative hypothesis?

"But to affirm that the Sun is really fixed in the center of the heavens and that the Earth revolves very swiftly around the Sun is a dangerous thing, not only irritating the theologians and philosophers, but injuring our holy faith and making the sacred scriptures false."

Robert Bellarmine, Cardinal of the Roman Catholic Church and the foremost Vatican theologian of the seventeeth century.

And this proves what?  Only that what mfbukowski said, there are human beings on both sides. 

Edited April 20, 2016 by ERayR

[thesometimesaint]

3 minutes ago, ERayR said:

The best explanation from which POV?

From whatever the empirical evidence shows. As such it is always tentative. IE; Great man that he was Galileo wasn't the last word on Gravity, neither was Newton, Laplace, Einstein, nor even the Theory of Quantum Mechanics.

[cdowis]

1 hour ago, thesometimesaint said:

Agreed, but what is your alternative hypothesis. 

To understand the difference between opinion (consensus of opinion)  and facts/ data.

(I am influenced here by Nibley)

Edited April 20, 2016 by cdowis

[thesometimesaint]

23 minutes ago, cdowis said:

To understand the difference between opinion (consensus of opinion)  and facts/ data.

(I am influenced here by Nibley)

Again I agree. What we have in science is an explanation of an observation. IE;

 

41 minutes ago, ERayR said:

And this proves what?  Only that what mfbukowski said, there are human beings on both sides. 

We can all look out into the night sky and see the "wondering stars"(now known as planets) That is the facts. The explanation of those facts is the science.

SEE

ERayR:

No question that we are all human even scientists. But lets take that to a logical conclusion. Say my heart is not dong well. Do I schedule an appointment with my heart doctor, or my local house plumber? After all they are both human and deal with plumbing.

Edited April 20, 2016 by thesometimesaint

[cdowis]

3 hours ago, thesometimesaint said:

No question that we are all human even scientists. But lets take that to a logical conclusion. Say my heart is not dong well. Do I schedule an appointment with my heart doctor, or my local house plumber? After all they are both human and deal with plumbing.

You are being silly.  You are pretending that I am arguing that we completely ignore the experts.
The expert is the one who has the knowledge and experience.  But still be WATCHFUL when they then offer their conclusion.  

The point is ====>>>>> Given the facts/ data, would I come to the same conclusion.  That means, to ask tough questions and why he is coming to a certain conclusion.  If it is a complex issue, they should be able to break it down into its component that an educated layman could understand.

A practical application is the health field where serious medical decisions are being made, even contrary to the doctor's advice.  In one particular case, it had to do with heart surgery, and the patient's decision turned out to be correct.

Edited April 20, 2016 by cdowis

[The Nehor]

Psychology is borderline quack science. It helps many people but there are so many variables and approaches it consists mostly of educated guesswork. Still better then chiropractors.

The problem is we have tried to inculcate the scientific worldview into our population but unless you are a scientist (professional or amateur) it doesn't really work. Most people are more baffled by the science coming out to us then the uneducated peasant of previous centuries was baffled by the Catholic Mass in Latin. The Dunning-Kruger effect hits hard here. We have ignoramuses convinced they know a lot about science because they read a book or an article on Wikipedia. For a more extreme example see politics where the conclusively disproven is an article of unquestioned faith to many.

I trust the hard sciences more then the soft sciences but I try not to let them rule me. When I rule the cosmos as a God I will start accepting scientific certainties.  

[longview]

10 hours ago, thesometimesaint said:

Again I agree. What we have in science is an explanation of an observation.

Digging up fossils is an observation.  Dating the layers is more conjectural (based on a whole raft of assumptions).  Providing a "narrative" on the "story" of the dinosaurs is mostly metaphysical.   Hope that clears it up for you.

[mfbukowski]

12 hours ago, thesometimesaint said:

Agreed, but what is your alternative hypothesis?

"But to affirm that the Sun is really fixed in the center of the heavens and that the Earth revolves very swiftly around the Sun is a dangerous thing, not only irritating the theologians and philosophers, but injuring our holy faith and making the sacred scriptures false."

Robert Bellarmine, Cardinal of the Roman Catholic Church and the foremost Vatican theologian of the seventeeth century.

There you go.   Perfect quote.  The truth of science is tied to spiritual truth because truth is ONE and objective.

THAT is the basis really of the apostasy.  This is the problem with the correspondence theory of truth.

So now Dawkins is of the same belief but the scientific tables have changed.  There is no observable God- no proof of God scientifically therefore God does not and cannot exist.

There is a straight line between Bellarmine and Dawkins and the theory of truth they support.  It is as obvious as it can be!!  And here we have debates on evolution with the supporters of Bellarmine who say that evolution is anti-religion because science must explain religion and evolution does not.  What a paradox!  The assumption is that religious truth is observable and this idea leads to both Dawkins AND Bellarmine AND fundamentalist literalism.

The answer is D&C 93- 

Quote

 30 All truth is independent in that sphere in which God has placed it, to act for itself, as all intelligence also; otherwise there is no existence.

Science is in an independent "sphere" and so is religion.  THAT is the answer.  Dawkins is wrong, Bellarmine is wrong, and the fundamentalist literalist is wrong.

And how do we decide the truth of religion?

Alma 32

Quote

 28 Now, we will compare the word unto a seed. Now, if ye give place, that a seed may be planted in your heart, behold, if it be a true seed, or a good seed, if ye do not cast it out by your unbelief, that ye will resist the Spirit of the Lord, behold, it will begin to swell within your breasts; and when you feel these swelling motions, ye will begin to say within yourselves—It must needs be that this is a good seed, or that the word is good, for it beginneth to enlarge my soul; yea, it beginneth to enlighten myunderstanding, yea, it beginneth to be delicious to me.

By how delicious it is.  NOT "how it corresponds to reality" which is unknowable here.

Objection:  "But then it is "subjective"!

Uh yeah, if it is not objective it must be subjective.  As Rorty says, there are no truths outside of sentences and there are no sentences without a human mind to construct them.

Truth is dependent upon a human mind- upon human sentences- to be constructed.  The context gives them meaning and either the alleged "truths" work or they don't.

Science works so it is true. Space vehicles fly to Mars- it works.   Religion works- beliefs become "delicious" to us and give us peace and testimony.  Science is wed to objectivity, religion is wedded to subjectivity.  Anything else leaves us in the fix in which we find ourselves where Dawkins, Bellarmine, and the fundamentalists are all on the same team.

[mfbukowski]

12 hours ago, thesometimesaint said:

 

No question that we are all human even scientists. But lets take that to a logical conclusion. Say my heart is not dong well. Do I schedule an appointment with my heart doctor, or my local house plumber? After all they are both human and deal with plumbing.

Scientists and plumbers are on the same objective page.  False dichotomy.

The question is more do you read about talking donkeys to do science?

Or Tyson to teach you what the scriptures are really about?

[thesometimesaint]

9 hours ago, longview said:

Digging up fossils is an observation.  Dating the layers is more conjectural (based on a whole raft of assumptions).  Providing a "narrative" on the "story" of the dinosaurs is mostly metaphysical.   Hope that clears it up for you.

Radiometric dating is a well established science based on observation. BTW Jesus didn't ride around on dinosaurs.

[thesometimesaint]

16 hours ago, cdowis said:

You are being silly.  You are pretending that I am arguing that we completely ignore the experts.
The expert is the one who has the knowledge and experience.  But still be WATCHFUL when they then offer their conclusion.  

The point is ====>>>>> Given the facts/ data, would I come to the same conclusion.  That means, to ask tough questions and why he is coming to a certain conclusion.  If it is a complex issue, they should be able to break it down into its component that an educated layman could understand.

A practical application is the health field where serious medical decisions are being made, even contrary to the doctor's advice.  In one particular case, it had to do with heart surgery, and the patient's decision turned out to be correct.

Since we are all human prone to all the faults and foibles of everyone else why should you trust anyone? IE; Bill Gates is far from perfect. Why do you use a personal computer?

There is nothing precluding anyone from being watchful. I personally recommend it. But lets not bury our heads in the sand denying reality. IE; 1/4 adult Americans still believe the sun revolves around the earth.

I'm reasonably well educated in Sociology, but the further I get from that narrow field to more I have to trust those that know more than I do. Such is with all of us. IE; I can't do open heart surgery. So if faced with having to undergo the operation I must of necessity trust the doctors who do.

What would have happened if the patient has turned out to be wrong?

[cdowis]

22 minutes ago, thesometimesaint said:

Radiometric dating is a well established science based on observation. 

I have not done detailed research on this issue, but I understand that for extremely long periods of time in the past, it is based on non-observable assumptions.   Also, I have read about situations where certain artifacts do not match with the dating. (do not have the sources)

 

Edited April 21, 2016 by cdowis

[cdowis]

13 minutes ago, thesometimesaint said:

What would have happened if the patient has turned out to be wrong?

I did not go into details, but was later, two months after the decision, that he was told that the surgery was only to prevent possible future discomfort (shortness of breath) , and would not help to prevent a heart attack.  It was OPTIONAL, with the very real possibility to have permanent liver damage.

Edited April 21, 2016 by cdowis

[thesometimesaint]

8 minutes ago, cdowis said:

I have not done detailed research on this issue, but I understand that for extremely long periods of time in the past, it is based on non-observable assumptions.   Also, I have read about situations where certain artifacts do not match with the dating. (do not have the sources)

Depends of the type of radiometric dating, and the length of of the time period in question. C-14 is accurate to about 40,000 years ago. With some methods going back some 4.5 billion years, or more.

There will always be exceptions, but they all must conform to the laws of physics.

SEE http://www.talkorigins.org/faqs/dating.html