Replication Crisis

Why So Much Science is Wrong, False, Puffed, or Misleading | 9/27/2020

… College London with lots of experience in debunking poorly-made research, particularly in his own field of psychology. For the last decade or more, this discipline has been the unfortunate poster child for the “ Replication Crisis ,” the discovery that – to use Stanford’s John Ioannidis’ well-known article title – “ Most Published Research Findings Are False .” Take the example of former Cornell psychology professor Daryl Bem and his infamous “ psychic pornography ” experiment …

What’s wrong with social science and how to fix it | 9/12/2020

… what the average predictions looked like after seeing the papers: from 53.4% in 2009 to 55.8% in 2018 (difference not statistically significant; black dots are means). I frequently encounter the notion that after the replication crisis hit there was some sort of great improvement in the social sciences, that people wouldn’t even dream of publishing studies based on 23 undergraduates any more (I actually saw plenty of those), etc. Stuart …

Follow Replication Crisis:    

Unmaskers unmasked? A different perspective on replication studies | 9/8/2020

A paper by Bryan, Yeager & O’Brien (2019) sheds a new light on the replication crisis in psychology. An image that has emerged here and there of original authors as tinkering cheats and replicators as holy defenders of scientific morality must be revised. Replicators often take too many liberties, both in the design of their research and in the way in which they analyze data. Read full post …

5 September, 2018 (w/updates) RSS 2018 – Significance Tests: Rethinking the Controversy | 9/6/2020

Sir David Cox, Nuffield College, Oxford Deborah Mayo, Virginia Tech Richard Morey, Cardiff University Aris Spanos, Virginia Tech Intermingled in today’s statistical controversies are some long-standing, but unresolved, disagreements on the nature and principles of statistical methods and the roles for probability in statistical inference and modelling. In reaction to the so-called “replication crisis” in the sciences, some reformers suggest significance tests as a major culprit. To understand …

Fantasy Football Draft Strategy Has a Replication Crisis | 9/6/2020

… was a story that was told and retold in academia, and applied to everything from parenting (of course) to business strategy (really). But, here’s the thing: the first story wasn’t the full story. The Replication Crisis In 2018, researchers from New York University published a study that added a vital component to the original marshmallow test: nuance. It turns out that the original study had its shortcomings: most prominent among …

Replication crisis in mathematics | 9/5/2020

… close to Mochizuki believe that he does? In fact, the very fact that he can publish it in a journal that has himself as chief editor shows without doubt that mathematics will have a replication crisis (however small) if mathematicians do not maintain a list of reputable journals. $\endgroup$ – user21820 16 hours ago 3 $\begingroup$ @user21820 I deliberately phrased my remarks about the “serious disagreement” about the “proposed proof” to …

Top 10 Reasons You Can’t Trust Science Right Now | 9/1/2020

I’m sorry, can you repeat that? No? Then why the hell is it published in a scientific journal? Unbeknownst to many, science – especially social sciences and psychology – is in the midst of a replication crisis , [1] a wide-ranging inability to successfully repeat the findings of previously published papers. In fact, right now, around seventy percent of scientific studies are failing when attempts are made to replicate their findings. That’s …

On being ‘a voter’ in the 2020 US election | 8/31/2020

… years immediately after these findings were published, psychology found itself weathering a storm of evidence that many published results cannot be independently reproduced . And one of the most contentious kinds of effect within psychology’s “ replication crisis ” is behavioural priming . Like the nouns-vs-verbs study, most behavioural priming studies feature subtle manipulations with dramatic effects on subsequent behaviour. But many of these have failed to replicate—such as the finding …

Book review: Science Fictions | 8/29/2020

This weekend I finished reading the new book Science Fictions , by Stuart Ritchie . It’s about the replication crisis in science, the underlying causes and ways to make science more reliable. In this article I’ll review it and ponder some of the issues it raises. I’ll also suggest some topics that it didn’t cover. Not got time to read a book? No problem! There’s a comic version of it too !  —  @wellcometrust …

Replication crisis | 8/27/2020

Public policy, medical treatments and the way we see the world may have been built on the shakiest of foundations. In June, the latest chapter in the replication saga featured a highly influential study on memory. In 2010, in a blockbuster article in the journal Nature, New York University researchers found that it was possible — without the use of drugs, brain stimulation or anything invasive — to “rewrite” a person’s memory …

What is an “un-reproducible result”? – academia.stackexchange.com | 8/25/2020

add a comment 1 Wikipedia has an excellent article on the current Replication Crisis (or Reproducibility Crisis), and I can’t hope to improve on it. I would recommend that you start by reading that article, the many examples included, and the other referenced links therein. The replication crisis (or replicability crisis or reproducibility crisis) is, as of 2020, an ongoing methodological crisis in which it has been found that many …

El Simio que entendió el Universo (entrevista a Steve Stewart-Williams) | 8/23/2020

… a mathematical proof. But that’s true of every claim in science. Scientists can only ever nudge our confidence up or down. Perhaps this is harder in EP than in some fields (although the replication crisis in psychology and elsewhere suggests that it’s not as easy as we thought in any field). But saying that it’s harder is very different than saying that it’s impossible. And in …

Google Alert - science | 8/22/2020

science As-it-happens update ⋅ August 22, 2020 NEWS Yes, we need to ‘trust science ‘ Minneapolis Star Tribune Science is suffering from a replication crisis . Too many landmark studies can’t be repeated in independent labs, a process crucial to separating flukes … Flag as irrelevant 5 Questions: Alice Popejoy on race, ethnicity and ancestry in science Stanford Medical Center Report 5 Questions: Alice Popejoy on race, ethnicity and ancestry in science …

Re: We rely on science. Why is it letting us down when we need it most? | 8/22/2020

We rely on science. Why is it letting us down when we need it most?by Stuart Ritchie, Los Angeles TimesCredit: CC0 Public DomainScience is suffering from a replication crisis . Too many landmark studies can’t be repeated in independent labs, a process crucial to separating flukes and errors from solid results. The consequences are hard to overstate: Public policy, medical treatments and the way we see the world may have …

We rely on science. Why is it letting us down when we need it most? | 8/20/2020

… any number of creation/evolution issues. But as this LA Times op-ed notes, sometimes scientists themselves contribute to the lack of trust by the public. The article states: Science is suffering from a replication crisis . Too many landmark studies can’t be repeated in independent labs, a process crucial to separating flukes and errors from solid results. The consequences are hard to overstate: Public policy, medical treatments and the way …

Letters to the Editor: One failure to replicate a scientific study doesn’t mean there’s a crisis | MSN | 8/20/2020

… HBO) To the editor: In his Aug. 14 op-ed article, psychology professor Stuart Ritchie reports on a recent failure to replicate a study of ours and uses this as an example of a replication crisis in science . He lauds efforts by the scientific journal Cortex to promote the replication of seminal findings as a means to address this crisis and laments that such efforts are too infrequent in science …

Time-of-day influence on resting-state fMRI: an overlooked factor contributing to the replication crisis? | 8/20/2020

Findings from neuroimaging studies show low replication rates, but the origin of the problem is not clear. The current project explored the extent of the time-of-day dependent metabolic variation to the replication crisis. Using data from the human connectome project (HCP) release S1200, cross-spectral density dynamic causal modelling was used to analyse time-dependent effects on the hemodynamic response and effective connectivity parameters in the spontaneous fluctuations …

Stuart Ritchie: We rely on science. Why is it letting us down when we need it most? – Twin Cities | 8/20/2020

Science is suffering from a replication crisis . Too many landmark studies can’t be repeated in independent labs, a process crucial to separating flukes and errors from solid results. The consequences are hard to overstate: Public policy, medical treatments and the way we see the world may have been built on the shakiest of foundations. In June, the latest chapter in the replication saga featured a highly influential study on memory …

Their findings don’t replicate, but they refuse to admit they might’ve messed up. (We’ve seen this movie before.) | 8/20/2020

Link: http://datacolada.org/82 2 Effect: “Independent consumers who choose on behalf of large groups tend to choose more selfishly, whereas everyone else tends to choose less selfishly” Replication: Succeeded but then failed (after removing typo) Reply: Effect is real. Happy for directional trend Explanation: Study conducted in a different time of the year Link: http://datacolada.org/83 3 Effect: “Consumers with ‘low self-concept clarity’ are more motivated to …

How The Replication Crisis Undermines Government By Bureaucracy | 8/19/2020

By Lewis M. Andrews August 19, 2020 For decades, critics of secular progressivism have been skeptical of the many billions Congress and state legislatures annually spend on university research, especially in the social sciences. What leftists like to tout as a valuable investment in better understanding and improving the human condition is increasingly seen by others as a cleverly camouflaged marketing effort to justify bigger and more intrusive bureaucracy. It …

We rely on science. Why is it letting us down when we need it most? | Phys Org | 8/19/2020

Science is suffering from a replication crisis . Too many landmark studies can’t be repeated in independent labs, a process crucial to separating flukes and errors from solid results. The consequences are hard to overstate: Public policy, medical treatments and the way we see the world may have been built on the shakiest of foundations. In June, the latest chapter in the replication saga featured a highly influential study on memory …

US Leftists Are Nuts - Self Report Mental Health Issues - 17 Aug 2020 | 8/18/2020

… aged 18 to 29 had been diagnosed with a mental illness. Moderates were more mentally stable than Liberals but less so than Conservatives. This is only one analysis, and of course there is a replication crisis in social science. But a new study by Danish social scientist Emil Kirkegaard (right) has replicated the Pew Research findings with a vengeance, revealing that, in a huge dataset and on multiple different measures …

Some thoughts inspired by Lee Cronbach (1975), “Beyond the two disciplines of scientific psychology” | 8/16/2020

… All this sounds a lot like Meehl, and Meehl is great , but Cronbach adds value by giving lots of specific applied examples. In the article, Cronbach makes a clear connection between interactions and the replication crisis arising from researcher degrees of freedom, a point that I rediscovered—40 years later—in my paper on the connection between varying treatment effects and the crisis of unreplicable research. Too bad I hadn’t …

Op-Ed: We rely on science. Why is it letting us down when we need it most? | MSN | 8/14/2020

… when we need it most? © (Brian van der Brug / Los Angeles Times ) Test media and supplies at the UCLA Clinical Microbiology Laboratory. (Brian van der Brug / Los Angeles Times ) Science is suffering from a replication crisis . Too many landmark studies can’t be repeated in independent labs, a process crucial to separating flukes and errors from solid results. The consequences are hard to overstate: Public policy, medical treatments and the way …

Viewpoints: Since Masks Save Lives, Why Is There A Debate On Them?; US Lessons On Embracing Preventive Measures | Kaiser Health News | 8/14/2020

… open and sniff all my spice jars. Nothing. This is spooky.” (Ann-Sophie Barwich, 8/14) Los Angeles Times: Why Is Science Letting Us Down When We Need It Most? Science is suffering from a replication crisis . Too many landmark studies can’t be repeated in independent labs, a process crucial to separating flukes and errors from solid results. The consequences are hard to overstate: Public policy, medical treatments and the way …