Here's another Scientific American blog article (Oct 4, 2011), "Drawing the line between science and pseudo-science," By Janet D. Stemwedel. Denimbius has offered some constructive comments:
While the example of creationism can be informative, perhaps the greater challenge to the problem of demarcation is the example of astrology, which the author mentions but doesn't directly address. Astrology is worthy of much more scientific scrutiny than it has received, especially in light of the findings of the late Michel Gauquelin. How have the scientific claims against astrology held up under Popper's criteria?
Astrology presents many "indications" or "significations," which due to the complexity of the subject and the insistence of astrologers on a holistic approach, are notoriously difficult to test.
To address this complexity, the 1985 double-blind experiment done by magician-scientist Shawn Carlson, set out to test what he called the "scientific hypothesis." Even a holistic approach to astrology, he suggested, could not be supported by the tests he designed. The tests in his experiment, published in NATURE [Dec. 1985, (318), 419-425] and by far the most frequently cited detailed claim against astrology, asked astrologers to match natal charts with psychological (CPI) profiles.
Although Carlson concluded his experiment with the assertion that the astrologers failed, recent peer-reviewed reassessments of the study have been critical of Carlson's analysis, pointing to numerous flaws, such as not following his own protocol, irrelevant grouping of results, and not reporting results that could have clarified a curious "statistical fluctuation" in the study.
As it turns out, when correctly assessed according to Carlson's own stated protocol and the normal evaluations of the social sciences, the data from the study actually supports the astrologers performance significantly better than chance. This critical evaluation might be the first well-documented case where the "scientific hypothesis" against astrology has been falsified according to Popper's criteria of demarcation.
This falsification of the scientific claim against astrology is a crucially interesting development for science. Clearly, the experiment needs a fair replication, which incorporates the improvements and safeguards offered by the critics. Until this happens, the claim that astrology is pseudoscience is seriously questionable. With the hypothesis for this widely cited experiment now broken, how does science explain the results?
Here are the links to the original Carlson article and some of its criticism:
http://muller.lbl.gov/papers/Astrology-Carlson.pdf
http://www.theoryofastrology.com/carlson/McRitchie_Carlson1985.pdf
Saturday, February 18, 2012
Magicians, misdirection, and science: Who is fooling whom?
An older article (Dec. 2008) in Scientific American, posted on the SA blog, "Magic and the Brain: How Magicians 'Trick' the Mind" elicited the following response from reader Denimbius (comment 20, Feb. 13, 2012):
This article reminds me of a criticism I read of the "Double-blind test of astrology" by magician-scientist Shawn Carlson, published in Nature (Vol. 318, 5 Dec. 1985 pp. 419-425). The criticism seemed to suggest that Carlson used misdirection techniques in the published article. This is in addition to the criticism that the tests were unfairly designed to begin with, making them extraordinarily harder than they needed to be.
Carlson's stated protocol required the participating astrologers examine each natal chart they were supplied and to select the correct CPI from 3 supplied for each chart as either the 1st or 2nd choice. Yet, in his evaluation Carlson draws our attention to the 3rd choice, which was chosen no better than chance, and declares that the 1st and 2nd must also have been chosen no better than chance! The data in the article shows that the first two charts were actually chosen at a marginally significant rate.
Carlson's 3-choice test finding helps confirm the illusion that another test depended on its 3 groups. The astrologers rated (scale of 1-10) the accuracy of each of the 3 CPIs supplied for each chart (110 CPIs in this test vs. 116 for the 3-choice test). Applying the three bogus categories and drawing attention to the negative slope of the 1st category, Carlson declares that the result was no better than chance. However, his data shows that when ungrouped, the 10-rating test was significant for the astrologers, better than the 3-choice test.
In a control group test, Carlson found that the volunteer students could not identify their correct chart interpretation, written by the astrologers, out of 3 supplied, yet the control group successfully chose the "correct" interpretation with a high significance, which Carlson explains as a "statistical fluctuation."
In a related test, the students ranked the accuracy (scale of 1-10) the individual paragraphs within the same astrological interpretations. This test might have clarified whether the surprising "statistical fluctuation" results had somehow gotten switched. But Carlson complained that he couldn't be guaranteed that the volunteers had followed his instructions and discarded this test without giving the results.
If read uncritically, with the bias that typical readers of Nature have against astrology, then the reader sees exactly what he or she expects. Yet if read critically, it is another story. Was Carlson inadvertently fooling himself?
Read the Carlson article: http://1.usa.gov/e2dTGc
This article reminds me of a criticism I read of the "Double-blind test of astrology" by magician-scientist Shawn Carlson, published in Nature (Vol. 318, 5 Dec. 1985 pp. 419-425). The criticism seemed to suggest that Carlson used misdirection techniques in the published article. This is in addition to the criticism that the tests were unfairly designed to begin with, making them extraordinarily harder than they needed to be.
Carlson's stated protocol required the participating astrologers examine each natal chart they were supplied and to select the correct CPI from 3 supplied for each chart as either the 1st or 2nd choice. Yet, in his evaluation Carlson draws our attention to the 3rd choice, which was chosen no better than chance, and declares that the 1st and 2nd must also have been chosen no better than chance! The data in the article shows that the first two charts were actually chosen at a marginally significant rate.
Carlson's 3-choice test finding helps confirm the illusion that another test depended on its 3 groups. The astrologers rated (scale of 1-10) the accuracy of each of the 3 CPIs supplied for each chart (110 CPIs in this test vs. 116 for the 3-choice test). Applying the three bogus categories and drawing attention to the negative slope of the 1st category, Carlson declares that the result was no better than chance. However, his data shows that when ungrouped, the 10-rating test was significant for the astrologers, better than the 3-choice test.
In a control group test, Carlson found that the volunteer students could not identify their correct chart interpretation, written by the astrologers, out of 3 supplied, yet the control group successfully chose the "correct" interpretation with a high significance, which Carlson explains as a "statistical fluctuation."
In a related test, the students ranked the accuracy (scale of 1-10) the individual paragraphs within the same astrological interpretations. This test might have clarified whether the surprising "statistical fluctuation" results had somehow gotten switched. But Carlson complained that he couldn't be guaranteed that the volunteers had followed his instructions and discarded this test without giving the results.
If read uncritically, with the bias that typical readers of Nature have against astrology, then the reader sees exactly what he or she expects. Yet if read critically, it is another story. Was Carlson inadvertently fooling himself?
Read the Carlson article: http://1.usa.gov/e2dTGc
Psychological manipulation and the questionable ethics of professor Chris French
Another Chris French blog article (Feb 7, 2012) appeared on the website for TheGuardian. "Astrologers and other inhabitants of parallel universes: Followers of pseudosciences such as astrology often draw spurious parallels between their beliefs and established science" This posting elicited the following response (Feb. 9, 2012) from reader Decloud:
Sunday, February 5, 2012
Featured on journal Nature blog - response to Chris French and the Shawn Carlson double-blind study of astrology
Yesterday, I was searching for a way to get some mainstream science attention for the criticisms published by various authors and myself of "A double-blind test of astrology" from Shawn Carlson, published in the highly-regarded science journal Nature in 1985. I found that the Nature website has a new guest blog section called "Soapbox Science" and one of the featured short articles is a recent piece named "The rise of anomalistic psychology - and the fall of parapsychology?" by University of London Professor Chris French. Because French mentions astrology and this is the Nature website, I thought it might be a good place to bring up the Carlson issue and I added my comments. All comments are subject to review and at this point I don't know if my comments will make it onto the blog.
Here is the URL: http://blogs.nature.com/soapboxscience/2011/12/19/the-rise-of-anomalistic-psychology-%E2%80%93-and-the-fall-of-parapsychology#comment-110
Here is what I wrote:
Chris, Before you get too cozy with this emerging discipline, it is interesting to reflect on how modern skepticism is transforming itself to become anomalistic psychology. First we have the transformation of CSICOP (Committee for the Scientific Investigation into Claims of the Paranormal) into CSI (Committee for Skeptical Inquiry). CSI dropped the pretense of being scientific in the wake of the Dennis Rawlins "sTarbaby" exposé of the committee's unscientific shenanigans regarding their tests of the Mars effect discovered by the late Michel Gauquelin. (BTW, in 1988, this effect was elevated by Suitbert Ertel through objective data ranking to the status of the Mars eminence effect). This more recent anomalistic psychology approach seems to favor skeptical rhetoric and lawyering under the guise of "critical thinking" over the scientific evaluation of evidence.
Consider the popular claim that astrology works by cognitive bias. This is a widely held belief among intelligent people, but no one has ever demonstrated it. Although classroom Forer-type tests have been performed hundreds of times, and are widely assumed to disprove astrology, they only demonstrate the Barnum effect, and cannot legitimately claim to refute astrology. The reason is because the sample of "astrology" to be evaluated by the students is carefully cherry picked to maximize the Barnum effect. This is not a scientific test of astrology with proper samples and rating of choices. Nothing of the sort has ever been published. Yet it is always trotted out in discussions against astrology. Is this an example of anomalistic psychology that you teach your students?
It is interesting to find this "soapbox" in the online pages of Nature. Research in astrology is done by interested amateurs, funded from their own pockets, as Gauquelin had done. There has not been a mainstream scientific forum for them to exchange ideas.
For example, In 1985 Nature published an article by Shawn Carlson entitled "A double blind test of astrology.” This article brought instant fame to its author and is still one of the most frequently cited studies to have claimed to scientifically refute astrology. Yet this article was not properly vetted by peer review because it contains design and methodological flaws that should have been caught, if perhaps they were not so well concealed. When evaluated against the actual design of the experiment (which Carlson sets out but does not follow) and the normal evaluations for significance used in the social sciences, the data marginally supports the participating astrologers.
It is frustrating to astrological researchers to see such a prestigious study as Carlson's go unchallenged in mainstream media. For those who wish to pursue the Carlson controversy, and Chris I strongly suggest that you do, please refer to the following links for references to the original article by Carlson and the ensuing peer reviewed discourse by Ertel and myself. My article includes a discussion of follow-up studies. The Carlson study needs a replication that incorporates the many useful suggestions that have been made through the discourse of these and other authors to make a fair and scientific study.
"A double-blind test of astrology": http://muller.lbl.gov/papers/Astrology-Carlson.pdf
"Appraisal of Shawn Carlson’s Renowned Astrology Tests": http://www.scientificexploration.org/journal/full/jse_23_2_full.pdf#page=7
"Support for astrology from the Carlson double-blind experiment": http://www.theoryofastrology.com/carlson/McRitchie_Carlson1985.pdf
Here is the URL: http://blogs.nature.com/soapboxscience/2011/12/19/the-rise-of-anomalistic-psychology-%E2%80%93-and-the-fall-of-parapsychology#comment-110
Here is what I wrote:
Chris, Before you get too cozy with this emerging discipline, it is interesting to reflect on how modern skepticism is transforming itself to become anomalistic psychology. First we have the transformation of CSICOP (Committee for the Scientific Investigation into Claims of the Paranormal) into CSI (Committee for Skeptical Inquiry). CSI dropped the pretense of being scientific in the wake of the Dennis Rawlins "sTarbaby" exposé of the committee's unscientific shenanigans regarding their tests of the Mars effect discovered by the late Michel Gauquelin. (BTW, in 1988, this effect was elevated by Suitbert Ertel through objective data ranking to the status of the Mars eminence effect). This more recent anomalistic psychology approach seems to favor skeptical rhetoric and lawyering under the guise of "critical thinking" over the scientific evaluation of evidence.
Consider the popular claim that astrology works by cognitive bias. This is a widely held belief among intelligent people, but no one has ever demonstrated it. Although classroom Forer-type tests have been performed hundreds of times, and are widely assumed to disprove astrology, they only demonstrate the Barnum effect, and cannot legitimately claim to refute astrology. The reason is because the sample of "astrology" to be evaluated by the students is carefully cherry picked to maximize the Barnum effect. This is not a scientific test of astrology with proper samples and rating of choices. Nothing of the sort has ever been published. Yet it is always trotted out in discussions against astrology. Is this an example of anomalistic psychology that you teach your students?
It is interesting to find this "soapbox" in the online pages of Nature. Research in astrology is done by interested amateurs, funded from their own pockets, as Gauquelin had done. There has not been a mainstream scientific forum for them to exchange ideas.
For example, In 1985 Nature published an article by Shawn Carlson entitled "A double blind test of astrology.” This article brought instant fame to its author and is still one of the most frequently cited studies to have claimed to scientifically refute astrology. Yet this article was not properly vetted by peer review because it contains design and methodological flaws that should have been caught, if perhaps they were not so well concealed. When evaluated against the actual design of the experiment (which Carlson sets out but does not follow) and the normal evaluations for significance used in the social sciences, the data marginally supports the participating astrologers.
It is frustrating to astrological researchers to see such a prestigious study as Carlson's go unchallenged in mainstream media. For those who wish to pursue the Carlson controversy, and Chris I strongly suggest that you do, please refer to the following links for references to the original article by Carlson and the ensuing peer reviewed discourse by Ertel and myself. My article includes a discussion of follow-up studies. The Carlson study needs a replication that incorporates the many useful suggestions that have been made through the discourse of these and other authors to make a fair and scientific study.
"A double-blind test of astrology": http://muller.lbl.gov/papers/Astrology-Carlson.pdf
"Appraisal of Shawn Carlson’s Renowned Astrology Tests": http://www.scientificexploration.org/journal/full/jse_23_2_full.pdf#page=7
"Support for astrology from the Carlson double-blind experiment": http://www.theoryofastrology.com/carlson/McRitchie_Carlson1985.pdf
Subscribe to:
Posts (Atom)
As I spoke to the students after the lecture it became apparent that the struggling students who sat near the rear of the lecture hall were the most at risk to being affected by those fears. The professor in that case was a philosopher and as I spoke to him afterward it quickly became evident that he was not aware of the potential psychological harm he was inflicting on his students and he apologized. He might simply have been passing on his own fears in much the same way that he had learned those fears himself.
However, in the present case Chris French is a psychology professor, and an excuse of unintentional psychological manipulation on his part would be far less convincing. Administrators at Goldsmiths, University of London, should take note that French, in an article on "Astrologers," leads with descriptions of abhorrent practices of ritual abuse, exorcism, racial differences, devil worship, sexual perversion, human sacrifices, forced abortions, and cannibalism. Considering the responsibility of his position as a psychology professor, we should all question the ethics of the way French has framed his arguments and whether he or his department should receive funding to further this agenda.
The arguments that French makes are far less serious than the way he has packaged his delivery. They are the same tired old arguments that show little knowledge or understanding of the discipline that he is trying to refute. Astrologers generally do not claim that astrology works by physical forces as he suggests. Where are the references? His opposition to that view, which is not supported by astrologers, even conflicts with Laplace's demon, which was typical of the mechanistic optimism of the same period in early modern science from which French draws his paradigm. His straw man argument even has its own internal problems.
If French would study the literature, he would have to argue against the "as above, so below" concept that has guided astrology from its beginnings. There are countless symmetries in nature, from snowflakes, to pine cones, to galaxies and quantum entanglement. It is a mystery why nature prefers symmetry and astrology is part of the natural curiosity of observing and attempting to correlate and understand that mystery.
Let's hope that French's "anomalistic psychology" is indeed a psychological anomaly as its name suggests and is not typical of the sort of ideas that psychology or the public should accept as being ethically or reasonably supportable.