152 Professional Astrologers vs. Science
I want to tell you today about the time I ended up testing 152 astrologers, how that came about and what happened and what really surprised me about the results. All of this began at a party of all places. A friendly acquaintance with whom I’ve been chatting unexpectedly asked me if I’d be up for her giving me an astrological reading based on my birth date and location. I told her I’d be totally up for trying as long as she’s okay with me
Being brutally honest about how accurate or inaccurate each part of the reading is. She agreed. So, we sat off to the side in a less crowded area of the party and the reading began. She actually got quite a number of things right in her reading. They were accurate and resonated with me. But there were also some egregious misses, claims she made based on my chart that were so far off the mark they were essentially the opposite of what’s true. What really
Struck me though is not so much the accuracy or inaccuracy, but the fact that whenever I told her that she had hit, she seemed pleased. But when I told her she was way off the mark, she seemed really shocked. her eyes would go wide and she looked concerned. This made me wonder when people are getting readings, do they not tell the astrologer when they’re totally wrong? Do they soft pedal any negative feedback so that even very wrong readings appear less wrong
Than they really were? At the end of the reading, I asked her a question that ended up having significance that I wasn’t aware of at the time. I asked her, “If it turned out that astrology doesn’t work, in that case, would you want to know that it doesn’t work or would you prefer to believe in astrology even if it isn’t real?” Without hesitating, she told me that if astrology doesn’t work, she wants to know that it doesn’t work. And I found
That response of hers really admirable. It just so happened that around that time I had my first opportunity to test astrology in a small way. We were running a study on personality looking to see which personality tests most accurately make predictions about people’s lives. For example, are MyersBriggs style personality tests the best or does the big five framework work better? Or maybe the enagram. Thinking back to my recent astrology experience, I had the idea of adding one more test
To the study. Astrological sun signs. These are based simply on what day of the year you are born, which determines whether you’re an Aries, Virgo, Pisces, etc. My idea was that we could use the same method that we’re using to measure the accuracy of personality tests, but apply it to astrological sun signs. Note that this wasn’t yet a test of astrologers. It was simply a test of astrological sun signs. We were looking at how accurately they can predict 37
Facts about your life, like how satisfied you are with your life, how many friends you have, and how religious you are. So, what did we find? Well, we found that whereas personality tests were able to predict facts about people’s lives to some degree, astrological sun science had no predictive ability at all for these 37 facts. When we released this result on our website and social media, I really wasn’t prepared for the backlash we’d get. Honestly, a lot of astrologers were
Pissed off at us. They argued that this is not a fair test of astrology. Some of their critiques were totally valid, but others less so. For instance, some argue that even if astrological sun signs work, there’s no reason we’d find that we could use them to predict these 37 life outcomes. I disagree with this critique. Claims are constantly being made about how sun signs tell you important things about people’s lives and personality, such as this one from
The New York Post. Tauruses value security and stability above all. They enjoy the finer things in life and are attracted to the full exploration of the senses. These people are especially hardworking and like to take a practical approach to life. This often leads them to be excellent with money and attracted to them like a magnet once they focus their skills and talents. If these claims were true, we would very likely find differences in facts about people’s
Lives based on their sun sign. For instance, in this example, they claim that some sun signs attract money. So, we’d expect to find a difference in income or how many times they failed to pay a bill. But, we don’t find these kinds of differences. But astrologers did have some strong critiques of our test. Sun signs alone are insufficient to yield good results. You need to take into account all planets and all houses. Otherwise, you’re compressing all of
Humanity into only 12 dimensions. Sun signs. I mean, yeah, if you’re only using their sun signs, it’s not going to yield accurate results. If you take into account houses and degrees and planetary positions, etc., it’s going to be much more complex and accurate. Surprise, We are unique beings. Did you speak with a single professional astrologer before jumping into this? The reality is though, lots of people do believe that the 12 zodiac signs alone are enough to tell you a lot about a
Person. Because of that, I think it’s fair game to test whether they really do predict facts about a person’s life. At the same time though, I think these critiques are valid. Real astrologers use far more than just the sun signs. They often do readings based on people’s full natal charts. To give you an idea of what one of these is like, here is the natal chart for the musician Michael Jackson. As you can see, it’s extremely
Complex and goes way beyond the 12 sun signs. There are different opinions on the best way to read such charts, but many astrologers agree on these basics. The chart is divided into 12 houses, each representing different areas of life. For example, some interpret the second house as being related to finances. Each such house has a zodiac sign which has an influence on the characteristics of that area of life. For example, some interpret a sixth house cusp in Virgo as being related to
A serviceoriented approach to health. Note, however, that not all astrologers use the same method to assign signs to houses. Then there are planets. Planets are placed in houses and signs affecting specific life areas or traits. For example, some interpret Mars in the third house as influencing communication style. As I learned more about this, I started to wonder, how can we give astrology a fair scientific test? The first idea I came up with is to show a
Natal astrological chart and then ask astrologers to predict which of five celebrities this was the real birth chart for. The idea of this test is that if natal charts really do tell us a lot about people and astrologers are able to accurately read these charts, then they should be able to tell us at a rate significantly above random guessing which celebrity each chart belongs to. But when I showed this test design to astrologers, they pointed out that it
Had at least two fatal flaws. The first flaw is that some astrologers actually already know the natal charts of some famous people because they study them to practice or they just study them out of interest. The second flaw is that some astrologers are actually able to figure out a person’s approximate year of birth from their astrological chart. Not by using astrology, but simply by using facts from astronomy combined with knowledge of how these charts are constructed. So, I scrapped that design
And went back to the drawing board. How can we create a scientifically valid test of astrology? Thankfully, six very nice astrologers independently agreed to help me during the process of designing the test. With their help, we settled on a final design. So, here’s the idea behind the test. We recruited 12 anonymous volunteers who were certain about their birth date, time, and location. We then had each of them answer more than 40 questions about themselves. Most of these were
Open-ended questions like how would you describe your personality and how do you deal with conflicts in your relationships? But they also included some factual questions like how many times you’ve ever been married and what your level of education is. But how did we choose these questions? Well, we asked astrologers. If you were trying to figure out someone’s astrological natal chart, what questions would you want to ask that person to help you figure it out? Then we generated the real
Astrological natal chart for each of our 12 anonymous subjects using both the Placidus and whole sign systems. Finally, for each person, we generated four decoy astrological charts. These are real charts, but they are generated using random birth dates, times, and locations. We applied restrictions to these so they couldn’t accidentally be nearly identical to the real chart. We also made sure that these were within the same year as the real chart since as we mentioned it’s possible to figure out
The year from a chart. We then put the astrologers to the test. Their goal was to one by one for each of the 12 anonymous people read the real information about that person and then look at the five natal charts that were presented. They had to predict which is the real natal chart for each person. If astrology works, then astrologers should be able to learn a lot about a person by reading their chart. And hence, they
Should be able to match a person to the chart at a rate that’s significantly better than random guessing. We also offered a $1,000 prize to the first astrologer who could get at least 11 out of 12 correct during the period of the study. So, what happened when we put this test out in the world? 152 astrologers who believed they could do this task at a rate better than random guessing enrolled in the challenge. Note that not all astrologers believe they
Could do this task, but we only analyzed data from the 152 of them who believe they could actually do it. In fact, 25% of these astrologers said right before they began the challenge that they believe they were going to get at least 11 out of 12 right and win the prize. So, what were the actual results of the study? Did they do as well as they thought they would? Well, to understand the results, we first have to understand
What would happen if the astrologers had no skill at all and were simply guessing completely at random. In that case, we’d expect astrologers to get one in five of the answers right. So for 12 anonymous subjects, that means they get an average of 2.4 out of 12 correct. So how many questions did they actually get right on average? The astrologers who took our test got only 2.49 right out of 12. This is not statistically significantly different from the average of 2.4 that
They’d get if they were guessing completely at random. But perhaps this is just because only some of them have skill. Well, let’s take a look at the distribution of how many answers correct the astrologers got. If astrologers were guessing completely randomly with no skill, we’d get a chart that looks like this. This chart reflects the fact that with random guessing, a lot of astrologers would get two or three questions right, but very few would get more than five questions right. So, what
Did we actually find for the real distribution? Well, the actual distribution of how many answers they got right matched this almost perfectly. In other words, astrologers really did not do better than you’d expect if they were guessing their answers completely at random. Despite 25% of these astrologers saying that they believed they would get at least 11 out of 12 questions correct, not a single one of them got more than five questions right. We also asked astrologers about their
Experience level. Maybe the inexperienced astrologers dragged down the results. Well, we found no link at all between how well astrologers performed on the task and how experienced with astrology they said they were. This leaves open an important question and it’s actually the thing that surprised me most about our results. Our data makes it clear that these astrologers could not do this task better than random chance even though the theory of astrology says they should be able to and even though they
Themselves believe they could do it. But if astrology is a systematic or perhaps even scientific system like some practitioners say, astrologers should at least agree with each other on which chart best matches each person. So did they? No, barely at all. The least experienced astrologers agreed with each other only slightly more than 20% of the time, which is close to what they’d get if they were actually picking truly at random. But even the most experienced astrologers only agreed with each other
About 27% of the time. This suggests that the methods of astrology may be much less systematic than many people believe. Perhaps a lot more goes into an astrological reading than the chart itself. For instance, the intuition and gut feelings of the person who’s doing the reading. If you believe you have astrological ability or you know someone who does, you can still take our test. Just go to our website clearthinking.org and take the astrology challenge. And if
You want to learn a lot more about how this research was conducted, just check out the link in the description below. And if you’re curious about our research that I mentioned earlier on the accuracy of different personality tests, we have a whole video about that that you can check out. And if you found this interesting, I’d really appreciate it if you’d subscribe to our channel.