Studies show that correlation ≠ causality
July 7th, 2008bullshittery, drugs, Jamie Lee Curtis, parenting, psychology
I can’t tell you how many times I’m reading news online and I come across some crackpot study by a “psychologist” making a hyperbolic claim with ridiculously weak data that may or may not even relate to the claim.
Well, it happens a lot.
Just a couple months ago there was an article on mixx entitled “smarter women have bad sex” that was a perfect example of this bullshittery. I blew that story out of the water with mixx’s official “greatest comment of all time,” but this kind of thing happens a lot. (Not the great comment, that’s Honest Ape’s domain, I mean the crackpot study.)
I think about this sometimes because I’m interesting in psychology, but it wasn’t until yesterday when I was listening to NPR that I realized its true danger. There was some really cheesily done public service announcement with Jamie Lee Curtis talking about how “studies show that kids whose parents eat dinner with them regularly are less likely to do drugs” or something. I scoffed at the radio and ranted to Brennan all that way home about how misguided they were.
First of all, the study she mentioned was more than likely a non-experimental, correlational design. (Wordpress doesn’t think correlational is a word, but it is, I promise.) That means the psychologists went around handing out surveys asking people how often they ate with their family and whether or not they or their kids did drugs. This type of design is not highly regarded in psychology because it’s very weak. Your results rely entirely on what people decide to tell you and you just have to assume they’re telling the truth while remembering correctly. A lot of parents don’t have a clue what their kids are doing, so collecting accurate data for this particular study would be nearly impossible. A psychologist can pretty much make any claim they want about people and “prove” it with such a design. Sometimes longitudinal studies are taken seriously, but I highly doubt this study was such.
Secondly, correlation does not equal causality! People have such a hard time with this for some damn reason. Even if it is true that families who eat dinner together are less likely to have children with drug habits, it’s not a testament to the magical power of eating dinner together. Eating dinner together is a lot more likely to be a symptom of a healthy family for a wide variety of reasons; parents who make the effort for family time are involved, caring, supportive, etc. If a family is already hopelessly broken, then forcing the kids to the dinner table is only going to aggravate the situation.
The final reason why these studies are dangerous is because they lead people to believe that small actions can solve problems that require a great deal of work. If you’re a parent who has a kid on drugs, please don’t ask them to dinner and think you’re doing something about their drug problem. It takes a lot to overcome or prevent addiction and, in my humble opinion, psychologists who lead people to believe otherwise are a shame to the profession.
Very well said, Julie! I bitch about such things to my wife all the time. So many studies are flawed in this way. Like studies that show that kids with a father in their life will be more rounded, which means Lesbians shouldn’t raise kids. Well, how many studies have been done for kids with two moms? Maybe it’s the two loving parents working together that make a difference, not because one’s a man.
People pick and choose data to get their personal points of view across. Shit, according to commercial, a study said that eating Sugar Pops (or some sugary cereal) in the morning makes your kids do better in their first class. No shit. Ever hear of a sugar rush? What they don’t tell you is what the kids are doing in their second and third class. Sleeping.
Your blog is too well written. Makes my blog look like fail.
@honest_ape
Oh come now. Comparing our blogs is like comparing fresh apple pie to smashing people in Halo. Both have a special place in people’s hearts, but not the same place at all
I love both of you equally………….My mom would say to me and my brothers, but yeah Julies blog is the better
Sike! no for reals I love both of your blogs and they are apples and orangutans. Two different beast, but both swift of feet and strong in heart….Ace’s
Thanks for the link, Julie!!!
Also, as a sociology minor in college, I totally feel your pain as far as causality/correlations. I read “Freakonomics” recently (cuz apparently I’m kind of an econ geek too? I don’t know) and it mentioned this very thing, and how confusing the two is like assuming that cities with both lots of police and lots of crime should get rid of the police. (Well, maybe that’s not the worst idea for L.A.)
First of all you should be ashamed ….
… for saying bad things about JLC
In the way you frame it here, you’re absolutely right. There’s a lot of junk or pop science that follows the maxim, if it feels good do it, or believe it.
A lot of it just FEELS right and if you want to or are predisposed to believe it, you will.
For example it sounds really good to say that “studies show that kids whose parents eat dinner with them regularly are less likely to do drugs” because what it’s really saying is, attentive parents are healthier for families. And, of course, largely this is true and there ARE long-term studies for that. But to specify “dinner” and “drugs” is to render the statements false, or at least on less firm ground, as you say, with the truth unknown.
smarter woman have bad sex doesn’t even make real sense on the surface. But again, it sounds “good” and has a well known idea attached to it - that men are scared of smart women therefore it’s better to play stupid (blah, it didn’t even feel good to type that). smart women MAY have less sex because they other things to pursue; they MAY be able to find men they don’t respect on all levels so the sex is correspondingly less satisfying and doesn’t lead to O ETC ETC. But that’s not even what the study is about.
Better lovers equal orgasms, but sometimes even then there are, yes psychological things going on.
You do make the assumption that psychologists were behind both of these studies. The smarter sex article appears in the Sun Magazine which is an authority on Page 3 girls’ nipples and not much else. The NPR PSA could have been done by psycologists but mre likely it was sociologists or soe other research.
So your main point: that it’s too easy to say shit that don’t mean shit I agree with completely.
(• Everyone, did you like the comment you just read? You can subscribe to my newsletter and buy my t-shirt, slightly smelly after a long day. Or you can go feed yourself.)