Mar 14, 2012

meat is murder

ARRRGHH!

"More red meat, more cancer, more heart disease"

Believe me when I say that I enjoy reading well written and well thought out research that challenges my beliefs. Not because I suspect I can shoot down the research (although I do get a buzz out of that) but because I'd rather know the truth than be right. Whenever I get really religious about some thing I believe I generally start trying to argue with myself. I do this because I've been really wrong on lots of things in the past. The only useful tool I gained by being a Wall Street analyst was a good boss who always asked challenge my beliefs and to release myself from any agenda I might have at being correct. Holding onto your beliefs in the face of provocative data was called 'anchoring'. So reading research that goes against my beliefs arms me with some ammo. One belief I have is that all unprocessed meat is not bad for you. And most of the processed stuff probably isn't bad for you either. It may not be good for you but it's not bad. Other things are bad but not meat.

So this article above was EVERYWHERE the other day. Well everywhere where people care about health (which is the blogs and journals I read). Vegans were dancing on Atkins grave. High fives all around at the Club Dean Ornish. But also in the Paleo groups and the Atkins groups. There were a lot of deer in headlights posts going on there. And why not? It's an awfully dangerous sounding study.  Let me count the ways:
  1. It's from Harvard.  They're smart aren't they?
  2. It's peer-reviewed and published in a good journal (Archives of Internal Medicine)
  3. As a basis it used two well known and well regarded studies (Nurse's Health Study and Health Professionals's Follow-up Study)
  4. No sample size issues - 83,644 women and 37,698 men were in the studies; followed for 28 years
And most importantly the main findings:
  1. Eating a daily serving of red meat increased mortality by 13% in the study period.
  2. Eating a daily serving of red meat increase your odds of getting cancer or heart disease by 14%.
  3. Eating a daily serving of processed meats increased those percentages to 20% and 18% respectively.
Guess what I had for dinner tonight?  

Steak.

You know why?  Because this study is entirely USELESS.  Here's why.

1) It is an observational study.
There are two basic types of health studies. Observational studies run by epidemiologists where a group is tracked and measured and Clinical studies where you randomly chose participants and control for the variation of a single variable.  Observational studies tell you nothing for certain because you aren't controlling for variables.  They are only useful for suggesting causal relationships which would then mean you need to do a clinical study.  I don't pay any attention to reports of observational studies.  You know why? Because they are almost always wrong.  Almost always. In fact I can't think of one that wasn't wrong. Does this sound like a random group of people in this study. "Nurse's Healthy Study"? Hmmm? No because it is tracking nurses.  Not the general population. 

Really you can stop right here.  This study is useless.  As are 95% of the studies reported in the press. Because they are observational.

2) This study was done by Walter Willett.
Willet was part of the team that put another report like this that is world famous in its wrongness among the health community who loves digging into research.  In particular it was Willet and others who started the whole 'estrogen reduces heart disease' speculation in the 80s. Even as a boy I remember it.  That finding was done using the SAME technique and the SAME Nurse's study. When a clinical follow up was done, estrogen INCREASED the likelihood of heart disease. Not decreased it. Increased it. Why?

3) Observational studies typically suffer from selection biases and these are very difficult to mathematically remove.
The problem with the estrogen nurse study was that there was a confounding variable. Or a variable which was the actual cause of people having lower heart disease and also for them taking estrogen. What was it? People who actively try maximize their health. If you were to section out a group of people from any group based on a simple question, "Do you actively make healthy choices", do you think those people live longer, have less heart attacks, etc? Yes. Because those people probably don't smoke, don't drink too much, don't eat ice cream all day, go see the doctor when they have a problem, don't sit on the couch watching TV 6 hours a day, take their vitamins, etc. These people also take estrogen. Or at least they did back then because it was supposed to lower chances of osteoporosis. These people had lower heart disease rates not because they took estrogen but because they were health conscious.  Because of that they took estrogen and did a whole bunch of other 'healthy' things. To be blunt, that fucks up your results.

Bottom line: The key reason everyone needs to be skeptical of public press reports of health research is not because the press sucks at reporting science. It's because health research is not conducted by scientists. Scientists do not do observational studies and report findings as fact and make health recommendations for everyone. They don't do that. Idiots do that. Scientists determine facts and understand causal relationships. And then you do your damnedest to prove yourself wrong. You don't get to guess in science and then report it in a journal.

So what could be going wrong here. It could simply be the same effect. Healthier people eat less meat. Not because they eat less meat but because they do a myriad of other things that are healthy like exercise and don't smoke and so forth. In fact it has already been pointed out in this very study that as you go from the healthiest quintile to the most unhealthy quintile all the other parameters of health get worse. Things like BMI, blood pressure, smoking prevalence, amount of alcohol consumed, lack of exercise, etc. For example smokers in the healthiest quintile were 5% of the group but 15% of the unhealthiest quintile.  Could this explain most of it?  All of it? More than all of it implying that the meat eating kep them healthier?

The FUNNIEST variable tracked however was cholesterol.  It actually went down as you moved to the unhealthiest quintile.  15% of the healthiest quintile had high cholesterol yet only 8% of the unhealthiest quintile had high cholesterol.  Everyone still think cholesterol has anything to do with heart disease? Let's hope not.

Vegetarians, even though I think they are doing it wrong, get a lot of stuff right outside of their diet. Ultimately what you are comparing is vegetarian, non-smoking, non-drinking, annual checkup, yoga mat toting, marathon runners to overworked, overstressed, lower income class, beer swilling, never go to the doctor, watch 6 hours of TV a day, steak eaters. 

You can see where I'm going with this. Public health policies get reinforced this way. It's really exceptionally dangerous stuff. 

4) Data is collected verbally. 
The dietary intake in the study was not directly measured. It was reported by participants in questionnaires. "Mr. Johnson did you have a cupcake yesterday at your son's birthday party". "No ma'am. Not me."

5) For an observational study the effect isn't that large.
It may seem large but the relationship between things that we have done clinical studies on have much higher increases in effect. Things like smoking and lung cancer or radiation and cancer.  In those cases where clear causal effect have been identified the increases are enormous. Thousands of percent higher rates of incidence.

Even more damning is that this rollover effect should have had a larger effect.  The case of healthy people doing both actual healthy and actual unhealthy things that are collectively thought to be healthy should produce a larger effect.  It actually suggests that the meat part of this might actually be reducing the approximately 20% mortality increase from something higher.

6) Only clinical trials matter. Randomized controlled trials (RCT) 
And these have been done in regards to "Atkins" type diets in comparison to others.  How have they turned out? 16 out of 16 RCTs show the lower carbohydrate studies win.  Not only in weight loss but in blood metrics, other disease risk factors, cognitive performance, and others.

See The Diet Doctor's compilation of these studies here.

More Reading:

No comments: