Tuesday, 4 November 2008

No need to shoot the TV to save the kids.

You see stories like this crop up every now and again: "TV Sex content linked to teen pregnancy rate!" NPR, BBC, Fox, etc, all covered various versions of this story this last week. Previous iterations have been violence on TV, whether you dream in black and white or colour, and so forth. It makes for good news for them, because just about everyone has a TV, and many people have children to be protective of. Also, it makes them feel sciency.

However, it's long been a complaint of mine that science reporting is crap. I mean, completely and utterly bum. Even in other countries. Their treatment of these studies only reinforces my worldview that they could properly report science to save their lives. Let me illustrate with an example:

Say we're measuring two variables, number of leaves and number of berries. We're doing an observational study, so we can't tinker with things. Instead, all we can do is copy down that plant a has so-many leaves, and so-many berries, plant b has blah, plant c blah blah etc. When we're done, we plug all our numbers into a big stats package, and we find out that the number of leaves can be used to predict the number of berries. This is to say that they're correlated, or linked.

Now. What the media would have us believe from their reporting that the factors share a causal link with eachother - leaves cause more berries to be produced. It might even make intuitive sense: "Oh, well, plants need leaves to photosythesize, so if we put more leaves on, we get more berries." But we haven't demonstrated causality. You can't demonstrate causality through corrolations, because of the `un-measured third factor.` The thing is, there could be a third factor that drives both berries and leaves. In our example, the third factor could be light. It migth be that you could add leaves to your hearts content, and you wouldn't get any more berries. Or, it could be soil nutrients that vary both the leaves and berries, and you could remove leaves and still get berries. These 3rd factors are impossible to exclude in corrolational studies.

Let's go back to Sex and the TV. The media is reporting this as if watching risqué content on the TV causes teenage promiscuity. However, they can't demonstrate this causality. There might be a third factor driving both - for example, consider the hypothesis that randy teens tend to watch more sexually explicit content. They're not randy because they watch TV, but rather they begin hormonal, and therefore could be drawn to more explicit content. This inherient predisposition could also causes them to be more promiscious. They cannot preclude this possibility with their study design.

We saw shades of this with violence on TV - who watches more violent TV? People with high stress hormone levels to begin with. But we're told that seeing a gun on TV is a driving factor in being violent ourselves - something that is clearly not `proven` by any use of the word. Without exceptionally clever design, I wouldn't go any further than saying they're suggestive of something, and only when there's previous, compelling evidence for a mechanism. The media, however, puts about as much time and effort into getting the details of science stories right as they do juggling chainsaws. I.e., not much.

No comments: