What do we really know?

A fascinating report came out this week from a committee organised by the Academy of Medical Sciences, which I was a member of. It was prompted by the many and conflicting claims made about such things as the link between diet and cancer, MMR and autism or mobile phones and children, and tried to clarify when we can be confident about when X causes Y.  Much of the report is a careful summary of the state of the art in statistical analysis, and the relative virtues of randomised control trials, non-experimental survey data and natural experiments. Its main conclusion is that you should never believe a single piece of research.  Even when research is well-designed (and most isn’t) you should only become confident in findings when they have been confirmed by many different pieces of research coming at the issue from different angles. The report includes recommendations for the media, funders of research, and for government. Its confirmed two conclusions I reached in government. The first is that researchers shouldn’t be kept separate from policy-makers and implementers (which was one of the implications of the Rothschild principles established in the 1970s). Instead they need to be much more embedded – so that they can help decision makers interpret complex pieces of research.  The second was that every department should maintain a ‘systems map’ – a visible account of how it thinks about causation in fields like road traffic, unemployment, school results or cancer. These should make explicit what is and isn’t known, drawing on all the relevant disciplines – partly to guide decision makers but also to guide research funding. The recent Foresight review on obesity shows one way of doing these maps. But they are very rare – not least because they always reveal just how much isn’t known.