Doctors and the public on health

The Sydney Morning Herald reports on a study comparing doctors' and tradesmen's knowledge of nutrition and exercise. The full study is here, from Australian Family Physician, a publication of the Royal Australian College of General Practitioners. Table 1, below, has the results.


Six of twenty questions were answered incorrectly by a majority of doctors, compared to eight of twenty for the tradesmen.

The authors don't run a t-test for differences in mean group responses. Across the set of all questions, we can reject equality with a p-value of 0.017 on a two-tailed test: doctors scored better than the public, with an average "percent incorrect" score about 10 points lower than the public (median difference 6 points). Doctors' mean % incorrect fell below the public's 95% confidence interval nine times out of twenty and was never above that confidence interval.

Another problem: the authors note that "many of these statements remain controversial", with the authors picking "true" or "false" based on their best read of the evidence. But surely they'd then need to assign a confidence interval to their own estimates of whether the statement is indeed true or false. If the state of the literature is that about 70% is consistent with the statement being true, and so the authors label the statement as true, it wouldn't be shocking to find 30% of doctors disagreeing with the statement and being labelled "incorrect". But where the relevant measure is the difference between the public and doctors, this shouldn't cause too many problems. A final problem: both groups were surveyed while going into some seminars on health. We might expect that the doctors going to these kinds of seminars might be drawn from the lower tail, but the tradesmen from the upper tail, of distributions of prior knowledge for their relevant groups.

All that said, though, not a great showing for the doctors. Normally these kinds of surveys (Caplan, Althaus) will build a measure of the "enlightened public": how members of the public would have responded had they had demographic characteristics equivalent to the expert group but not the expert training. That would likely be impossible in this case as there's almost certainly no region of educational overlap across the two groups: you need at least a few members of the tradesmen group who have about the same number of years of formal education as members of the doctors group. Doing that usually closes the gap, at least somewhat, between the "enlightened public" and the experts. But the gap here isn't all that large.

I'd suggest that the folks at Alcohol Action NZ note the "Fruit juice is about as fattening as beer" factoid, but that might encourage the creation of Fruit Juice Action NZ. Hit the link before you laugh, and note especially the story linked at the end.

HT: Peter Martin's picks