Statistics and medicine (Day – 279)

Statistics and medicine

So 75% of patients survived the surgery? Does this mean it is a good procedure? A great breakthrough in medicine? Something that every vet everywhere should be doing?

Well it depends…

The big craze that appeared recently is Evidence Based Medicine (EBM) where you use “evidence” from books and the latest literature to guide your treatment of a patient. Sounds great right as it means that everyone gets the same thing. However something that we are taught with research is that statistics are important.

So it turns out that if I have 4 patients and by pure dumb luck only one of them dies then statistically I have a 75% success rate. If I take a larger sample of 10 patients and I have a success rate of 75% then it means that 2 and a half patients died…

Yeah there is no such thing as a half patient unless we start talking about body and brain separately so maybe the third just suffered neurological damage.

However how accurate is it really saying 3 out of 4 survived? How much role did luck play? Or the environment? Or the skill of the doctor treating these patients? So does it really mean that if I attempt the same thing that 75% of my patients will survive?

However once you get out of the small sample sizes then you may get a better image… with 10 or so luck may have an influence… with 100 it may still be there however there will be smaller margin for luck to play a role… and when you start looking at studies with 1000 patients then you may definitely start to get reliable and repeatable results.

However studies with 1000 patients are difficult in clinical practice, and are also difficult when you are working with surgery techniques. Even for common things such as a spay, a vet may only do 1 or 2 a week so getting to the 1000 would take 10 years.

I always make the effort to read the full study to find out the sample sizes and the methods used. Statistical tests are notoriously difficult to understand and select the correct test to use, sometimes it appears that the test is chosen which shows a positive or statistically significant result even if it was inappropriate to use.

Reading studies in this way however takes a lot of time. And when there are a lot of studies then it becomes even harder. I’ve seen quite a few journal clubs pop up recently which are reviewing new studies in certain areas – however its almost like we need papers to review papers.

For now however I will try to focus on the papers that are directly relevant to what I am doing, and will take all my statistics with a grain of salt. Hopefully in the future there will be a public peer review system for vets where a journal article can be critiqued publically for all to then use.

Can the British Nutrition Foundation count? Possible error in survey results….

Error in British Nutrition Foundation survey! Fail!

Well today I couldn’t help but notice all the hype from a press release from the British Nutrition Foundation claiming that 29% of children thought cheese came from plants, and nearly 10% believe fish fingers to be made from chicken… So I was going write an article and share my insights with you so started my due diligence into the accuracy of this (as I do with every post) to ensure that my information was up to date and correct.

Now with a little searching (and waiting for their extremely slow website to load) I came across the original press release here:

So in the article (and on all the 100’s of websites that have syndicated it) the claim is that 27,500 children were surveyed for these results… The original release includes the geographic breakdown by country and Primary or Secondary school as follows:

1. The research was conducted among 27,500 children of primary and secondary school age across the UK. Samples from each participating country were as follows:

• England: Primary – 9,575; Secondary – 10,742

• Scotland: Primary – 1,794; Secondary – 3,763

• Wales: Primary – 323; Secondary – 449

• Northern Ireland: Secondary – 1,458

So obviously you’d expect these demographics to add up to 27,500….

Error in British Nutrition Foundation survey! Fail!I was pretty shocked when I added these up to get the portion of primary school children to allow me to work out the numbers behind the percentages to get 28,104. Believing I had made an error typing it in I did the simple addition again and got the same result…

I got a total of 11692 primary school children, with a total of 16412 secondary school children giving a total of 28,104 participants.

I’ve always been wary of statistics when used to promote a cause and in this case with such a simple error here, and doubt as to the actual number of participants the survey is called into question. As the introduction to the article says 29% of primary school children, and adding up the numbers of primary and secondary school children doesn’t give the quoted 27,500 its impossible to work out just how many children this 29% actually is…

Sorry British Nutrition Foundation, in this case I think you have some explaining to do….