So 75% of patients survived the surgery? Does this mean it is a good procedure? A great breakthrough in medicine? Something that every vet everywhere should be doing?
Well it depends…
The big craze that appeared recently is Evidence Based Medicine (EBM) where you use “evidence” from books and the latest literature to guide your treatment of a patient. Sounds great right as it means that everyone gets the same thing. However something that we are taught with research is that statistics are important.
So it turns out that if I have 4 patients and by pure dumb luck only one of them dies then statistically I have a 75% success rate. If I take a larger sample of 10 patients and I have a success rate of 75% then it means that 2 and a half patients died…
Yeah there is no such thing as a half patient unless we start talking about body and brain separately so maybe the third just suffered neurological damage.
However how accurate is it really saying 3 out of 4 survived? How much role did luck play? Or the environment? Or the skill of the doctor treating these patients? So does it really mean that if I attempt the same thing that 75% of my patients will survive?
However once you get out of the small sample sizes then you may get a better image… with 10 or so luck may have an influence… with 100 it may still be there however there will be smaller margin for luck to play a role… and when you start looking at studies with 1000 patients then you may definitely start to get reliable and repeatable results.
However studies with 1000 patients are difficult in clinical practice, and are also difficult when you are working with surgery techniques. Even for common things such as a spay, a vet may only do 1 or 2 a week so getting to the 1000 would take 10 years.
I always make the effort to read the full study to find out the sample sizes and the methods used. Statistical tests are notoriously difficult to understand and select the correct test to use, sometimes it appears that the test is chosen which shows a positive or statistically significant result even if it was inappropriate to use.
Reading studies in this way however takes a lot of time. And when there are a lot of studies then it becomes even harder. I’ve seen quite a few journal clubs pop up recently which are reviewing new studies in certain areas – however its almost like we need papers to review papers.
For now however I will try to focus on the papers that are directly relevant to what I am doing, and will take all my statistics with a grain of salt. Hopefully in the future there will be a public peer review system for vets where a journal article can be critiqued publically for all to then use.