[HSF] More on the Traysylol saga

[Mark Levinson, MD]

On Oct 5, 2006, at 3:21 AM, Ben Bidstrup wrote:

> The power that Journal  Editors have is such that they must be  
> answerable for their actions. ........ Now if a statistician who  
> has such an exalted post cannot indicate the weaknesses in this  
> study, then he should withdraw his paper and offer an apology, to  
> all readers of the journal, to the public and to patients who  
> suffered as a result of the non use of aprotinin. He should also be  
> joined in any action.

Ben:

I appreciate your comments.     We have discussed the role of  
statistical consultation for each manuscript submitted to the HSF  
journal during our Board meetings.    Although this seems like the  
answer for  blocking papers with bad science, there are problems.     
First of all, not all statisticians agree.    There are many tests  
that can be applied to the same data, of which the conclusions of  
each test may be different.    In some cases, one statistical  
consultant will claim a test is not even valid for the data, while  
the other insists it is the test of choice.    Who can arbitrate  
these differences?

Then there is the consistent dilmena of what to do with a statistical  
result.    Does a p value of 0.05 mean that the study has proved  
cause and effect?

I agree that statistical analysis is an improvement over the era  
where there was no statistics.   However, there are misused as  
well.   One of the problems is that now everyone intrinsically trusts  
the final conclusions of the statistical analysis.     If a p value  
is found, then readers believe it establishes truth, and even worse  
"cause and effect".    Some things that could not be related are  
forced into a relationship by the presence of a statistical result.

For example, if you were to perform a study about the mortality rate  
of intubation, you will find the hospital  mortality is higher for  
intubated patients than non-intubated patients.       Does this mean  
that intubation caused their deaths?    Even if multivariant analysis  
shows that intubation is still associated with a higher mortality  
rate, this does *not* establish cause and effect.   We all know that  
the cause of death is the disease process which ends up with  
intubation and the tube is not the cause.       Cause and efffect is  
only established be eliminating each variable and repeating the  
experiment until the observation goes away.     Since this is  
impossible in patient care situations, we utilize less perfect  
methods initially developed in the behavioral sciences and poorly  
adapted to our needs in surgery.

Recently there was a paper discussing the falacies of statistical  
analysis and centering on the t-test (the most common test used in  
reporting differences between groups).    I was amazed at how limited  
the scope of the t-test actually is.     After reading this paper, I  
realized that almost all of the t-tests we use are based on incorrect  
methods or assumptions.       So, we are standing on soft ground with  
even the most basic statistical analysis and we must understand this....


> Readers of the HSF who feel so inclined, should express their  
> dismay at this turn of events and send an open email to the editors  
> of the Journal and to Mr David Armstrong, (David.Armstrong at wsj.com)  
> the author of the article in the Wall Street Journal (Hal's  
> favourite medical source). Furthermore, you should encourage your  
> library to cancel their subscription to the NEJM. The Massachusetts  
> Medical Society should also be contacted.
>

Personally, the main failure of the NEJM is that they published a  
paper on a surgical subject when they are a medical journal.  They  
are no experts in surgical sciences and should have referred the  
author to another journal.    They are not set up to evaluate  
articles on surgical subspecialties.      The results analyzed were  
surgical endpoints.    I do not know who the reviewers were, but I  
doubt any surgeon, and particularly any cardiac surgeon, reviewed  
this paper prior to publication.    Also, whoever reviewed this paper  
exceeded the normal tolerance for personal diatribes in the  
discussion section.       Inflammatory statements made in the  
discussion section of the Mangano paper should have raised a flag and  
caused a secondary review, or be returned to the author for  
deletion.       It is my opinion that if the same paper has been  
submitted to the HSF board, it would have been severely criticized or  
rejected outright.    But the HSF has cardiothoracic surgeons on its  
Board while the NEJM does not.    And of course Mangano chose not to  
submit his paper to the Annals, JTCVS or another respected  
cardiothoracic journal.   I suspect he knew that such an audience  
would have no problem finding the weaknesses and rejecting the paper  
apriori.

Just my opinion....


Mark Levinson, M.D.
Founder, Editor-in-Chief
The Heart Surgery Forum®
URL:   http://www.hsforum.com
Email:  mmlevinson at hsforum.com