A little discussion on Twitter led me to elaborate on red flags. Back in the days when I taught at a residency program, the Internet was exploding and it was important to apply the principles of critical assessment in order to evaluate the credibility of such vast access to new information. These are the same skills and principles we use in evidence-based medicine (EBM). The whole thing begins by not believing what you hear or read and being systematic and rigorously in analyzing the reasons not.
This Twitter interchange pretty much followed the same pattern.
Shelley Petersen is a London based journalist and social networking marketer who focuses on innovative medical technologies and maintains an active twitter feed chock full of medical science articles. One of her articles raised a lot of red flags for me and when she asked me what those flags were I thought it was too much for 140 characters. It was time to tramp back over the article and provide a detailed analysis of what struck me as improbable and unreliable about the information.
The
article was from a Pittsburgh paper and was fairly balanced, presenting some contrary perspectives, so the red flags were about what the principal was saying. In brief, the CEO of a private company was discussing how a long recognized function of telomere shortening and how it explained aging and all the diseases associated with aging.
1. Everyone has an inherent bias, mine is to be skeptical. The CEO of a company is to sell something. In this case, I suspect it is to sell a narrative to investors. I am dubious at the get-go, so red flag number 1.
2. Most research happens at universities. Scientists know that some of the best research happens at smaller universities, but when the only experts cited in any article are from Harvard and MD Anderson, I begin to suspect spin. Certainly I would expect some degree of balance between “name” institutions and other excellent university departments which are less of household names. Over-citing can raise flags.
3. Telomeres are not new. Telomere shortening is not new. When one individual claims he has insights that nobody else has been smart enough to figure out over decades, it is wise to be suspicious. Moreover telomere shortening is an epigenetic phenomenon. Nobody believes that epigenetic phenomena occur at the same pace everywhere in all tissues at the same time. Telomere shortening in fibroblasts does not predict telomere shortening in ovaries, or any other tissue. So wildly expansive claims should leave the reader incredulous that decoding this one phenomenon will end aging, cure all cancer (where telomere shortening may eventually pan out to a mechanism in one or another subtype of cancer), get rid of heart disease, cure wrinkles and fix your stiff knees..
4. Activating the immune system is a phrase that is a throw-away for naturopaths and quacks. Ask anyone with allergies or rheumatoid arthritis. An active immune system is not a good thing and depending on your definition, may well lead to a cancer, like say… lymphoma, a cancer of the immune system. Let’s face it, we all need a well-regulated immune system, but “activation” is meaningless in any scientific context.
5. Anyone who tries to market an innovative pharmaceutical product as a supplement is selling snake oil, plain and simple. The FDA evaluates specific therapeutic claims for specific pathological entities. They do not even evaluate absolute efficacy, drugs only need to demonstrate superiority over placebo. No such regulation exists for supplements, so many companies that do not have the ability to go through a rigorous review and demonstrate scientific merit will circumvent the process and come to market as a supplement. With the notable exception of nutritional deficiencies, there is usually no merit to claims that vitamins, supplements or minerals cure cancer, heart disease or any other disease, except maybe obesity and weight loss issues supplemented by means of
ECA stack, etc. The idea that taking something as simple as a supplement may be appealing but has no basis in fact, since it is still an exogenous product, natural or otherwise.
The principles of critical assessment are to assess date, author, quality of the publication and citations. Biological plausibility is important in assessing a scientific article, along with the link between the data and the claims made. This can be applied to journalistic information as well and when claims are expansive or vague, skepticism should take over. Any attempt to circumvent well-established processes to verify and certify scientific findings completely invalidates the speaker’s credibility in my mind. Being skeptical means you will believe only a small fraction of what you read, but this is not necessarily a bad thing. There is very little truth in much of what you come across on a daily basis, but there is plenty of hype and plenty of reasons for people to try and attract your attention.
Shelley’s work exposes me to a wide variety of information, which I find useful. The article itself was well-balanced and prudently presented, but the company and the spokesman were presenting snake oil under the guise of science.