Bolstering results from a 2011 study, a team led by Buck Institute researcher Gordon Lithgow found the compound Thioflavin T to increase the lifespan of a variety of strains of nematode worm, some of them enjoying double as many days alive as their control counterparts. Thioflavin T (ThT) is popular in labs as a stain for clumps of amyloid protein, which can be found in brains with Alzheimer’s disease and have long been suspected as a major culprit behind the disease’s origin. Don’t expect to ever see it on the shelves of your pharmacy, though–ThT is toxic at a dose just barely above the one required for its longevity benefits. However, the result should spark interest in investigating chemicals with similar properties.
The compound beat out nine other substances, including resveratrol, the antioxidant quercetin, and aspirin, which had all shown promise in the past as anti-aging interventions. Several had variable efficacy across worm strains, lengthening some lifespans and shortening others, while some seemed not to work at all.
Though most of the interventions tested were far from resounding successes, the truly impressive aspect of this study was its design: conducted across three independent labs and with 22 strains of the roundworm Caenorhabditis, the researchers tackled replication and questions of cross-species applicability all in one go, displaying a rare commitment to rigor in a field with a long history of poor replication.
Following in the NIA’s footsteps
The NIA’s Interventions Testing Program, begun in 2003, was the inspiration for Lithgow’s Caenorhabditis Interventions Testing Program. Each year it tests four to seven promising therapies on mice in three separate labs, and has so far confirmed immunosuppressant rapamycin as an effective treatment, with less positive results for aspirin, green tea extract, methylene blue, and many others. Due to the expense of conducting the trials, usually only one dosage is tested.
The CITP diverged from the ITP by testing multiple dosages of each substance, capitalizing on the cost-effectiveness of invertebrate models. And in contrast with the NIA’s stock of closely related mice, the worms’ relative cheapness allowed the CITP researchers to diversify their test subjects, using strains as genetically distant from one another as humans are from mice. The hope was that this would increase the likelihood that positive results reflected improvements in aging pathways that were conserved across species, and could eventually be translated into human models.
If a less genetically diverse test set had been used, it’s unlikely the researchers would have caught the confusing pattern shown by the dietary restriction mimetics. Designed to simulate some of the molecular mechanisms of caloric restriction, DRs such as resveratrol work on metabolic and nutrient sensing pathways, and many scientists hope they will be able to stave off heart disease, cancer and even dementia. But of the three substances tested, none showed clear benefit to the worms, and some strains’ lives were even shortened. The researchers hypothesized that the apparent difference in response was due to genetic variability in metabolic set points, and that some of the worms may naturally exist in a state close to DR.
Costly mistakes and unreliable results
Well before it failed the NIA’s ITP in 2012, resveratrol was briefly a media darling and favorite anti-aging compound when it was found to extend the lifespans of yeast and mice, but fell from grace as increasingly troubling reports of failed replications and scientific misconduct emerged. It’s only one of the more famous in a long line of disappointments, both within the field of anti-aging science and in the broader scientific community, resulting from the substandard methodology and poor coordination that’s led to the “replication crisis”.
Flawed and irreproducible research has been estimated to cost the US $28 billion per year, a burden that’s mostly borne by the people trying to translate findings from the lab into real-world therapies. Low chances of replication success–less than 50% on average, and as low as 10% in fields like cancer biology–hamper commercial drug development. Since pharmaceutical companies must invest between $500,000 and $2 billion replicating preclinical research, the hottest new findings are likely to seem more risky than tempting, which delays the introduction of many genuinely useful interventions to the market.
So what’s so hard about reproducible science? For starters, reporting on a study’s methods in sufficient detail can be time-consuming, while not doing so makes it impossible for other researchers to conduct exact replications. Issues with data analysis and reporting, as well as non-standard practices and experimental materials, are also common. And in any case, there are few real incentives for scientists to replicate research when journals regard such papers tepidly at best.
One problem especially salient to longevity research is that interventions with small effect sizes, the overwhelming norm in purported aging remedies, are particularly unlikely to replicate in small studies due to random chance. Compounding this difficulty is the fact that the aging process is complex and may be highly sensitive to subtle changes in the test environment and procedures. Using tightly controlled methodology and materials, the CITP researchers were able to achieve nearly identical average outcomes on longevity measures across all three labs, but within labs the picture was less clear:
“Although the results obtained on any given day of a replicate trial tended to be fairly consistent with one another, conducting the same assay a month later could yield results as different as looking at a strain from a different species.”
The evidently fickle forces at work highlight the need for multiple replications in longevity trials, a rallying cry Lithgow and colleagues hope reaches the ears of fellow researchers, and with any luck study designs like that of the ITP and CITP will only grow in popularity.