Dennis Bier, MD, was this year’s W.O. Atwater Memorial Award Winner and Lecturer at ASN’s Scientific Sessions and Annual Meeting on Tuesday, sponsored by USDA-ARS and ASN. His talk, titled “Traveling the Road From Precision to Imprecision- Have I Gone in the Wrong Direction?” was a broad critique on the state of nutrition science, contrasting his long research career using isotopics for precise kinetic measurements in humans to some of the more imprecise techniques being used in nutrition. Human nutrition today remains an immature science, says Dr. Bier, because it is so difficult to accurately measure what people eat.
Among his critique along this line, he noted that individual nutrient intake measurement methods are “validated” against other imprecise methods, and even these correlations are generally weak within the 0.2 to 0.5 range. Further, it generally takes between 5-15 24-hour diet recalls, sometimes many more, to achieve an adequate estimation of nutrient intakes in overweight and obese participants, which is never done in practice. Another problem is the seasonal variability of nutrients in foods, using vitamin C as an example, or variation from changes during processing and cooking. If such variation is considered in epidemiological studies associating foods with disease outcomes, it greatly reduces the power to detect a statistically significant result, or can change a result from a positive association to a neutral one.
Should we think about nutrition as a science in the same manner as other disciplines like physics? Bier thinks we should. The differences are obvious now. It is extremely difficult to do long-term experiments in people for many reasons of practicality and cost. But if we concede to this difficulty, our confidence in certain areas of nutrition science will remain stagnant. There are some examples where observational data seems to have led us astray, for example with vitamin E and a follow-up negative trial, and similarly with homocysteine reduction with folic acid and a follow-up negative trial. Dr. Bier noted that more than 15 randomized controlled trials have failed to support nutrient hypotheses generated from observational studies of food intake. There may be many reasons for this: differences in subjects, inclusion criteria, dose or duration, therapeutic window, etc., but we also must consider that the observational data may lead us astray. There are underappreciated interdependencies in observational variables that cannot all be statistically accounted for. Bier’s cautions on such data should be uncontroversial. Associations uncovered in observational studies are hypothesis, they cannot infer causality, and drawing conclusions from them are fine as long as their uncertainty is acknowledged where they are used. In practice, however, these rules are often not followed.
Bier concluded by talking about the many issues that plague science in general that make the literature less reliable. For instance, 95% of the biomedical literature contains a significant result, suggesting a severe publication bias in favor of positive results. Other issues arise with a large researcher degrees of freedom; that is, the number of choices the researcher makes when designing a study or analyzing the results. Asking a lot of questions, changing the primary endpoints, focusing on positive endpoints and discounting negative ones. In reporting of results, implying causality from associations is often done inappropriately, or inflating the effect of the finding by only reporting relative risk instead of absolute risk.
“Transparent science, like transparent government, means releasing your tax returns.” Dr. Bier wants the field to think more about reporting. Among a long list of ways that scientific integrity can be improved: require a priori registration of all studies and a data analysis plan, report all primary endpoints together, require perspective in reporting of the results (e.g. effect sizes, confidence intervals, absolute risks, NNT, etc.), mandatory use of reporting guidelines, report alternative analyses (e.g. alternative models that fit the data with equivalent statistical confidence), making the original data available for scrutiny, and improving the disclosure of conflicts of interest. Everyone has some form of conflicts, for example, money, grants, fame, etc., and Bier thinks that allegiance biases are at least as common as financial ones, so we need to come up with a universal conflict of interest system to make this standardized and fair.
While Dr. Bier’s views may be too idealistic for some, he promotes a necessary conversation about how we improve nutrition science and reduce our uncertainties. To that end, the field should continuously strive to enforce policies and practices that better our measurement techniques and limit bias.
