How much evidence should be demonstrated before enacting obesity-targeted health policy? This difficult question was debated between two speakers Sunday at ASN’s Scientific Sessions and Annual Meeting, as part of the Obesity Research Interest Section Forum, chaired by Andrew Brown, PhD.
The first speaker, Laura Schmidt, PhD, defended the need to set such policies early on. According to Schmidt, one reason we cannot afford to wait for perfect evidence is because the National Nutrition Research Roadmap indicated that the average amount of time that it takes for research to go from bench to bedside/community is 17 years. Dr. Schmidt noted that how research is translated to policy needs to be strategic and carefully considered, the evidence must be robust and systematically reviewed, but that science can only inform, not drive policy decisions. Often it is not possible to enact policy at exactly the specifications that research suggests. For example, a 20% tax on sugar sweetened beverages was indicated to show an effect on sales, but San Francisco couldn’t get a bill passed into law until the tax was reduced below 20%. Standards for evidence are often higher in the scientific community, she said: they utilize systematic reviews, expert panel summaries, and formal guidelines by federal and global agencies. When the results of a large number of different types of studies that use different measures and outcomes point in the same direction (i.e. observational, clinical trials, and mechanistic), we can be confident in the strength of the evidence. Schmidt gave an example of some issues she perceives haven’t reached a body of research big enough to act on yet: taxing 100% juice or diet sodas, even though there is emerging concern on each from the literature. Finally, Dr. Schmidt noted that we need to be concerned with industry funded research as some evidence suggests it may bias conclusions on a topic. To summarize: we should acknowledge the need to translate research in ways that can inform policy and that best practices and standards for evidence-to-policy are shaping up, but that challenges remain, including scientific bias due to conflicts of interest.
As a contrasting perspective, Michael Marlow, PhD, outlined his concerns with setting policies without a very high level of confidence that they will succeed. In other words, caution must be exercised because researchers don’t yet know optimal policies. He outlined his concerns as follows: 1) There are good intentioned hunches over scientific exploration. Confirmation bias and common narratives may lead to policies that don’t reflect reality. 2) Many research methods promote type-I errors, such as P-hacking, often a consequence of tenure and grant requirements and journal editor demands. 3) The quality of dietary data is poor. Dr. Marlow pointed to a study that found that for 95% of a study sample, fast food, soft drinks, and candy had no association with BMI. There are a number of possible interpretations to this: the data sources (diet recalls) may be so seriously flawed that it is ok to advocate laws that only affect 5% of the population. Should we enact policy or wait until data collection is improved? And, 4) There is naive modeling of interventions that goes into estimating policy efficacy. Linear relationships are often assumed between availability of nutrition information and behavioral changes, for instance. Because of what is overlooked, Marlow’s simulations of policy success range from 6.25% with optimistic probabilities of effects to 0.01% with less optimistic assumptions. To summarize: policy proposals need solid theoretical and empirical support, data quality needs more attention and acknowledgement, measures for policy success need major rethinking, uncertainty and unintended effects need acknowledgment, and we need to resist ill-advised albeit good-intentioned policies from citizen pressure.
There is no easy answer to how long we should wait before enacting health policies that target obesity. Translating research to policy is difficult and policy as a natural experiment can help us understand if we can impact obesity. A common thread of agreement is the need to ensure that we have high quality research methods and to reduce bias wherever possible. Perhaps then the question would be easier to answer.