How We Read Scientific Studies

The science of genetic engineering is rapidly evolving, and it is changing the environment, agriculture and food manufacturing and consumption in profound ways. New applications of existing tools to genetically modify plants and animals are under development around the world. Novel and more precise genetic engineering methods will open new horizons for commercial applications, while simultaneously raising new questions about environmental and human health impacts.
Projecting and managing the performance, costs, and risks arising from genetically engineered (GE) crops, animals, and agricultural inputs is a complex challenge. The tools and science available to accomplish these tasks have lagged behind the excitement over new market opportunities.
New issues will arise as more crops are genetically engineered, especially in the case of fresh fruits, vegetables, and other whole foods. Both the nature of GE food risks and the challenges inherent in identifying and quantifying them are rapidly evolving. The very notion of “safety” remains murky in the case of many GE crop technologies.
As physicians and scientists, we read health-oriented scientific studies every day, looking to gain deeper understanding of the roots of disease and to find better ways to preserve and restore health. The content of this website draws upon our collective experience in interpreting science and helping patients.

How science advances

The scientific method involves making a hypothesis and designing a study to support or contradict this hypothesis. If a substance is given to test animals in a long-term feeding trial and we find that their kidneys and liver have normal weight and appearance by the end of the study period, all we can say is that under the conditions of this experiment, there was no evidence of an adverse or toxic impact on these vital organs.
If a number of other experiments involving different species of animals, conducted in different labs using different dose levels and perhaps alternative routes of exposure all reach essentially the same conclusion, the weight of evidence supporting no appreciable harm to the kidney and liver could reach a point where scientists and regulators conclude that the substance poses little or no risk to these organs.
However, a study that looks more carefully into liver and kidney function, and not just appearance, may reach a conflicting conclusion. The first step is therefore to carefully review study protocols, records, and analytical and statistical methods in order to identify why study results differ. It usually takes several more studies to fully understand what happened and why. In some cases, questions linger even after dozens of studies and heavy investments in specialized studies.
Dealing with uncertainty is an inherent and unavoidable part of risk assessment and regulation, and indeed of day-to-day life. The challenge for policy-makers is specifying the rules governing what happens in the face of uncertain science and potential risks of unknown severity, reach, and longevity.
With advances in analytical technologies and risk assessment tools, we can look at entirely different aspects and interactions between genetics and lifestyles, and between biology and health. Such advances are welcome and needed, and will hopefully allow public health scientists and regulators to avoid any unwelcome surprises following approval of GE crop and animal technology.

The nuts and bolts of sound risk assessment

The best type of experiment is a double-blind placebo-controlled study in a well-characterized human population. For example, if a team wanted to assess the safety of a GE corn trait presently on the market, the scientists would need to take a large number of people and separate them at random into two groups.
The simplest experiment would study a specific strain of GE corn grown under typical conditions (this already involves a number of assumptions). One group of individuals in the study would consume a defined amount of the GE corn over a specified period, while another group would consume the same amount of corn, but from ears harvested from a field planted with the GE corn’s isoline (the same background genetics, but without any GE traits).
People participating in the study would not know which group they are in. The team would then decide the types of impacts from ingestion of the corn that warrant tracking and perform comparisons between the groups. Possible impact parameters would include changes in weight, metabolism, blood pressure, and issues related to the gastrointestinal tract.
In some cases where non-invasive tests point to abnormal organ function or worrisome changes in physiology in some treated individuals, the team might call for a biopsy of intestines, kidney, or liver. An MRI scan might be carried out to look at organ size, or special tests ordered to assess changes in the gut microbiome.
Scientists can screen blood for changes in the levels of a number of hormones and proteins that are indicative of a given biological response or progression toward disease.
In order to avoid bias, researchers on the team examining samples of tissues and conducting histopathological assessments need to be unaware of which group the samples came from (this is the second “blinded” aspect of a double-blind, placebo-controlled study).
The team would then analyze the data and compare impacts in the two study groups, looking for treatment-oriented differences.
Human feeding studies require strict control of diet. Any straying from diet protocols dilutes the statistical power of the study (i.e., the ability to detect differences in outcomes, if they indeed exist). For the best quality outcome, such research is done in a residential setting where all the food is provided and intakes are controlled and measured. While similar studies are routinely conducted in the course of testing new medications, they are rarely done to assess environmental or food safety risks because of the very high costs entailed in conducting a long-term human feeding study.
Because of the difficulties inherent in human feeding studies, among other reasons, it is appropriate and necessary for medical researchers to use biochemical markers and other indicators of disease-risk rather than to study just the diseases themselves.
Such markers might be antibodies – natural chemicals that indicate a targeted immune response – or changes in the composition of gut bacterial species. This approach can be difficult to understand by people who lack the technical training and considerable experience necessary to interpret how changes in these markers may lead to, or correlate with, disease. This is understandable, given the complex linkages between dietary choices and food intake on the one hand, and human health and disease on the other.
Contributors to many popular websites often criticize the characterization of human health risks through changes in markers as being too indirect and prone to error, and sometimes even of leading to far-fetched conclusions. While it is certainly possible to misinterpret the cause or significance of changes in marker levels in certain patients, the medical literature contains thousands of studies reporting marker-based results that have advanced the understanding and treatment of a variety of diseases and chronic health problems.
Studies focused on markers have now conclusively shown that the gut microbiome is a powerful determinant of health.^[1] It is therefore a crucial parameter to measure in the context of the human diet and GE foods. While all scientists would prefer results from well designed, randomized-controlled double-blind studies as outlined above, we have to accept that such studies very likely will never be conducted on GE crop traits because no one is willing to pay for them.
Without this evidence, however, assertions that foods derived from GE crops have been proven safe are premature.
In top-tier scientific journals, debate and re-examination of assumptions and accepted conclusions is active and ongoing. Critical analysis of research methods is also essential. For example, in July 2014, the Journal of the American Medical Association published an article on how doctors should use information from meta-analyses (popular research methods that integrate the results from several studies on the same topic). They sound a note of caution: even a tool that appears very powerful can have its drawbacks.^[2]

Conflict of interest

The more comprehensive the research, the more expensive it can be. Bias can find its way into studies through many avenues, some inadvertent and others intentional. This is why all reputable, high-impact journals utilize rigorous peer review and have conflict of interest and funding disclosure requirements. These mechanisms can serve to promote transparency and can alert readers (and reviewers) to be on the lookout for “devils in the details.”
Journals’ best efforts to combat bias and to disclose conflicts of interest do not make up for the lack of sufficient studies. GE crops, traits and foods need to be studied more thoroughly, and that should be done by government-funded scientists lacking any sources of bias or conflicts of interest. Currently the agencies responsible for regulating GE crops and the pesticides required for them to work rely completely upon safety studies sponsored by the developer. Neither the EPA nor FDA independently carry out risk assessment studies.
There is also general agreement that the conclusions put forth by government agencies on GE technology would benefit from additional attention to, and weight on, science conducted by teams independent of technology developers. There is also agreement that the publication of the results in high-quality, peer-reviewed journals by laboratories not sponsored by the developer would be beneficial.
The majority of science supporting government regulatory decisions on agricultural biotechnologies has been paid for and/or carried out by technology developers. The protocols used typically arise from decades-old testing requirements originally adopted to test chemical food additives or pesticides, not GE plant proteins that might prove toxic or allergenic. The study designs, methods, diagnostic procedures, and statistical tests are rarely subjected to peer review and seldom encompass all proven and readily available, state-of-the-art tools and techniques.
The quality of studies conducted by GE technology developers has been uneven. This has become evident over the last several decades in a number of investigations that have uncovered inappropriate methods, problematic data collection, and even, in some cases, outright fraud.^[3]
Several studies have examined the influence of funders with a vested interest in the outcome of research on a study’s conclusions. Teams have reported that a study is 2 to 5 times more likely to be favorable to a funder with a vested interest in a particular outcome, compared to a similar study on the same subject funded by a neutral source.^[4],[5] Over time, the steady pressure of conflicts of interest has done much to skew the focus and quality of science supporting regulatory and public health decision-making, but this is far from what it should be for novel food products.
GMOScience recognizes that restoring integrity in the assessment and regulation of agricultural biotechnology is a complex and difficult mission, but believes it is a vital one if the nation is to benefit from the power of this exciting new area of technology.

First, do no harm

When it comes to a patient’s health, physicians hold dear a simple principle: “First do no harm.” Thus, when doctors read scientific studies, they look hard for evidence of possible harm.
If a study does point to possible harm, some obvious next questions should be asked: Did anyone follow up with a more powerful study that looked at the same organs or biochemical markers, and did that study resolve or reinforce concerns over risk? Is there any biological plausibility to a link between certain health problems and consumption of certain foods, and/or the GE proteins and pesticide residues within them? Can doctors extract from published science insights regarding what population cohorts might be most vulnerable to an adverse effect, or most responsive to a given therapeutic intervention?
To be recommended by a physician, a substance must demonstrate evidence of benefit that clearly outweighs its potential for harm. When less harmful alternatives with similar benefits exist, the choice is simple. This is the basis of all FDA new drug approvals, but is rarely discussed in the context of GE crops.

Wide recognition of the need for better science

The European Network of Scientists for Social and Environmental Responsibility, a group that comprises 1300 members, has issued a statement rejecting claims by GE seed developers and others that there is “scientific consensus” that all GE foods are safe.^[6]
In light of the “voluntary consultation” process currently in place at the Food and Drug Administration (FDA), the American Medical Association (AMA) has called for mandatory testing of GE foods, which would be a major and long-overdue change in policy.^[7]
The American Academy of Pediatrics (AAP), in a 2012 report, concluded that organic food (which excludes GE crops/foods) might offer advantages over non-organic food. These advantages have not yet been studied in well-designed clinical trials. Nevertheless, the AAP recognized that certain pesticides used on foods impact children’s health, particularly children of farmworkers and of mothers exposed to pesticides in the first trimester of pregnancy.^[8]
The American Academy of Environmental Medicine (AAEM), another group accredited to provide continuing medical education for physicians, has highlighted several human health risks that may become elevated with the consumption of GE foods.^[9] Since the AAEM is composed of physicians with a particular interest and expertise in environmental causes of illness, its views on the public health consequences of today’s GE crop technology carry substantial weight.
The World Health Organization states, “All GM foods should be assessed before being allowed on the market.”^[10]
The AMA and AAP further state that while they see no evidence that GE foods are harmful, nor are there adequate studies that show that GE foods are unequivocally safe. The scope and kind of studies necessary to support a definitive safety finding are not required by the FDA and have not been conducted on a single GE trait or crop.
Under current FDA policy, technology developers conduct all the safety assessments and have to state their conclusions regarding safety to the FDA. These conclusions are then typically accepted without critical review or corroboration (for more details, see “Regulating GMOs”).
Fortunately, the scientific literature has begun to publish an increasing number of independent studies on the consequences of feeding animals GE foods, and there is widespread agreement that the FDA must become more actively involved in ensuring that sound science supports GE food regulatory decision-making.
On GMOScience, biomedical and public health science and experience are drawn upon in reviewing what is known, and what is uncertain, about the safety of GE foods and GE crop technology. Our guiding principle is to ensure that GE crop and animal technology “does no harm” to people and the health of the environment.
© 2015 GMO Science. All Rights Reserved

Share on Facebook

Tweet

Save