Interpreting the Science in Scientific Studies

American Cancer Society
Friday, 7 March 1997

How do you feel when you read an article that says something causes a 100% increase in the relative risk for cancer? Or when you hear on the news that some anti-oxidants may be harmful in high doses, right after you took your multi-vitamin pill containing anti-oxidants?

Advances in medical science are reported each day, in newspapers and television broadcasts across the country. These news reports are often based on studies published in scientific journals, the accepted way scientists communicate among themselves. However, a problem arises when the results of those studies are reported to an audience who -- unlike the scientists reading the articles -- may not know the background of the research or understand the statistics.

"When science is not put into context, it can cause problems," notes Joann Schellenbach, director of media relations for the American Cancer Society. "To its credit, the media does an admirable job of reporting complex stories to a cross-section of Americans. However, we get calls from confused people all the time about something they just read or heard."

In the examples mentioned above, it may be helpful to know that a 100% increase in relative risk may not be that bad if the original absolute risk was only 1 in 100,000 people. A 100% increase would bring the total cancers to 2 in 100,000, probably nothing to be unduly alarmed over (see below for further information). The anti-oxidant example was based on a National Cancer Institute research study that examined the effects of beta-carotene on smokers to see if it reduced their rates of lung cancer. The study was halted when it showed smokers getting beta-carotene were developing lung cancer at a higher rate than the group given nothing.

It may be useful to keep the following thoughts in mind when reading or hearing about a new study:

- where was the study originally reported? The most well-known journals, such as The New England Journal of Medicine, Science, and Cancer, have strict peer-review criteria before they publish the study. Peer review means that the study was evaluated by scientists working in the same field in which the research was done. This process lends more credibility to scientific reports.

- where did you read or hear about the study? Network news programs, national newsmagazines, and city newspapers have science reporters who carefully distill scientific article into reports for the general public.

- where was the research done? Most cancer research requires sophisticated scientific training, facilities, and equipment. Therefore, it is no surprise that most advances in cancer research come from well known cancer centers, hospitals, and universities.

- does the study support or contradict past research? The more evidence there is for something, the more likely it is to be true. The search for answers comes through small steps in a long process.

- who were the research subjects? Some news items report studies from animal research -- important, but not always immediately pertinent to humans. Sometimes specific age, racial, or ethnic groups are examined, such as African American men over 75 with a high risk of prostate cancer or Ashkenazi Jewish women with a tendency to develop breast and ovarian cancers. Specific findings from such studies may not always apply to other groups.

- what type of study was it? A randomized clinical trial, where groups of individuals randomly assigned to different treatment groups are compared, generally is considered the "gold standard" in research done to evaluate new treatments.

- if the study is a clinical trial, what phase was it? Phase I clinical trials are very early and as such, are intended to examine the safety of the intervention. Phase II trials, the next step, examine the effects of a research treatment on the specific disease under study. Phase III trials are planned to compare the new treatment with the standard treatment to see which is more effective.

- how long did the study last and how large was it? Generally, studies that affect medical policy or guidelines are the result of years of research examining a large number of people. It usually takes a number of such studies before the experiment or trial is considered definitive.

- do you understand the terminology used to describe cancer risk in this study? Terms such as relative and absolute risk, are a stumbling block for many non-scientists. To take the aforementioned example further, if a certain behavior or exposure is associated with a 100% increase in the risk of developing cancer, it sounds ominous at first. However, if only one person out of 100,000 currently develops that particular type of cancer, a 100% increase in risk results in only 2 in 100,000 people.

For more information, or to contact American Cancer Society, see their website at: www.cancer.org