A heart stent manufactured by Boston Scientific Corp. bsx +2.86% Boston Scientific Corp. U.S.: NYSE $12.21 +0.34 +2.86% Oct. 14, 2013 4:00 pm Volume (Delayed 15m) : 24.36M U.S.: NYSE $12.29 +0.08 +0.66% Oct. 14, 2013 7:25 pm Volume (Delayed 15m): 88,618 P/E Ratio N/A Market Cap $15.94 Billion Dividend Yield N/A Rev. per Employee $296,917 08/22/13 The Daily Startup: VCs Take Lo... 08/12/13 Companies Ease Up on Non-Compe... 07/25/13 Morning MoneyBeat: Another Bea... More quote details and news » and expecting approval for U.S. sales is backed by flawed research despite the company's claims of success in a clinical trial, according to a Wall Street Journal review of the data.

Boston Scientific submitted the results of the 2006 trial to the Food and Drug Administration to gain U.S. approval for the Taxus Liberte, which already is one of the top-selling stents abroad. Coronary stents -- tiny scaffolds that prop open arteries clogged by heart disease -- are one of the most popular methods for treating heart patients, and have been implanted in more than 15 million people world-wide.

But Boston Scientific's claim was based on a flawed statistical equation that favored the Liberte stent, a Journal analysis has found. Using a number of other methods of calculation -- including 14 available in off-the-shelf software programs -- the Liberte study would have been a failure by the common standards of statistical significance in research.

Boston Scientific isn't the only company to use the equation, known as a Wald interval, which has long been criticized by statisticians for exaggerating the certainty of research results. Rivals Medtronic Inc. mdt +0.36% Medtronic Inc. U.S.: NYSE $55.19 +0.20 +0.36% Oct. 14, 2013 4:00 pm Volume (Delayed 15m) : 3.43M U.S.: NYSE $55.12 -0.07 -0.13% Oct. 14, 2013 4:42 pm Volume (Delayed 15m): 18,834 P/E Ratio 15.72 Market Cap $54.85 Billion Dividend Yield 2.03% Rev. per Employee $362,283 09/27/13 Medtronic's 'Artificial Pancre... 09/20/13 FDA to Require ID Numbers for ... 09/06/13 Fifteen Patients Possibly Expo... More quote details and news » and Abbott Laboratories abt +0.71% Abbott Laboratories U.S.: NYSE $34.00 +0.24 +0.71% Oct. 14, 2013 4:00 pm Volume (Delayed 15m) : 6.80M U.S.: NYSE $34.00 0.00 0.00% Oct. 14, 2013 6:11 pm Volume (Delayed 15m): 19,925 P/E Ratio 13.37 Market Cap $52.47 Billion Dividend Yield 1.65% Rev. per Employee $345,430 09/24/13 KKR Seeks to Milk Rise in Chin... 09/23/13 KKR Seeks to Milk Rise in Chin... 09/19/13 Wealth Adviser: Missing Out on... More quote details and news » have used the same equation in stent studies.

But in those cases, any boost provided by the Wald equation wouldn't have changed the outcome of the study. In the Liberte study, the equation's shortcomings meant the difference between success and failure in the study's main goal.

The difference also sheds light on the leeway that device makers have when designing studies for the FDA. Studies designed to satisfy the requirements of the FDA's medical-device branch can be less rigorous than those aimed at winning U.S. approval for drugs. That is partly because of a 1997 federal law aimed at lessening the regulatory requirements on device makers.

The FDA declined to specifically discuss its deliberations of the Liberte, which is still under review by the agency.

Boston Scientific doesn't agree that it made a mistake or that the study failed to reach statistical significance. "We used standard methodology that we discussed with the FDA up front, and then executed," said Donald Baim, Boston Scientific's chief scientific and medical officer. "We have no obligation to show that we can meet any arbitrary test."

Boston Scientific, of Natick, Mass., has roughly 40% of the $5 billion a year world-wide market for coronary stents. The company's Taxus Express is the top seller in the U.S. In considering approval of the Liberte, the FDA asked the company to show that its anticlogging performance in patients was "non-inferior" to the Taxus Express.

In 2006, the company said that it met this test. Using a standard probability measure known as the "p-value," it said that there was less than a 5% chance that its finding was wrong. In April 2007, in the Journal of the American College of Cardiology, the company's researchers for the first time specified the p-value: 4.87%.

Scientists generally regard studies with p-values above 5% to be failures, and medical journals typically won't publish them, because they don't provide a high level of certainty needed to ensure that results would apply to all patients, not just those under study. Put another way, science traditionally requires 95% certainty that a study proved its premise.

But the Journal's calculations found that the Liberte study's p-value was about 5.1% -- failing to rule out the possibility that patients getting the Liberte stent will have markedly more artery recloggings than those receiving the Express.

Although the difference seems small -- 0.2 of a percentage point -- it is the difference between success and failure for a product on which Boston Scientific has spent some tens of millions of dollars.

The Liberte is made of thinner metal than the Express, making it easier to snake through patients' arteries to get to a blockage. That ease of use appeals to doctors, and would help the company compete against newer stents from rivals' Medtronic and Abbott.

Neither Boston Scientific's original news release reporting the Liberte's success nor the 2007 cardiology-journal article disclosed that the study had used the Wald method, an approximation technique that is among the easiest and least accurate for analyzing statistical results.

A number of experts in statistical methods have found fault with it over the years. "The Wald interval is broken," said Lawrence Brown, a statistics professor at the University of Pennsylvania. Dr. Brown called the method a "deficient technique" that "should not be used." Brent Coull, a statistics professor at Harvard University, said the Wald method "overstates the certainty" of clinical results.

Boston Scientific says the study, known as Atlas, was a success because the company's methodology, including the Wald equation, was accepted by the FDA before the study began, and it was bound to follow that method.

Gregory Campbell, director of the biostatistics division of the FDA's device branch, said the Journal's analysis raised "good questions" but declined to comment on the Atlas study or the Liberte stent. Mr. Campbell called Boston Scientific's calculation approach "a standard methodology."

The FDA hasn't approved the Liberte for sale in the U.S. But the agency has given Boston Scientific an "approvable" letter that indicates it will OK the stent when it lifts a new-product moratorium it imposed on the company two years ago for reasons unrelated to the Liberte application. The company has said it expects the moratorium to be lifted this year.

In the Atlas study, Boston Scientific tested the Liberte in 871 patients. The company has said that partly because the Liberte stent uses a design similar to its predecessor, the Express, it was allowed to conduct an easier test than would be required for more-novel products.

The difference reflects the device branch's discretion in interpreting a 1997 law that required it to ask manufacturers only for the "least burdensome appropriate means" of proving that new devices work. No comparable leeway exists for drug approvals.

The FDA permitted Boston Scientific to omit a comparison, or control, group of patients newly implanted with the Express. Instead, Boston Scientific was allowed to compare the Liberte's fresh results with data it had collected on Express patients a few years earlier.

Boston Scientific didn't have to "blind" participants in the study. Patients and doctors were aware that the stent being implanted was the model on trial -- a Liberte. That can lead to a placebo effect -- or patients saying they felt relief merely because of the psychological effect of knowing they were getting a new treatment.

Boston Scientific wasn't required to prove that the Liberte was "superior" than a previous treatment, the agency decided-only that it wasn't "inferior" to Express. Boston Scientific proposed-and the FDA okayed-a benchmark in which Liberte could be up to three percentage points worse than Express -- meaning that if 6% of Express patients' arteries reclog, Boston Scientific would have to prove that Liberte's rate of reclogging was less than 9%. Anything more would be considered "inferior." Control groups, blinding and a superiority standard are features that the agencies' drug branch usually requires to approve new products.

"The burden is lower for devices than for drugs," said Jeffrey Tice, an internist at the University of California, San Francisco. "I don't believe the FDA would approve a new drug on the basis of non-randomized, non-blinded trials," he said.

In the end, after nine months, the Atlas study found that 8% of the patients suffered reclogging. In comparison, historical data on 991 patients implanted with the Express stent show a 7% rate. Boston Scientific then had to answer this question; could the study have gotten such results if the Liberte truly were inferior to Express?

Answering that question calls for a bit of statistical hypothesis since observing reclogging in just 871 patients is only an estimate of the total population. Assume that the rates of reclogging actually do differ by at least three percentage points. In that case, what would be the chance that a trial with 871 patients would report a difference equal to or less than one percentage point, which was the actual difference among patients in the study? If the chance is less than 5%, by statistical convention, the assumption that the Liberte is inferior can be rejected.

In a vital part of its calculation, called the "standard error," Boston Scientific assumed that the difference between the true rates of reclogging between Liberte and Express was one percentage point-what the trial showed. In a sense, the company's equation assumed that Liberte definitely isn't inferior. That's a problem, since the equation was supposed to be deciding whether Liberte might be inferior after all. As a result, Boston Scientific's equation gives wrong answers that consistently favor the Liberte.

To remove that flaw, the standard-error calculation should be modified so that it assumes the true difference between rates is three percentage points. When done that way, the chance falls above 5% -- meaning the results failed to reach statistical significance.

Jonathan Shuster, a statistics professor at the University of Florida, said the Wald method is commonly used, but imperfect. "Most statisticians would accept this approximation," he said. "But since this was right on the border, greater scrutiny reveals that the true, the real, p-value was slightly more than 5%."

The Journal also asked a mathematician, Travis Schedler, to confirm its calculations. Dr. Schedler is a researcher at the Massachusetts Institute of Technology and a 2008 fellow of the American Institute of Mathematics, of Palo Alto, Calif. His calculations agreed with the Journal's.

"We're not going to say there's anything mistaken in your mathematical calculations, but the issue is really the relevance of that analytical thread," Boston Scientific's Dr. Baim said.

—Charles Forelle contributed to this article.

Write to Keith J. Winstein at keith.winstein@wsj.com