Abortion, the talking point goes, is a private matter between a woman and her doctor.  But, after a series of missteps by companies offering early genetic screening, pro-choicers might want to append “… and a mathematician” to that phrase.

As reported in the Boston Globe, new techniques for screening pregnancies are leading more women to choose abortion, but the data with which they’re being supplied isn’t very reliable:

Two recent industry-funded studies show that test results indicating a fetus is at high risk for a chromosomal condition can be a false alarm half of the time. And the rate of false alarms increases the more rare the condition, such as Trisomy 13, which almost always causes death.

Companies selling the most popular of these screens do not make it clear enough to patients and doctors that the results of their tests are not reliable enough to form the basis of a diagnosis.


Errors in preliminary screening are serious because the initial test may be the only one that a patient sees.  When a screening test comes back with a worrying result, one in 20 women elect abortion immediately, without following up with another test (such as amniocentesis) to confirm the preliminary result.  Sometimes, even after receiving a clean bill of health from a follow-up exam, women choose abortion just to be safe, confident that the initial screening test was accurate.

The Boston Globe did a good job reporting the symptoms of this screening problem; the diagnosis that it offered isn’t quite on the mark.  The companies aren’t necessarily providing flawed tests or lying about those tests’ accuracy.  It’s just that the very way we talk about the accuracy of a medical test is flawed.

Usually, when advertisers tout a statistic of 99 percent accuracy, they’re talking about a true positive rate.  That is: If you do have a genetic abnormality, the test will say that you do 99 percent of the time.

The trouble is, although a high true positive rate is nice, it’s not sufficient to make a test useful.  At the extremes, I could run a very shady fetal screening company that had a 100 percent true positive (“accuracy”) rate, if I wasn’t above a bit of flim-flam.

All I’d have to do is take a blood sample from each woman who came to my clinic, wait a few days after her appointment, and then gravely call her to say that the results had come back positive—without ever having screened her blood at all!  I’d be guaranteed to have a 100 percent true positive rate, because I would just tell every woman that her fetus was abnormal, meaning that I’d end up telling the truth to every woman whose pregnancy was actually unusual.

That would be cold comfort to all the women with run-of-the-mill pregnancies whom I’d misled.

The companies under scrutiny aren’t doing anything as shady as my hypothetical clinic, and a good many of them may not realize how badly they’re counselling their patients.  After all, this kind of error isn’t limited to OB-GYNs, and it can cause major medical confusion at every stage of a patient’s life.  The woman who chooses an abortion after receiving a misinterpreted test result may find herself, a few years later, receiving an unnecessary biopsy—or worse, mastectomy—after her oncologist overweights the significance of her unusual mammogram.  Further, the dangers of “accurate” screening aren’t limited to women.  Men may find that they’ve submitted to an invasive, sexually-disruptive surgery after they go in for a prostrate screening.  Wherever we prize finding every aberration over maximizing the welfare of every patient who walks into a clinic, we’re prone to overtreat, sometimes with disastrous effects.

So how do doctors go so wrong?  The problem is base rate neglect.

Most diseases we screen for during pregnancy are rare.  And, if a counselor gives you a “kitchen sink” lineup of tests, your pregnancy may be screened for diseases you’re exceedingly unlikely to have.   For example, for a non-Jewish couple, the chances of having a baby with Tay-Sachs disease would be one in 62,500. (The risk of a non-Jewish person being a carrier is 1/250).

So, prior to learning anything from a screening test, it would be very unlikely that this gentile couple was having a baby with Tay-Sachs; the base rate of the disease is very low.  But suppose that the couple gets a positive result on a Tay-Sachs test.  Imagine even that the test was 100 percent accurate (in the true positive sense) but had a 10 percent chance of giving a false positive.

One of the easiest ways to reason about these kinds of statistics is just to remember your initial base rate (1 : 62,500 :: Tay-Sachs : Unaffected) and imagine that you’re actually dealing with a roomful of 62,501 children—only one of whom has Tay-Sachs.

That one child gets flagged by the 100 percent accurate (true positive) result.  But 10 percent of the healthy children get flagged, too.  Since you started with so many healthy children proportional to the one sick child (because Tay-Sachs is so rare among Gentiles), this means that 6,250 healthy pregnancies look like Tay-Sachs pregnancies.

So, with each positive test result, the chances that the woman is actually carrying a child with Tay-Sachs is 1 : 6250 or 0.016 percent.  The test did change the woman and her doctor’s initial expectation (they would have started off by thinking that it was a 0.0016 percent chance), but the risk is still so low that the woman might well decide that she wants to continue with her pregnancy.

That’s if the doctor explains the risk thoroughly, talking about false positives and the final odds.  But the odds of this happening, though not as low as the risk of Tay-Sachs to a non-Jewish couple, are still pretty unlikely.

No one’s done a survey of genetic counselors to check how well they understand the statistics of risk, but in a survey in which gynecologists were asked a similar question about screening and risks, over half of the physicians estimated that the chance of a woman with a positive mammogram having breast cancer was at least 80 percent.  The true risk of breast cancer (correctly supplied by only one in five doctors) was 1 percent.

Correcting these biases and misunderstandings isn’t just a matter of urging a patient not to take a test too seriously, whether the results are abortion or biopsies.  Sometimes the right—though counterintuitive—answer is not to do a kitchen sink screening test at all.

It makes no sense to check whether a non-Jewish couple has Tay-Sachs (or, in the eyes of the United States Preventive Services Task Force, whether a woman with no unusual risk factors has breast cancer).

Because the base rates are so low, even a positive result will still represent a chance of danger far below the threshold at which the patient or the doctor wants to take action.  And if you already know that you shouldn’t act based on a positive test result, what’s the point of running the analysis?

Unfortunately, the temptation to run every test available on a given sample remains, especially as scientists get better at amplifying fetal DNA and can run tests earlier, using a simple blood draw rather than requiring anything as invasive as amniocentesis.

In order to serve their patients well in the face of this impulse toward completionism, doctors need to review their statistics basics.  And also that line from medicine 101: “First, do no harm.”