SSR: What we’re doing is studying diagnostic testing errors, which can occur anywhere along the testing process – from when a sample is taken from a patient, to transportation to the lab, to errors within the lab, to errors in diagnosis, and then to errors in reporting the information to the clinician and to the patient. We first define error, benchmark institutions for frequency of error, and then we study why institutions have different error frequencies. We try to target errors that have the most clinical impact, and we perform chart reviews to determine how diagnostic testing errors affect patients. Then we implement changes in a number of hospitals in the U.S., looking at how we can affect pathology and clinical practice where tests are obtained. We continue to follow data benchmarking institutions so that we can see how our interventions actually affected patient care.

The collaborative started in 2002 and currently includes: University of Pittsburgh, University of Iowa, Henry Ford Health System, Drexel University, Wake Forest University, New York University, Northern California Kaiser Permanente, and Loyola University. UPMC is the primary institution that stores all the data and standardizes the processes of error definition and collection across all the institutions. It coordinates all the interventions and troubleshoots problems related to interventions or data collection at all the other sites.

SSR: Probably anywhere from two to five percent of all laboratory tests may be associated with an error. However, the ones that we have focused on are anatomic pathology tests for diagnosing cancer. Types of errors include those of interpretation, specimen identification, problems with a specimen itself, and problems of the report. The frequency of these error types varies across institutions. Of the errors in diagnosis, 98 percent are related to a false negative and one to two percent are false positive diagnoses. The majority of false negatives are due to a poor sample and/or difficulty in interpreting a poor sample.

The main difficulty is a poor sample: there may be a lot of blood admixed with a cancer sample that obscures the cancer cells. Or there may be that there are only rare tumor cells obtained in a sample and it’s extremely difficult to make a diagnosis when you have a small sample with not much tumor. The rare event is an obvious tumor on a microscopic slide that is actually missed – that happens very infrequently, but it does happen, and it may be related to experience, expertise or rare tumors that people aren’t used to examining. Other errors may be caused by wrong ancillary information provided, for example, the patient could be identified as a 60-year-old when in fact the patient is an 18-year-old and the diagnosis may differ depending upon age. Some patients with cancer who were diagnosed correctly may not be followed up because the information was never conveyed in the appropriate fashion to the right clinician.

We do not know specifically how many diagnostic lab errors occur annually, but we found that there is a huge cost burden for errors, usually from the need for repeat testing. Given the level of testing in the United States, this cost burden adds up to billions of dollars. We’re just beginning to do cost-benefit analysis of interventions.

SSR: We do interventions in two ways. We target a highly frequent error across all the institutions, trying to do the same intervention at each institution. Every institution is also asked to target any error type that they want within their own institution, then share those data with the other institutions.

One example of an intervention across all institutions was that we had two pathologists look at every slide of specific tumor types. In the diagnosis of lung cancer, there often is a higher rate of error because of false negative diagnoses – often related to a poor sample, but some of it is related to the interpretation. We believe that if two pathologists looked at every case, it would lower the error frequency, as what one pathologist may miss another pathologist may detect. We found that it lowered the error frequency for the diagnosis for lung cancer in cytology specimens across all the sites.

One of the other interventions across institutions was the introduction of immediate interpretation of fine needle aspiration. One of the problems is that a clinician obtains a specimen and doesn’t know if it’s a good quality specimen. When one obtains an immediate interpretation of a fine needle aspiration biopsy, then the clinician can know right on the spot if the specimen is a good specimen. That reduced specimen error false negatives – depending on institution and specimen type – by five to 10 percent, sometimes 20 percent.

SSR: As an example of improving the reporting format, one institution began issuing qualifying statements on biopsy cases to guide clinicians, for example, in cases that didn’t correlate with a previous diagnosis. Another intervention involved trying to reduce the use of indeterminate diagnoses, which usually entail the need for a follow-up test. One institution began using different techniques to reprocess the specimen before issuing a benign diagnosis if there was a clinical suspicion of cancer.

One of the things that we’ve found, paradoxically, is that when we began reporting error from individual institutions, their error rates went up – probably because they were just becoming good at finding error and they were standardizing themselves – which is a good thing. There probably is then some global decrease when the institution has to focus on error, which is not related to the individual intervention itself. When you begin to think about error, the first thing that happens is you find more, and the second thing is you begin to reduce them because you’re aware errors can occur and you begin to do checks to prevent them from happening.

SSR: We publish some of our data, we give national presentations at pathology meetings and we travel to other institutions to talk on this. We’re trying to have other institutions join so that it becomes less of a research study and more of a practical application of how institutions perform and interpret diagnostic testing services. Although the collaborative is research-oriented, we’re trying to see how we can get institutions to participate, where they’d be willing to change rather than being part of a grant to do it. So, we’re trying to expand our focus nationally to invite new members to join where they can learn about what works and what doesn’t from the existing interventions that have been performed. Institutions have to be able to collect some of their own error data, and that usually takes some resources. Laboratory error tends to be challenging to detect because there aren’t that many systems in place to detect them and you have to implement a collecting method in your laboratory, which takes time and resources.

SSR: Some. It tends to capture the sentinel events – the major errors – but the majority of errors are not captured because having a patient come back for a repeat test isn’t often viewed as a sentinel event.