Some press accounts also declared the report to be a watershed consumer guide allowing hospital-to-hospital comparisons of infection frequency, cost, length of stay and mortality – even though PHC4 cautioned against using it in that manner.

The council was up front about some of the report’s data limitations – noting that hospitals may vary widely in their infection tracking and reporting abilities; that the infection categories collected by the council had changed mid-stream; and that hospital charge data – the cost variable – do not reflect actual reimbursement amounts, which are significantly lower.

The consensus among stakeholders appears to be that making hospital-specific HAI data publicly available is a necessary and important spur to self improvement among providers, and that data refinements will further enhance the utility of future reports. "Our experience in Pennsylvania is that shedding light on outcomes has led to quality of care improvements, and providers are in the best position to make those improvements," according to PHC4 Executive Director Marc P. Volavka.

But key data validity issues that were raised by the council’s previous HAI reports remain largely unresolved in the current report, which continues to invite comparisons of mortality, length of stay, and cost of care between HAI and non-HAI patient hospitalizations without separating the impact of the infection from the underlying disease or condition that brought the person – often a complex and elderly patient with a weaker immune system and who is inherently costlier to treat – to the hospital. Those criticisms were communicated in detail to the council a year ago by the Hospital & Healthsystem Association of Pennsylvania, and by 14 medical directors and chief medical officers of hospitals throughout Pa., including a physician who was co-chair of PHC4’s HAI advisory panel and chair of the federal Healthcare Infection Control Practices Advisory Committee (HICPAC).

The current report did make two concessions to advocates of patient severity adjustment – by excluding burn and transplant patients, and by creating hospital peer groups to ensure that hospitals offering similar types and complexity of services and treating a similar number of patients are displayed together.

The significance of patient severity as a contributor to HAI susceptibility was called into question by three new studies released on the heels of the council’s new report, and promoted in tandem with it at a PHC4 press conference, which suggest that hospital processes play a much larger role in HAI risk than previously believed, exposing – as the report puts it – the "myth of inevitability" surrounding the issue, and further highlighting the moral and clinical imperative of focusing on hospital process improvement to reduce or eliminate HAIs.

Given the report’s methodological limitations, its minor improvements over previous versions, its accompaniment by new research minimizing the importance of its limitations, and its data displayed in a format that seems to encourage inappropriate comparisons of one hospital’s numbers to another hospital, the meaning and significance of the report become complicated questions. Those questions have important implications for how various stakeholders – consumers, providers, payors, insurers and regulators – should use the data, and how the validity and utility of the data can and should be improved.

Pa.’s Latest HAI Report

PHC4’s first HAI report, released in July 2005, generated significant national attention as the first public glimpse at actual reported numbers – 11,668 HAI cases confirmed and submitted by Pa. hospitals for the year 2004 – rather than estimates or extrapolations. According to the council’s latest report, released this past November, Pa. hospitals identified 19,154 HAIs during 2005, while the mortality rate, average length of stay and average hospital charge for patients with an HAI was 12.9 percent, 20.6 days and $185,260, respectively – compared to a mortality rate of 2.3 percent for patients without an HAI, an average length of stay of 4.5, and an average charge of $8,311.

According to Volavka, the ultimate goal of future editions of the HAI report is to allow hospital-to-hospital comparisons – as one tool, among many others, to stimulate good questions about a hospital’s process improvements – but he says the report’s most important use will be to make longitudinal comparisons. "Improvement is the goal, and the report can be used to compare rates of improvement over time across hospitals," he notes. "Every report the council puts out is a consumer guide," he says, noting that future HAI reports will allow a consumer to ask appropriate questions of a hospital such as, "I see your improvement rate is X and this other hospital’s rate is Y. Why is that?"

In developing the guidelines for HAI data submission, PHC4 adopted the Centers for Disease Control and Prevention (CDC) definition of an HAI: an infection is a localized or systemic condition that 1) results from adverse reaction to the presence of an infectious agent(s) or its toxin(s), and 2) was not present or incubating at the time of admission to the hospital. PHC4 expanded the CDC’s list of 13 major HAI site categories to include a category for multiple infections and to differentiate device related and non-device related infections, while readmissions due to a HAI are not required to be reported.

Reportable HAI categories changed over the course of the 2005 data collection period. For the first two quarters of 2005, hospitals were required to report four types of HAIs: indwelling catheter-associated urinary tract infections (UTIs); ventilator-associated pneumonias (VAPs); central line-associated bloodstream infections (CLABs); and surgical site infections (SSIs) related to circulatory system, neurosurgery and orthopedic surgery. For the third quarter of 2005, the requirements for reporting surgical site infections were expanded to ten body systems, including endocrine, gastrointestinal, genitourinary, reproductive, respiratory, hemic/lymphatic, and skin/soft tissue. For the fourth quarter of 2005, the requirements were expanded to include non-device associated UTI’s pneumonias and bloodstream infections. Facilities are required to submit the HAI data to the council on a quarterly basis within 90 days from the last day of each quarter, after which reports are generated and returned to each facility with an opportunity to correct any problems.

For all future HAI data collection, beginning with first quarter 2006, reportable categories were expanded to seven more infection types not associated with surgical sites: bone and joint, central nervous system, cardiovascular system, gastrointestinal system, lower respiratory system other than pneumonia, reproductive system, and skin/ soft tissue.

The latest PHC4 report displays data for each individual Pa. hospital – number of HAI cases, infection rate per 1000 cases, mortality number and percent, average length of stay in days, and average charge – for five types of infections: urinary tract, surgical site, pneumonia, bloodstream, and multiple.

In designing the methodologies used to analyze and report the HAI data, PHC4 considered how, or whether, to risk-adjust the publicly reported data. "One argument against risk-adjusting HAI data," the report says, "rests in the notion that we should all strive toward the goal of zero hospital-acquired infections. The reporting of actual, rather than risk-adjusted, numbers highlights actual results and serves to encourage root cause analysis of every patient who contracted an infection while in the hospital."

The report made two concessions to advocates of risk-adjustment. In a six-page-long "Readers Guide" that includes caveats on data interpretation, the report notes that patients being treated for burns, undergoing organ transplants, or being treated for complications of an organ transplant were excluded from the report because they may be at a greater risk of acquiring an infection while in a hospital. Second, four hospital peer groups were created based on number of patients, complexity of services and percent of surgical procedures, and those hospital are listed among others in their peer group.

Three hospitals that had used an electronic HAI surveillance system throughout 2005 were listed separately in their own peer group, with the acknowledgement that those systems may produce more comprehensive reporting of HAIs – and the hospital may not necessarily have more of them – than hospitals relying on manual HAI coding. Over time, the council believes that all hospitals will adjust to the HAI reporting regime and their rates should start to normalize over time, reducing variation stemming from identification and reporting factors.

The council has already seen significant improvements in reporting compliance compared to a year ago, when PHC4 identified 21 hospitals as under-reporting HAIs by noting large discrepancies between reported numbers and average expected numbers based on peer group comparisons. Compliance issues impacted data from some hospitals for the first two quarters of 2005 in the current report, says Volavka. The council continues to conduct reporting audits of facilities, and has seen much more uniform data reporting for the first two quarters of 2006, giving Volavka confidence that Pa. hospitals are making a good faith effort in their reporting compliance.

"We may not know if there were other factors that contributed to the outcome of a given patient’s case, including whether or not an infection contributed to a patient’s death," the Reader’s Guide cautions. "However, it is universally agreed that hospital-acquired infections in the aggregate have a significant impact upon the cost of care, as well as on patient care outcomes."

Report’s Proper Use and Improvement

"The report is not a consumers guide. It is a proof-of-concept intended to demonstrate the feasibility of reporting hospital-acquired infections," according to David B. Nash, M.D., MBA, chairman of Jefferson Medical College’s Department of Health Policy, and chair of PHC4’s Technical Advisory Group. As a statewide benchmark for hospitals, the report puts boundaries on the problem, illustrates the scope and breadth of the problem and furthers the self-evaluation process by hospitals, he adds.

Because of methodological issues, the report is not ready to be used by payors yet, says Nash. Although the CDC has been fielding questions about HAIs for 30 years, says Nash, there is still no national agreement of what an HAI actually is, and the CDC needs more uniformity in its classification of HAI definitions. As the data improve in future reports, Nash believes that managed care companies can and should use the PHC4 HAI reports as part of their network decision-making, in tandem with other quality data sources.

The data can be made more meaningful to providers for process improvement in ways that allow them to pinpoint connections between process variation and HAI frequency – for example, to study post-operative wound infection, the data could be made specific to types of procedures and stratified by age group, says Nash.

Nash believes that patient severity is an important consideration and has to be evaluated further to examine the magnitude of its impact on mortality and cost attributable specifically to HAIs. A related question, and a pivotal one for providers as they endeavor to reduce or eliminate HAIs, is the extent to which care processes – independent of patient characteristics – contribute to HAIs. Three pioneering studies published in the Nov/Dec 2006 American Journal of Medical Quality (AJMQ), of which Nash is editor, examined that question and concluded that it is the process of care, not the underlying clinical condition of the patient, that drives HAIs.

In his editorial accompanying the studies, Nash wrote that such a finding should raise the "anchoring heuristic" – a fallback belief that persists in the face of competing facts – that HAIs are simply an inherent risk in health care, "almost an expected outcome from the care of seriously ill patients, especially those in our high-technology settings such as the operating room, intensive care unit, or renal dialysis center," wrote Nash.

Because there is no established threshold of what constitutes good performance, the proper use of PHC4’s HAI report is to compare magnitudes of improvement of a given hospital over time, according to Richard Shannon, M.D., University of Pennsylvania Health System’s senior vice president of clinical affairs, Department of Medicine; and member of PHC4’s Technical Advisory Group.

Hospital-to-peer group comparisons will also become important in creating communities of learning to accelerate process improvements, notes Shannon. Smaller hospitals, for example, might be able to get to the theoretical limit of zero HAIs more easily than larger institutions, he adds. PHC4’s peer grouping is a reasonable first pass at categories, but can be improved further – for example, by splitting Peer Group 1’s 25 institutions into a Level 1 and Level 2 Trauma Center peer groups, says Shannon.

Shannon says "it doesn’t help" to include mortality rates in the report, as it is wrong to assume (as a caveat in the report notes) that the rates are attributable to HAIs, and it would be an enormous drain on resources to do the required individual case review to factor out HAI-associated mortality. Shannon says he is also less interested in hospital charge data based on DRG, and that a meaningful cost analysis needs to consider actual revenue and expenses of individual cases – also requiring detailed individual case review.

Shannon, as lead researcher and author of one of the AJMQ studies, did intensive analysis of 54 patients with CLAB who were matched for age, severity of illness on admission, and principal diagnosis, and found that costs of CLABs and their associated complications averaged 43 percent of the total cost of care, and resulted in a net revenue loss (averaging about $26,839 per case) to hospitals. "Hospitals are not making money on these infections. This is a call for reform that benefits everyone," he adds.

Preliminary studies on the importance of adjusting HAI data for patient severity reach nuanced conclusions.

Shannon’s study found that "neither age, severity of illness, nor principal diagnosis appeared to constitute a risk for CLAB, suggesting that process defects rather than clinical illness were more important predictors."

A second AJMQ study examined 2004 PHC4 HAI data in tandem with data from the Cardinal Health-Atlas Research Database – which groups cases into one of five admission severity groups based on many factors, including patient demographics, type of ICU, procedure and diagnosis codes, laboratory values, vital signs, and information on 67 key clinical findings. The multivariate matching study found that patients with HAI in Pa. hospitals were older and more severe on admission across many disease groups, which may have accounted for their poorer outcomes relative to non-HAI patients. The researchers wrote that "HAI patients may be sicker on admission as compared to the non-HAI patients, making it vitally important to adjust for this effect."

Even after compensating for admission mortality risk, however, the study found that HAI cases still demonstrated significantly higher mortality than non-HAI controls did. The researchers concluded that HAI significantly increases mortality risk and length of stay.

A third AJMQ study examined PHC4 SSI data from Oct. 2004 to Sept. 2005, also in tandem with Atlas severity ranking data, and found that patient characteristics and hospital factors were both significant determinants of surgical wound infections, but also that the risk of infections for roughly comparable patients varied significantly across hospitals – suggesting that much of the risk is due to hospital-specific practices and environments. The researchers used an engineering metaphor – "tolerance stack up" – as an exhortation to health care practitioners to pursue process improvements independent of patient-specific factors: "When the summed individual tolerances of several combined components exceeds the specified tolerance of the whole," there is a higher risk of failure.

Shannon believes that a consideration of patient severity is merited in HAI data presentation, but that it should not dilute the focus on care process improvement. "There is no doubt that sick, complex patients are at greater risk of hospital-acquired infections, but these studies showed that non-complex patients get these infections – that HAIs are not biologically determined, and that care-related factors play a significant role," says Shannon.

If PHC4’s HAI reports are used longitudinally, to compare the rate of improvement of the same hospital over time, Shannon notes, the absence of severity adjustment won’t matter much because an individual hospital’s case mix from one year to the next wonvary significantly.

PHC4’s Technical Advisory Group meets at the request of the council and provides guidance on how to publish the report, says Volavka. The group recommended that the council’s first hospital-specific HAI report focus on getting over the "anchoring heuristic" that HAIs are an inevitable outcome of care, and that the council not try to risk-adjust the data, beyond categorizing hospitals by peer group and excluding burn and transplant patients, Volavka notes. The group’s view was that HAIs are not about the risks that patients bring to hospitals, but about processes, some of which – like more vigilant handwashing – are uniformly effective and in need of improvement in all hospitals, he adds.

While the council has no plans to modify its existing peer group categories, Volavka says it would consider a hospital’s request to be placed in a different peer group. Hospitals can also request customized reports that group institutions into chosen subcategories, he adds.

Since late 2005, PHC4 has collected a widely accepted proxy measure for risk-adjusting some HAI data – a rate per 1,000 "device days," – for example, number of CLABs per 1,000 central line catheter days stratified by unit type. Although the council is still collecting device day information from hospitals that want to provide it, Volavka says the validity of the data is in question, as CDC’s device day reporting definition gives hospitals wide latitude on data capturing methods and time intervals, and has yielded disparate data.

The council has had conversations with the CDC about device days and learned that it established what people regarded as a normative benchmark, which Volavka says may give hospitals a false sense of accomplishment if they settle for "better than average," while some of their patients continue to die from HAIs. Volavka adds that the council’s HAI reporting categories have also expanded well beyond device-associated infections.

Reception by Stakeholders

Representatives of the consumer, hospital, employer and insurer communities express enthusiasm in their belief that PHC4’s report will make a material impact in HAI reduction efforts, and that future versions of the report will see improved data validity.

"We believe the public is sophisticated enough to understand the nuances of this report, if the data limitations are explained clearly. As time goes on, people will learn more about how to use the report," says Lisa McGiffert, Consumer Union’s project director of StopHospitalInfections.org. She points to a recent study, published in the American Journal of Medical Quality, that examined consumer attitudes about HAIs and hand hygiene practices and found that 93 percent of consumers said knowing infection rates for a hospital or doctor would influence their selection, while nine in ten said higher than average infection rates would be a very important reason to avoid a hospital or doctor.

"PHC4 was very cautious about how this report should be used, and emphasized that it is the first report of its kind. We are less cautious about how it should be used. It is not a perfect report, but it is way better than anything before. Every infection was reported by a hospital and did occur." McGiffert believes that longitudinal hospital comparisons are the best way to use the report, and that PHC4 will improve the information in the next report to make it more accurate.

Using the numbers and rates of infections, McGiffert says consumers should see how their hospital compares to like hospitals, and stimulate conversations with their physicians about it. "Local advocacy groups should also hold hospitals accountable to how they are improving," she adds.

She also regards the report as a critical tool to jumpstart an effort to reduce HAIs that she says has been languishing for decades. "The main problem is that this is a secret that has been handled internally by the medical community. The first step for us is to raise public awareness of the problem," she adds. Awareness of HAIs at the hospital level was minimal because the prevailing attitude was that HAIs were inevitable and couldn’t be prevented, she says. Many hospitals over the last few years have taken the issue seriously and become "evangelical about it," declaring, "We can have an impact on this," McGiffert adds.

Improvements McGiffert would like to see include making the data as timely as possible, with a "rolling annual rate" that is updated and presented quarterly. "Just once a year is a long time to wait to compare improvement," she notes.

The Hospital & Healthsystem Association of Pennsylvania (HAP) had called for institution-specific HAI report in the belief that getting data out in the public domain will stimulate attention to full and accurate reporting, and to improvement by hospitals and clinicians, while researchers can try to make collection and reporting better, according to Paula Bussard, HAP’s senior vice president, policy and regulatory services.

HAP does not believe that publicizing the data will have a punitive impact on low performers. "We’ve had 20 years of public reporting, and it hasn’t changed consumer behavior. It has changed provider behavior – for the benefit of consumers," says Bussard. Just as the primary users of PHC4’s hospital performance reports turned out to be the hospitals themselves, Bussard believes that hospitals’ medical quality committees will use HAI report data to make cross-hospital and peer group comparisons, examine self-improvement over time, and compare it to other databases – such as CMS process measure reporting – to spur internal improvements.

Cross-hospital comparisons should not be made until researchers further analyze and refine the data, says Bussard. Areas of refinement HAP would like to see include displaying rate trends over time; comparing risk of infection for comparable groups of patients; factoring out real cost attributable to infection from the cost to treat the underlying disease; and displaying infection rates per diagnosis and type of procedure. Such refinements would allow for greater precision in assessing the implementation of best practices, Bussard maintains.

HAP didn’t want to sacrifice a starting point for reporting the data in the public domain to wait for more perfect data, and it is comfortable with the caveats the report presented, e.g., cautioning readers that it is not known to what extent a hospitalmortality rate was attributable to HAIs, says Bussard.

While the report is extremely valuable in highlighting the importance of HAIs, its primary use is to allow hospitals to see how they compare to others and to improve their performance, believes Michael Madden, M.D., a medical director in the Quality and Medical Performance Management Department at Highmark Blue Cross Blue Shield. The report supports Highmark’s decision to have HAIs be a cornerstone of its QualityBlue hospital pay-for-performance program, for which about 50 percent of additional reimbursement is accounted for by improvements in CLAB, SSI and MRSA colonization, he says.

"Transparency reports like this are not ready to be used to direct care or to pick a hospital," because of variation in the ways that infection control specialists at different hospitals are using the definitions to identify each HAI type, Madden says. , we could include it in our transparency program when we feel comfortable with the data validity," he adds.

The best way to allow consumers to reliably compare one hospital to another, Madden says, is for hospitals to adopt an electronic surveillance system to reduce the "eye of the beholder" HAI coding variation, and Madden believes that data normalization will take two or three more iterations of the report.

To that end, PHC4 and the Highmark Foundation awarded grants to 11 facilities in 2006 to participate in the Reducing Hospital-Acquired Infections with Electronic Surveillance Demonstration Project, to assist them in implementing an approach which allows for more timely and comprehensive identification of hospital-acquired infections, says Madden.

Another improvement to the report would be to recognize the cost and mortality data in the context that sicker patients get more infections, and that the cost of the hospital admission affects the overall cost of care for a patient that may contract an HAI, says Madden. PHC4 did a good job of identifying limitations of the report, but some media outlets were irresponsible in their reporting of infection rate and cost comparisons, he believes. Electronic surveillance systems might help factor out some cost and mortality attributable to HAIs, but their science is not yet ready to do that on a large population without individual chart review and extensive severity adjustment – which would be labor- and cost-intensive, says Madden. "The challenge is not doable without better surveillance systems," he adds.

"This public reporting and sensitizing by all stakeholders to the HAI problem will result in Pennsylvania being demonstrably the best in the country in five years, with measurable dollar consequences that will dwarf the cost to get the valid data," believes Cliff Shannon, president of SMC Business Councils, which represents 4,000 employers, primarily in southwestern Pa.

Holding one’s performance up to public scrutiny, while being able to compare it to one’s peer performance, has prompted a healthy competition among physicians – as evidenced by improvement on mortality and morbidity over several years of PHC4 coronary artery bypass graft surgery (CABG) reports, says Shannon.

What is different about the HAI report, however, is that hospitals are not as readily able to extract and report information about patient safety as they are for the CABG report, says Shannon, who is also chair of PHC4’s Data Systems Committee. This first hospital-specific HAI report shows unevenness of HAI identification and reporting methods, Shannon believes, and a "shakedown" of the report will take another year or two, as the community of stakeholders must come to agreement about what is and isn’t an HAI. Early adopters of electronic surveillance systems puts them ahead of the curve in detecting problems and making significant improvements, adds Shannon.

Shannon also believes that the report’s largest practical impact is in boosting providers’ attentiveness to their patients. "As soon as physicians and nurses are aware that something – real-time – is harming or killing their patients, they become galvanized," he says. Ultimately, real-time data is needed to make actionable the tranformative belief that HAIs are almost always preventable, Shannon adds.