Data may mislead on quality

HOSPITAL quality metrics such as rates of venous thromboembolism could provide a “perverse incentive” to game the system, warns a leading clinical pharmacologist and specialist in hospital safety.

Professor Ric Day, professor of clinical pharmacology at the University of NSW and involved in Australian Institute of Health Innovation research on hospital patient safety, told MJA InSight that US research which supported “the more you look, the more you find” phenomenon was a red flag for the rating of hospitals in terms of safety and quality. (1)

The US researchers used discharge data for 954 926 surgical patients from 2786 hospitals to calculate venous thromboembolism (VTE) imaging and VTE event rates.

In the US, a risk-adjusted postoperative VTE rate measure is incorporated into numerous hospital quality improvement programs and public reporting initiatives.

They found that hospitals with higher quality scores actually had higher VTE prophylaxis rates but worse risk-adjusted VTE rates. Increased hospital VTE event rates were associated with increased rates of VTE imaging.

“Hospitals reported to have the highest risk-adjusted VTE rates may in fact be providing vigilant care by ordering imaging studies to ensure that VTE events are not missed”, the authors wrote.

“Patients selecting hospitals according to publicly available metrics may be misled by currently reported VTE performances …”, they wrote.

The researchers warned that hospitals with higher VTE imaging rates could be overusing VTE diagnostic techniques and detecting otherwise asymptomatic VTE, leading to treatment with anticoagulants or inferior vena cava filters that could cause harm.

Professor Day said that although VTE rates were not nationally monitored or reported in Australia, they were considered one of the top hospital quality indicators.

“VTE and pulmonary embolisms are a big issue, bigger than you might think”, he told MJA InSight.

“What this study shows is that we have to be very careful [when evaluating this kind of data]”, Professor Day said.

“We should keep in mind that the higher the quality of hospital, the greater is the intensity of looking for things like VTE, the more will be found and the more prophylactic measures will be taken.

“That might give data that are deceiving and we may be putting in place a perverse incentive to game the system — don’t look for VTEs and you won’t find them, leading to deceptive data about the quality of the hospital.”

Professor Day said the JAMA study also highlighted that the guidelines used by hospitals for VTE prophylaxis in Australia were largely out of date and that new guidelines produced by the American College of Chest Physicians should be adopted. (2)

Dr Diane Watson, chief executive officer of the National Health Performance Authority (NHPA), told MJA InSight that MyHospitals would be reporting data on all of the 17 performance and accountability framework (PAF) indicators of hospital quality in the coming months. VTE rates is not one of the 17 PAF indicators, she said.

“Information supports and drives quality improvement and we need good information”, Dr Watson said.

“The NHPA and MyHospitals contribution to that is to try to make direct, fair comparisons between hospitals using good data.”

The PAF indicators for Australian hospitals include death in low-mortality diagnostic related groups, inhospital mortality rates for acute myocardial infarction, heart failure, stroke, fractured neck of femur and pneumonia; unplanned hospital readmission rates; and health care-associated Staphylococcus aureus (including methicillin-resistant S. aureus) and Clostridium difficile infections. (3)

1. JAMA 2013; 310: 1482-1489
2. Chest 2012; 141: 7S-47S
3. NHPA 2012; Initial indicators for hospitals and Local Hospital Networks