Home Analysis A reality check for a widely-used performance measure

A reality check for a widely-used performance measure

September 2006

Published in the September 2006 issue of Today’s Hospitalist

Doctors know that when it comes to patient care, a one-size fits-all approach can cause problems. That belief now appears increasingly relevant to new pay-for-performance programs and public reporting initiatives.

The new breed of quality measures being introduced by payers like Medicare implies that physicians should meet certain patient care goals 100 percent of the time. Some of the higher profile measures track how well hospitals vaccinate their patients for the flu and pneumonia, for example, and whether they are delivering antibiotics to pneumonia patients within four hours of their arrival at the hospital.

The thinking behind the pneumonia measure is simple: Studies have shown that giving elderly pneumonia patients antibiotics sooner rather than later helps reduce morbidity. But many have questioned the wisdom of saying that all pneumonia patients should receive antibiotics in four hours, largely because of concerns about overmedicating a vulnerable population.

“We’ve all heard the anecdotes about everyone getting antibiotics if they have any hint of a pulmonary infection,” says Mark L. Metersky, MD, a pulmonologist and professor of medicine at the University of Connecticut School of Medicine in Farmington, Conn. “We know that 100 percent isn’t an appropriate target.”

When Dr. Metersky found little in the way of hard data on the topic, he and his colleagues decided to take a novel approach. They reviewed randomly selected charts of Medicare patients who had been discharged from hospitals throughout the country with a diagnosis of pneumonia. They wanted to determine how many of those patients could be definitively diagnosed with pneumonia early enough to allow antibiotics to be given within four hours of coming to the hospital.

Diagnostic uncertainty

What the researchers found, as reported in the July 2006 issue of the journal Chest, was that there was too much “diagnostic uncertainty” to deliver antibiotics to 22 percent of patients.

That uncertainly typically resulted from two common scenarios. In the first, the patient presented in an unusual fashion, but was later found to have clear evidence of pneumonia. Examples included patients presenting with abdominal pain, focal neurologic deficits or ventricular tachycardia.

In the second scenario, patients presented with pulmonary symptoms that could have been caused by heart failure, COPD exacerbation or pneumonia. In some of these instances, pneumonia was identified as one possible cause of problems, but other serious conditions had to be considered.

The study found that a lack of rales, oxygen desaturation or a chest X-ray suggesting pneumonia were all significantly more common in patients where a pneumonia diagnosis wasn’t certain. Another finding will interest anyone trying to diagnose pneumonia in an older population: the role that changes in mental status play in the decision-making process. Acute mental status changes were almost twice as common in patients with diagnostic uncertainty, although those weren’t statistically significant.

Among patients whose diagnosis was initially unclear, researchers disagreed over whether 74 percent of those patients could have been diagnosed early enough to receive antibiotics within four hours.

The source of the disagreement varied, but the article notes that in many cases, researchers disagreed about when ED physicians should go ahead and begin administering antibiotics, typically in patients who had respiratory symptoms that could have been caused by conditions besides pneumonia.

A question of timing

According to the study, that confusion gets at the big question: When should the emergency department administer antibiotics? “Should antibiotics be administered only if the physician is nearly certain that the patient has pneumonia, or even if there is only 20
percent certainly?" the researchers asked in the article.

This wouldn’t be an issue at all, Dr. Metersky explains, if payers like Medicare urged physicians to give confirmed pneumonia patients appropriate antibiotics quickly, perhaps within four hours “but to also accept that the four-hour goal is somewhat arbitrarily drawn. That change in thinking would give physicians a chance to make their diagnosis and give the appropriate medicine as quickly as possible, but without the added pressure of meeting a quality measure that may be tied to financial rewards.

Studies have found that “time to first antibiotic dose” does make a difference; administering antibiotics in less than four hours has been associated with a 15 percent reduction in the 30-day mortality rate among patients not pretreated with antibiotics in the outpatient setting, according to an accompanying editorial in the same issue of Chest.

Dr. Metersky says that the problem arises when the implied goal is 100 percent adherence “and when there is competition for money or for the reputation as the “best.” The natural inclination for many physicians, who are motivated by rankings, is to err on the side of “winning.”

Critics say that’s why payers need to determine a “best-practice” target that doesn’t reflect an all-or-nothing philosophy. Dr. Metersky’s research, for example, found that the goal for antibiotic timing for pneumonia should probably state that 75 percent to 80 percent of patients discharged from the hospital should have received antibiotics in the first four hours “not 100 percent.

Setting an appropriate target
The bottom line, Dr. Metersky says, is that “an implied target of 100 percent leads to inappropriate antibiotic use.” Given the current quality measure for pneumonia, the presence of respiratory symptoms alone may lead to the administration of antibiotics, whether or not there is a confirmatory chest X-ray.

For that reason, Dr. Metersky explains, the goal should come down. “We don’t want hospitals striving to get above 75 percent or 80 percent,” he says. “It appears that you can’t get there without giving antibiotics inappropriately.”

He suggests another helpful change: Publicly report only whether hospitals meet that target range, not the actual percentage achieved. That simple step, he says, could “prevent the negative consequences of competition between hospitals to achieve an unrealistically high performance.”

“If there is a pay-for-performance program based on who is best, and you are competing against a hospital down the street and they get 98 percent,” he explains, “you have to get 100 percent.”

There are signs that the Centers for Medicare and Medicaid Services will address this concern. It will soon be changing its performance measures to “require a pneumonia diagnosis in the emergency department and radiographic evidence of pneumonia before the [time-to-antibiotics] measure is applied, reducing the pressure to administer antibiotics prior to the confirmation of pneumonia,” according to the Chest editorial.

It’s a step in the right direction, but Dr. Metersky says that more research is needed. As more pay-for-performance initiatives begin, he points out, “People need to consciously consider what are the potential consequences when hospitals try and meet the goals by hook or by crook.”

Deborah Gesensway is a freelance writer who reports on U.S. health care from Toronto, Canada.