Ranking Physicians Based on Cost May Be Misleading, Says Study

Health insurance plans that push patients toward physicians who keep medical costs lower are based on unreliable estimates of physician performance and may not save money, according to a study in the March 18 edition of the New England Journal of Medicine.

Described as the first major assessment of physician cost profiling, the RAND Corp. study found that about one-fourth of the 13,788 Massachusetts physicians they reviewed would be misclassified under the system of cost ranking commonly used by insurance plans.

"One of the ideas that is pretty popular for saving costs is to squeeze the doctors. What we are concluding here is that would be great if you really knew which ones were expensive," John L. Adams, the study's lead author and a senior statistician at RAND, tells HealthLeaders Media.

"Our findings raise questions about the utility of cost profiling tools for high-stakes activities, such as tiered health plans, and the likelihood that wide use of these strategies will reduce healthcare spending. Consumers, physicians, and those who pay for healthcare are all at risk of being misled by the results from these tools," Adams says.

The study examined 28 physician specialties and found that only about 40% of physicians had cost profile scores that were at least 70% reliable. Fewer than 10% of physicians had cost profiles that were at least 90% reliable.

Among physicians in a hypothetical two-tiered insurance plan, for example, nearly 40% of internists and nearly two-thirds of vascular surgeons labeled as lower cost were not, the study found. Physicians in surgical specialties appear to have low reliability cost profile scores, while dermatologists' cost profile scores were the most reliable.

American Medical Association President J. James Rohack, MD, says the RAND study "verifies the AMA's longstanding contention that there are serious flaws in health insurer programs that attempt to rate physicians based on cost-of-care."

"The RAND study shows that physician ratings conducted by insurers can be wrong up to two-thirds of the time for some groups of physicians," Rohack says. "Inaccurate information can erode patient confidence and trust in caring physicians, and disrupt patients' longstanding relationships with physicians who have cared for them for years."

"Given the potential for irreparable damage to the patient-physician relationship, the AMA calls on the health insurance industry to abandon flawed physician evaluation and ranking programs, and join with the AMA to create constructive programs that produce meaningful data for increasing the quality and efficiency of healthcare," Rohack says.

Calls by HealthLeaders Media seeking comment from several health insurance industry trade groups were not immediately returned Wednesday.

Adams says he doesn't think revising the evaluation process has to start from square one. "There are a lot of pieces here that could be useful," Adams said. "There has been a failure on a number of levels. The people that use these tools should have done more physician engagement for two reasons: One, the physicians could have made the tool better; and two, why pick a gratuitous fight?"

The Massachusetts Medical Society (MMS) is involved in litigation against the Massachusetts Group Insurance Commission (GIC), which purchases health insurance for state employees and other public sector employees, because of its requirement that health plans tier physicians. That litigation involves GIC and two participating health plans, according to MMS.

Mario E. Motta, MD, president of MMS, said his organization is not against public reporting as long as it is accurate and fair. "It's critically important that patients and physicians get clear, accurate information about the cost and quality of healthcare. But this report, produced by an independent, renowned research firm, clearly demonstrates that these profiling programs fail to accomplish those goals."

Ranking systems not reliable

The RAND researchers analyzed information from insurance claims for 2004 and 2005 from four health plans in Massachusetts that provide coverage to about 80% of non-elderly adults with private insurance. The study examined the costs of treating common illnesses, such as diabetes and heart attack, assigning each care episode to a physician, and creating a cost profile for each physician based on all similar episodes of care.

The researchers evaluated the reliability of physician cost scores by considering factors, such as the number and types of patients physicians treated. The results show that the reliability of cost-profiling scores was unacceptably low for physicians in most of the specialty groups.

Researchers also examined how reliability scores might change under different scenarios, such as requiring at least 30 episodes of treatment to create a profile and different methods for assigning episodes to physicians. While some scenarios modestly increased reliability, the results still fell short.

"These ranking systems may be useful for some purposes, but they are not reliable enough at this point to make decisions about encouraging patients to see certain providers or excluding some doctors from insurance networks," Adams said. "Much work remains to be done to improve these systems before they are used for high-stakes activities."

He adds that the current systems may be useful for warning physicians that their treatment methods may cost more than those of their peers and urging them to reexamine their practice styles.

Adams says cost profiling "can be made better," but it can't be successful until better tools are developed to use claims data and other information to create reliable cost profiles for physicians.

"One of the things you need to do is get a handle on how difficult the provider's patient mix is. It's an illusion that you have it now," he says. "I have hopes that some of the noise can be taken out of this with better data and particularly the stuff we are going to get out of improved information systems. Even the simplest EHR is going to give us blood pressures and whether or not the person is overweight. That is going to be a big help."

"It will be an evolutionary process. I can see these things on the multiyear scale getting up to an adequate standard," Adams says.