Double the Savings? ACO Group Questions CMS Analysis of MSSP Performance

Over three years, Medicare Shared Savings Program ACOs saved a gross $1.8 billion, nearly twice as much as the amount CMS has reported, according to a NAACOS-commissioned report. The difference is not in the data itself but how it's analyzed.

The way the government has been analyzing performance data for accountable care organizations in the largest value-based payment model in the country severely underemphasizes the amount of money these ACOs are saving, according to a report released Tuesday.

Since most of the ACOs currently in the Medicare Shared Savings Program (MSSP) are in tracks without downside risk, the findings are poised to fuel debate over the Centers for Medicare & Medicaid Services' plan to require downside risk sooner—part of an overhaul that's projected to reduce the number of MSSP ACOs by nearly 20% over the coming decade.

The study, which was commissioned by the National Association of ACOs (NAACOS) and conducted by the healthcare consulting firm Dobson DaVanzo & Associates, found that MSSP ACOs generated estimated gross savings of $1.84 billion during performance years 2013 through 2015. That's nearly double the $954 million in estimated gross savings CMS had reported.

The report doesn't accuse CMS of fudging the numbers; rather, it attributes the discrepancy to differences in methodology. Instead of relying on the same benchmarking CMS uses to establish ACO targets and calculate penalties or bonuses, the report tabulated MSSP ACO savings using difference-in-differences regression analysis, which it described as "the gold standard for program evaluation."

After accounting for bonuses paid to high-performing ACOs, the NAACOS-backed report estimated MSSP generated net savings of $541.7 million to the federal government for 2013-2015. The CMS benchmarking methodology, by contrast, tabulated a net loss of $344.2 million.

NAACOS President and CEO Clif Gaus, ScD, said the report should "put to rest" any assertion that upside-only ACOs have failed to save Medicare money.

"The findings confirm the wisdom of giving ACOs adequate time to build the care coordination, information technology, and data analytics capabilities needed to manage financial risk successfully," Gaus said in a statement Tuesday morning.

'Gold Standard' Precedent
 

When CMS announced with fanfare last month that the inaugural cohort of 18 Next Generation ACOs had saved $62 million in their first year of operation, Administrator Seema Verma cited the results as "further evidence that ACOs succeed under two-sided risk." The data seem to buttress her support for hastening downside risk under the agency's proposed MSSP ACO overhaul.

Gaus applauded the agency's use of "the gold standard" difference-in-differences analysis, saying it clearly demonstrated the Next Generation ACO model's success. But, foreshadowing Tuesday's report, he also suggested CMS would find "billions of dollars" in savings if it conducted a comparable analysis for MSSP ACO savings using the same standard.

Related: Even Upside-Only Medicare ACO Orgs Have Saved Money

Related: Proposed ACO Models Would Hasten Downside Risk

Related: Next Gen ACO Model Saves $62M in First Year

When asked by HealthLeaders why CMS hadn't conducted the same analyses on both ACO models, an agency press representative noted that MSSP, which was established by statute, has different evaluation requirements than do models tested by the CMS Innovation Center.

Those differing requirements could explain why MSSP ACOs haven't received the same analysis as their Next Generation counterparts, says Aledade founder and CEO Farzad Mostashari, MD, a former National Coordinator for Health Information Technology.

"It's an explanation, but it's not really a reason," Mostashari tells HealthLeaders. "I do agree that they should consider certainly the full benefits of the program when making policy decisions."

At the same time, Mostashari says the NAACOS-backed report doesn't seem to be saying anything with which CMS actuaries would substantively disagree. The agency has acknowledged in its own regulatory impact assessment that benchmarks are underestimating MSSP savings.

"Given this fact, what are the policy implications of that?" he adds.

Joe Damore, vice president of population health for Premier, said in a statement that the report's findings provide a significantly more accurate picture of the impact MSSP ACOs are having.

"We agree that measuring program savings against a benchmark of potential spending is a flawed way to assess true impact," Damore said. "The measure should not be whether spending was reduced relative to a national target that does not take into account regional costs and patient acuity over the contract period, but rather whether ACOs generated any savings for Medicare over and above historic [fee-for-service] spending for the ACO's beneficiaries."

Policy Recommendations
 

To coincide with the release of Tuesday's report, Gaus penned a Health Affairs blog post with Robert E. Mechanic, executive director of the Institute for Accountable Care, outlining three suggested changes to the proposed MSSP ACO overhaul:

1. Don't cut the shared savings rate in half. Currently, MSSP ACOs that hit their targets split the savings 50/50 with Medicare. The proposed rule for 2019 would cut that shared savings rate in half, to 25% in the initial year for the new upside-only track. Gaus and Mechanic said the 50% rate should stay.
    
2. Allow new ACOs to linger in upside-only tracks for at least three years. Organizations that meet certain savings and quality goals could add another two years without downside risk under Gaus and Mechanic's suggestion. (The current cap for upside-only MSSP ACOs is six years, and CMS has proposed cutting that timeline to two years.)
    
3. Allow the beneficiary risk score to vary a bit more. The CMS proposal calls for an aggregate 3% limit over five years in changes to beneficiary risk scores. Gaus and Mechanic contend that limit should be loosened to 5% over the five-year enrollment period.
    

"We agree with the administration that ACOs need to evolve and take on more financial risk, but the administration's own estimates suggest that rather than growing, the Medicare ACO program would shrink by 20 percent over the next decade under its proposed rule. That would be a tragedy," Gaus and Mechanic wrote in the Health Affairs blog. "The changes we propose would go a long way towards improving incentives for growth of Medicare ACOs, while achieving the administration's goal of a faster movement to risk."

Editor's note: This story was updated to include a statement from Premier.