Department of Veterans Affairs
MISSION Act Quality Standards

Response to Request for Public Comment
Published September 22, 2018
Updated October 8, 2018

Download a copy of this paper.

The Veterans Healthcare Policy Institute, a non-partisan think tank focused on the provision of quality healthcare to veterans, is pleased to respond to the Department of Veterans Affairs Request for Information from the public to assist in implementing section 1703C(a)(3)-(4) of title 38 United States Code, added by section 104(a) of the VA MISSION Act of 2018. This is a unique opportunity to improve not only the quality of VA’s internal hospital care, medical services and extended care, but also that provided by Veterans Community Care Program (VCCP) partners as well.

At the heart of the MISSION Act’s goal of enhancing care is the measurement of quality. Quality metrics are to be utilized to, (a) compare VA to VCCP services so that veterans can make informed healthcare decisions, (b) designate underperforming VA clinics whose patients should be granted automatic VCCP vouchers, (c) renew contracts for VCCP providers, and (d) determine whether the VA should broaden pilot models of delivering private sector healthcare.  

Numeric scores for quality will be published on public websites for cross-sectional review by veterans and administrators when choosing providers. The basic presumption is that these numbers, on the face of it, will confirm whether “better quality” exists in the VA or VCCP. However, three countervailing deficiencies call that premise into question:  

  1. Use of substitute measures of quality. For many diagnoses, there are no available quality metrics, and instead, tangential measures are substituted.

  2. Comparison to the wrong population. Contrasting VA to non-VA health care is invalid because they are apples to oranges comparisons. The veteran and civilian populations are too dissimilar.

  3. Lack of sufficient data. The quality of VA and VCCP care cannot be accurately compared because many VCCP providers do not consistently report data.

Unless these problems are resolved, there is danger that veterans’ healthcare could be eroded rather than enhanced. Vouchers may be granted to tens of thousands of veterans without any assurance they will receive better care in their community. That in and of itself would be a grave disservice to our veterans. But it may also potentially harm veterans by accelerating a one-directional flow of patients and taxpayer dollars out of the VA to private sector providers, many of who are ill-equipped to care for veterans’ complex needs. This will progressively diminish the option for other veterans to seek VA care, since payment for community services comes at the expense of existing VA facility staffing, services, and programs.  

This article identifies the sections of the legislation that allude to quality standards, reviews the shortcomings, and makes recommendations that would accomplish the bill’s quality objectives.  

Sections of the VA MISSION Act That Reference Quality Measurement

Sec. 101. Veterans Community Care Program

  • Covered veterans can receive care in the community if a medical Service Line (i.e., a VA clinic) is not meeting VA’s standards for quality (and access).

  • Quality at VA clinics is compared to two or more quality measures at non-VA clinics.

  • VA monitors quality of care furnished through network providers.  

Sec. 102. Contract Agreements with Non-VA Providers

  • The quality of care is considered when renewing contracts of large community care entities (exceeding $5 million annually).

Sec. 104. Access and Quality Standards

  • By June 2019, VA publishes quality ratings of VA facilities on CMS’s Hospital Compare website. This is supposed to offer veterans relevant comparative data to make informed decisions regarding their health care. Publishing of VCCP data is not specified.

  • Veterans are surveyed on their satisfaction with quality of VA care regarding timeliness, effectiveness, safety and efficiency.

Sec. 106. High Performing Integrated Health Networks

  • A Quadrennial Review occurs to assess the effects of quality (and access) standards on capacity.

Sec. 133. Competency Standards for Non-VA Providers

  • VA establishes standards for VCCP providers for the provision of evidence-based treatments (EBT) in post-traumatic stress disorder (PTSD), military sexual trauma-related (MST) conditions, and traumatic brain injuries (TBI).

Sec. 152. Center for Innovation for Care and Payment

  • VA establishes a Center for Innovation for Care and Payment to conduct up to 10 pilot programs intended to cut costs without sacrificing quality of care. That includes a pilot program for an insurance model in which the VA is more of a payer than a provider of care.  

Problems with Measuring Quality of Veterans’ Healthcare

1. Quality measures can miss the relevant processes and outcomes that directly relate to treatment.

The MISSION Act dictates that relevant comparative quality metrics are to be made available to covered veterans and administrators to aid them in making informed health care decisions. But quality measures can miss the outcomes and processes that are most consequential to a patient’s diagnosis/symptoms/functioning.

Take for example the evaluation of quality care for PTSD, a signature disorder among veterans, and an emphasis of Section 133. The prevailing PTSD treatment standard is the provision of one of four first-line, evidence-based treatments as advocated by experts in the VA/DoD Clinical Practice Guidelines for PTSD and Acute Stress Reaction1 and endorsed by RAND.2 The use of PTSD EBTs are not reported on CMS Hospital Compare and VA’s Access to Care websites. Nor do these websites reference the recommended assessment of patients’ PTSD symptoms within the initial month of services. Nor do they document the recommended six-month evaluation of symptom and functional improvements. In fact, they don’t even list PTSD as a searchable category. The closest available quality metrics pertain to screening and management of depression and alcohol use, diagnoses that often co-occur with PTSD, but are distinct.

In short, by next spring when systems of monitoring quality are supposed to be implemented, there will be no scores available to veterans and administrators who wish to make decisions about the quality of PTSD treatment provided by the VA versus that provided by VCCP. Applying tangential metrics, such as for depression management, is not an adequate substitute.

Further, section 133 stipulates VCCP clinicians must fulfill PTSD training standards established by the VA. The MISSION Act does not specify that VCCP provider training will be as rigorous as the longitudinal training the VA requires of its own PTSD clinicians. Offering abbreviated one-shot trainings would be insufficient, since skills acquired without follow-up feedback and supervision have been shown to decay back to pre-training ability. Even if VA’s high internal standards were instituted for VCCP clinicians, the MISSION Act did not obligate them to post their use of EBTs or outcomes of treatments.

Not every diagnosis is as lacking in available quality measures as is PTSD. For example, if veterans needed cataract surgery, the Hospital Compare website displays scores for improvement in patient’s visual function within 90 days following surgery. But for a sizable number of conditions there is no searchable listing by disorder, no listing of treatment outcomes and no listing of whether evidence-based treatments are used. In those circumstances, veterans and administrators will be left to make healthcare decisions without applicable, relevant information, which was the explicit aim of the legislation.

2. Contrasting VA to non-VA care of veterans may be invalid because the compared populations are disparate.

Comparing VA to the community is error-prone because private sector quality scores are derived from non-veteran patients who, on average, are younger and have far fewer medical and mental health conditions than do veterans. Weaker outcomes are inherent for veterans with more severe symptoms and complex co-morbidities.

Unless community providers are required to keep track of data on veterans referred through the VCCP, apples-to-apples comparisons of quality care will be challenging.  

An approximate remedy to this type of obstacle was reasserted by the National Quality Forum’s Measure Applications Partnership (MAP),3 which consists of 150 healthcare leaders from 90 stakeholder organizations. MAP recommended that measures be accurately risk-adjusted (i.e. for age, comorbidities and past medical history) to enable fair and valid comparisons across disparate groups.

There’s an additional flaw intrinsic to Section 101. In identifying which of the VA clinics (up to 36) are underperforming and must provide vouchers for community care, an algorithm compares a clinic’s scores to two or more quality metrics at non-VA clinics. The legislation left open the possibility that non-VA clinics from other regions could be selected for comparisons. Were that to occur, VA clinics could be targeted even when their quality is superior to their own community’s.

3. Comparing the quality of VA to non-VA care may be flawed because non-VA providers don’t regularly report performance data.

The VA Office of Community Care has indicated that the majority of individual community providers are unlikely to agree to share detailed quality and performance data with VA due to the costs/burdens of such reporting. That’s been the case during the four years of the Choice Program.  

Clinic level data suffers the same deficits. Quality metrics are not reported for community Service Lines.

There isn’t a workaround to this problem. The VA can’t extract data from non-VA health care records needed for comparisons. RAND researchers noted such limitations in a recent analysis of the quality of care in the military health system.2 They were unable to utilize the records of purchased care in the community because of “the fact that the medical records were not accessible for abstraction.”

Until complete community data is available, we cannot compare differences. How would we know whether we are referring veterans to providers whose care is of worse quality?    

Discussion/Recommendations

Given the tight timeline, the VA may be tempted to consider adopting systems of measuring quality that are expedient, even though incomplete and/or tangential. MAP cautioned against federal programs taking shortcuts at times when obtaining meaningful quality data takes effort: “There may be negative unintended consequences if low burden measures are prioritized over meaningful measures with a higher burden.” Using substitute data is not a solution.

Just because something is labeled as “quality” and a number is affixed to it, this may have little bearing on whether quality care is in fact being provided to a veteran. As Albert Einstein is credited with observing: “Not everything that can be counted counts.”

Recommendations:

  • Before implementation of the VA MISSION Act and further expansion of VCCP, the VA should establish valid quality metrics for all healthcare conditions for

    • patient symptom / functional improvements (using subjective Patient Reported Outcome Measures, i.e. PROMs, and objective outcome metrics)

    • provider use of standard of care first line treatments and screenings (process metrics)

  • determine that the scores continue to exhibit reliability and construct validity 

  • Require quality scores to be listed according to diagnosis/condition so that veterans can readily search according to their disorder.

  • Require that the metrics used for determining VA and non-VA provider performance are identical.

  • Require that metrics are based on comparable populations. Require that the private sector keep track of the data on veterans referred through the VCCP so that the quality of care to veterans in the community and in the VA can be correctly compared. Until that occurs, ensure accurate risk-adjustments are applied.

  • Prior to final determination of underperforming 36 clinics and issuance of vouchers for non-VA care, require that VA quality metrics be compared to local clinics. Ensure that the quality of VCCP care is demonstrably better than VA’s before referring veterans.

  • Require all VCCP providers who treat veterans with PTSD, TBI and MST-related conditions be subject to the identical training and competence standards as are VA providers.

  • Patient satisfaction and patient experience are important components of health care but are different matters than outcomes and processes. Ensure that when scores for patient satisfaction/experience with care are obtained, they are not used in lieu of other quality measures.  

  • Set expectations for VCCP providers to use electronic health records that can be accessed by VA and used to evaluate quality of care.

Conclusion

Before rushing to expand private sector services for covered veterans, it’s imperative to first guarantee that the systems used to judge the quality of care are accurate, relevant, transparent, and accountable.



Footnotes

  1. VA/DoD Clinical Practice Guidelines for the Management of Posttraumatic Stress Disorder and Acute Stress Reaction 2017.  https://www.healthquality.va.gov/guidelines/mh/ptsd/

  2. Hepner, Kimberly A., Roth, Carol P., Sloss, Elizabeth M., Paddock, Susan M., Iyiewuare, Praise O., Timmer, Martha J. & Pincus, Harold A. Quality of Care for PTSD and Depression in the Military Health System: Final Report. Rand Health Q. 2018 Apr; 7(3): 4.

  3. http://www.qualityforum.org/Publications/2018/03/2018_MAP_Clinicians_-_Final_Report.aspx