Close

ONC Decision Support Interventions Certification Criteria

Released: January 9, 2024

Background

The Office of the National Coordinator for Health Information Technology (ONC) released the Health Data, Technology, and Interoperability: Certification Program Updates, Algorithm Transparency, and Information Sharing (HTI-1)1 final rule in December 2023, with provisions becoming effective 30 days after publication in the Federal Register. The final rule implements provisions directed by the 21st Century Cures Act and makes enhancements to the ONC Health IT Certification Program. ONC intends these provisions to advance interoperability, improve transparency, and support the access, exchange, and use of electronic health information.

Contained within the Health IT Certification Program, the HTI-1 final rule is an expansion of the clinician decision support (CDS) criteria requirements renamed decision support interventions (DSI).2 The requirements on DSI and predictive DSI contained within the HTI-1 final rule mark HHS’ first attempt at regulating AI tools beyond those contained in medical devices and regulated by the FDA. Aimed almost entirely at transparency, the DSI certification criteria will assist those utilizing certified health IT products to understand how DSIs are trained, and the data used to train them. Health IT developers are required to update their certified health IT to meet these new requirements by December 31, 2024, and must begin maintaining the ongoing maintenance of certification requirements on January 1, 2025.

Key DSI Requirements:

  • Includes 14 new source attributes for predictive DSIs that enable or interface with Health IT Modules, providing users with information on what data was used to train the DSI, how the predictive DSI should be used, updated, and maintained, and how the predictive DSI performs using validity and fairness metrics in testing and local data.
    • Those new source attributes also require the inclusion of social determinants of health (SDOH) data as well as other health equity data source attributes.
  • Incorporates new requirements into the Certification Program for Health IT Modules that support AI and ML technology to increase transparency related to DSI created by health IT developers.
  • Adds the CDS Criterion to the list of applicable certification criteria for real-world testing.
  • Requires developers of certified health IT to attest “yes” to an annual review and update of DSI documentation as necessary.
  • Requires developers of certified health IT with certified Health IT Modules to attest “yes” or “no” as to whether their Health IT Module is supplied with one or more predictive DSIs.

Areas of Focus for DSI Requirements:

  • Details and output of the DSI;
  • Purpose of the intervention;
  • Cautioned out-of-scope use of the intervention;
  • Intervention development details and input features;
  • Process used to ensure fairness in development of the intervention;
  • External validation process;
  • Quantitative measures of performance;
  • Ongoing maintenance of intervention implementation and use; and
  • Updates and continued validation or fairness assessment schedule.

Transparency and Efficacy in DSI:

As stated above, one of the requirements within the DSI criterion is that certified health IT must support 13 source attributes for evidence based DSIs and 31 source attributes for predictive DSIs. ONC’s intent is for these attributes to be used to create an industry standard floor of data utilized in the creation and use of DSI. By creating this floor, it will be possible for certified health IT that utilize DSIs to create tools like “advanced structured model cards.”3

The source attributes will also be used to assist organizations in the evaluation of Predictive DSIs to determine if they are fair, appropriate, valid, effective, and safe (FAVES). The FAVES methodology outlined by ONC helps determine a Predictive DSI’s quality and whether the information or recommendations it outputs can be trusted for use. ONC recommends organizations utilize the FAVES methodology when possible, in evaluating their implemented DSIs to ensure predictability and trust.

1 Available at: https://www.healthit.gov/sites/default/files/page/2024-01/DSI_HTI1%20Final%20Rule%20Presentation_508.pdf.

2 Available at: https://ahima.org//media/k1ilww1y/hti-1-final-rule-ahima-supported-provisions.pdf.

3 Available at: https://www.healthit.gov/sites/default/files/page/2023-12/HTI-1_DSI_fact%20sheet_508.pdf.  

 

⭠ Back to Main AI Regulatory Resource page.

Back to top