2026 AI Governance Checklist for Healthcare

SECTION 1 AI System Inventory Worksheet

Complete this section first. List every AI tool in use — approved by IT and shadow (unofficial). This is your compliance scope for all sections below.

Discovery instructions

Survey department heads, clinical leads, and IT. Ask specifically: (1) What AI tools does your team use for work, including personal subscriptions? (2) Has any AI tool been purchased or subscribed to without formal IT approval? (3) Does any staff member use a general AI assistant (ChatGPT, Copilot, Claude, Gemini) for work-related tasks involving patient data?

AI Tool Name

Vendor

Dept / Owner

Data Access

BAA?

Encrypted?

Status

Y / N

OK / Gap

Y / N

OK / Gap

Y / N

OK / Gap

Y / N

OK / Gap

Y / N

OK / Gap

Y / N

OK / Gap

Y / N

OK / Gap

Y / N

OK / Gap

Y / N

OK / Gap

Y / N

OK / Gap

Y / N

OK / Gap

Y / N

OK / Gap

Data Access: PHI (Protected Health Info) | PII (Personal Info) | Financial | Operational | Clinical Decision Support

Shadow AI tools identified (list all unauthorized tools found):

SECTION 2 Business Associate Agreement Audit

For every AI tool in Section 1 with PHI access, verify a signed BAA exists before any PHI flows.

HIPAA 2026 requirement

Every vendor that accesses, stores, transmits, or processes PHI on your behalf requires a signed BAA before any PHI flows to their systems. Older BAAs may not cover AI-specific processing — verify coverage explicitly.

AI Vendor / Tool

BAA Exists?

BAA Covers AI?

Expiry / Review Date

Action Required

Y / N

Immediate action: Any tool with PHI access and no BAA must be suspended until the BAA is executed. Document suspension and execution dates.

SECTION 3 Encryption Verification Checklist

Verify encryption for every AI tool with PHI access. HIPAA 2026 mandatory encryption applies to all storage and transmission — including vendor-side logs, caches, and output stores.

Primary database encryption at rest — confirmed and documented

Obtain written confirmation from vendor that primary data store is encrypted at rest using AES-256 or equivalent. Document encryption standard, key management, and date verified.

HIGH

Encryption in transit (TLS 1.2 minimum) — confirmed and documented

Verify all data in transit is encrypted with TLS 1.2 or higher. TLS 1.0 and 1.1 are deprecated.

HIGH

Prompt and conversation logs — encryption status verified

AI tools store prompt history and conversation logs that may contain PHI. Verify encryption status with vendor.

HIGH

Model output caches — encryption status verified

Output caches are often unencrypted by default. Confirm vendor encryption status for all cached outputs.

HIGH

API integration logs — encryption status verified

API integration layers may log full request content including PHI. Confirm logs do not store PHI in cleartext.

HIGH

Fine-tuning or training data — PHI inclusion confirmed absent

If vendor fine-tunes the model using customer data, confirm in writing that PHI is excluded from training datasets.

MEDIUM

Backup and disaster recovery storage — encryption verified

Backup copies must also be encrypted. Verify vendor backup and DR environment meets same standards.

MEDIUM

Encryption key management — documented and controlled

Verify who holds encryption keys and under what conditions the vendor can access encrypted data.

MEDIUM

Encryption gaps identified and remediation plan:

SECTION 4 Human Oversight Documentation

HHS 2026 AI guidance requires documented human oversight procedures for clinical AI. Complete one row per system.

What HHS requires

For every clinical AI system, document: (1) named human reviewer, (2) review frequency, (3) escalation path for unexpected outputs, (4) decision logging mechanism, (5) adverse event reporting process. A general policy statement does not satisfy this requirement.

Named human reviewer designated for each clinical AI system

Each clinical AI system must have a named, accountable individual responsible for reviewing outputs. Document name, title, and designation date.

HIGH

Review frequency documented and schedule established

Define review cadence appropriate to the clinical risk of each AI's decision support role.

HIGH

Escalation procedure written and distributed

Step-by-step procedure for unexpected, potentially harmful, or biased AI output. Who is notified? Who can suspend the tool?

HIGH

AI-assisted decision logging mechanism in place

Log: AI system used, output produced, who reviewed, whether reviewer concurred or overrode, final action taken.

HIGH

Adverse event reporting procedure established

Define process for reporting AI-related adverse events to clinical leadership and HHS under applicable requirements.

HIGH

Clinical staff training on oversight obligations completed

All clinical staff using AI decision support tools must be trained. Document completion dates and sign-offs.

MEDIUM

Oversight procedure reviewed and signed by clinical leadership

Each procedure should be signed by the relevant department head and CMO or designee.

MEDIUM

Human Oversight Log — Complete one row per clinical AI system:

Clinical AI System

Named Reviewer

Review Frequency

Logging Method

Procedure Date

SECTION 5 NIST AI RMF Gap Assessment Scorecard

Self-assess across all four NIST AI RMF functions. Rate each control: Full / Partial / None.

Scoring guidance

Full = documented, implemented, and verifiable with evidence. Partial = policy exists but implementation is incomplete or evidence is missing. None = not in place. Use the Current State column to record your finding.

NIST Function	Sub-Category	Control Description	Target
GOVERN	GV-1.1	AI governance policy established and approved	Full
GOVERN	GV-1.2	AI oversight roles and responsibilities assigned	Full
GOVERN	GV-2.1	AI risk tolerance defined and documented	Full
GOVERN	GV-4.1	AI acceptable use policy distributed to staff	Full
MAP	MP-1.1	Complete AI system inventory conducted	Full
MAP	MP-2.1	PHI exposure classified per AI tool	Full
MAP	MP-3.1	Regulatory obligations mapped per AI system	Full
MAP	MP-4.1	Shadow AI tools identified and documented	Full
MEASURE	MS-1.1	Encryption at rest verified per AI tool	Full
MEASURE	MS-1.2	Encryption in transit verified per AI tool	Full
MEASURE	MS-2.1	BAA confirmed for every AI vendor with PHI access	Full
MEASURE	MS-3.1	Human oversight procedures documented	Full
MEASURE	MS-4.1	AI output accuracy metrics established	Partial
MANAGE	MG-1.1	Identified gaps remediated with documented evidence	Full
MANAGE	MG-2.1	AI incident response procedure written and tested	Full
MANAGE	MG-3.1	AI vendor risk reviews scheduled quarterly	Full
MANAGE	MG-4.1	Regulatory monitoring process in place	Full

NIST Function

Sub-Category

Control Description

Current State

Target

Gap?

GOVERN

GV-1.1

AI governance policy established and approved

Full

GOVERN

GV-1.2

AI oversight roles and responsibilities assigned

Full

GOVERN

GV-2.1

AI risk tolerance defined and documented

Full

GOVERN

GV-4.1

AI acceptable use policy distributed to staff

Full

MAP

MP-1.1

Complete AI system inventory conducted

Full

MAP

MP-2.1

PHI exposure classified per AI tool

Full

MAP

MP-3.1

Regulatory obligations mapped per AI system

Full

MAP

MP-4.1

Shadow AI tools identified and documented

Full

MEASURE

MS-1.1

Encryption at rest verified per AI tool

Full

MEASURE

MS-1.2

Encryption in transit verified per AI tool

Full

MEASURE

MS-2.1

BAA confirmed for every AI vendor with PHI access

Full

MEASURE

MS-3.1

Human oversight procedures documented

Full

MEASURE

MS-4.1

AI output accuracy metrics established

Partial

MANAGE

MG-1.1

Identified gaps remediated with documented evidence

Full

MANAGE

MG-2.1

AI incident response procedure written and tested

Full

MANAGE

MG-3.1

AI vendor risk reviews scheduled quarterly

Full

MANAGE

MG-4.1

Regulatory monitoring process in place

Full

Priority remediation items identified (list top 5 gaps by risk):

SECTION 6 HIPAA 2026 AI Compliance Verification

Mandatory controls only. No risk-based alternatives apply.

Addressable safeguard review completed — all updated to required status

Controls previously classified as addressable must now be implemented or compensating controls documented.

HIGH

Encryption at rest mandatory — verified for all AI systems with PHI access

AES-256 or equivalent mandatory for all PHI at rest across AI systems, logs, caches, backups, and DR environments.

HIGH

Encryption in transit mandatory — verified for all AI data flows

TLS 1.2 minimum for all PHI in transit including API calls, browser/app data, and AI outputs returned to your systems.

HIGH

BAA executed with every AI vendor before PHI access

No AI vendor may access PHI without a signed BAA. Verify current BAAs cover AI-specific processing.

HIGH

Shadow AI policy implemented and distributed to all staff

Written policy prohibiting unauthorized AI tool use with PHI. Include approved tools, prohibited data types, and violation consequences.

HIGH

Human oversight documentation in place for all clinical AI

Per HHS 2026 AI guidance. Absence of documentation is a distinct compliance gap from other HIPAA controls.

HIGH

AI-related breach notification procedure established

Shadow AI incidents and unauthorized AI data disclosures qualify as potential HIPAA breaches under the 2026 rule.

HIGH

Risk analysis updated to include AI systems

HIPAA risk analysis must explicitly include all AI systems, their data access, encryption status, and specific risks.

HIGH

Workforce training updated to include AI security obligations

Training must include AI-specific content: approved tools, prohibited PHI use, shadow AI risks, and reporting procedures.

MEDIUM

AI vendor security assessments documented

Obtain SOC 2 Type II or equivalent from all AI vendors with PHI access. Review annually.

MEDIUM

Audit logs reviewed for AI tool access and use

Configure AI tool access logging where feasible. Review quarterly for unusual patterns.

MEDIUM

Incident response plan updated for AI-specific scenarios

Include playbooks for: shadow AI exposure, AI vendor breach, harmful clinical AI output, unauthorized PHI access.

MEDIUM

SECTION 7 Assessment Summary & Next Steps

Complete this section after finishing all six sections above.

Section

Total Items

Completed

Gaps Identified

Priority Actions

1 — AI Inventory

Worksheet

2 — BAA Audit

Worksheet

3 — Encryption

8

4 — Human Oversight

7 + Log

5 — NIST Scorecard

17

6 — HIPAA 2026 Controls

12

TOTAL

30-Day Priority Action Plan

Priority

Action Required

Owner

Due Date

Status

H / M / L

How to use this checklist