When an AI tool processes protected health information on behalf of a covered entity, HIPAA applies, and that vendor becomes a business associate under the law. Before any rollout, we’d tell you to lock down three things: a signed Business

Associate Agreement, a Security Risk Analysis focused on how the AI handles data, and technical safeguards like encryption, role-based access, and audit logging. A vendor’s “HIPAA compliant” badge on a marketing page means nothing without a contract and verified controls behind it.

TL;DR:

  • HIPAA applies when a covered entity sends PHI to an AI vendor for processing; patients’ independent use of consumer chatbots generally falls outside it.
  • Name the specific AI service and cloud subcontractors in the BAA; verify that browsing, plugins, and connectors remain covered before enabling them.
  • Ask whether prompts and outputs are retained or used for model training, and document deletion timelines, encryption standards, and who controls revocation keys.
  • Clinical notes and EHR connected billing workflows expose identifiable PHI; population health analysis depends on correct deidentification, especially for rare conditions or small groups.
  • After launch, repeat risk assessments annually or after material changes, monitor unusual access, and remember breach notification timing begins when exposure is discovered.

Table of Contents

When does HIPAA actually apply to AI?

HIPAA’s reach depends on who is touching the data and why, not on whether the tool involved is called “AI.” A covered entity (a hospital, clinic, insurer, or clearinghouse) that hands protected health information to a vendor for processing creates a business associate relationship, and that relationship triggers HIPAA obligations regardless of whether the processor is a person, a script, or a language model. The HIPAA Security Rule factsheet from HHS confirms that covered entities and their business associates must implement administrative, physical, and technical safeguards whenever electronic PHI is created, received, maintained, or transmitted, and the agency’s proposed updates would require annual written verification from a subject matter expert that those safeguards are actually in place.

Where HIPAA does not reach is just as important to understand. A patient who types symptoms into a general-purpose consumer chatbot, on their own initiative and outside any relationship with their provider, is not creating a HIPAA-covered disclosure because no covered entity is involved. Scholarly analysis of AI chatbots and HIPAA compliance describes exactly this gap: developers and vendors of large language models become business associates when they process PHI for a covered entity, but many consumer-facing AI products fall entirely outside that framework, leaving patients exposed to re-identification risk and FTC-level privacy rules instead of HIPAA’s.

A few scenarios that commonly confuse teams:

  • An AI scribe that listens to a clinical encounter and drafts notes for the EHR is processing PHI on the provider’s behalf and needs a BAA.
  • A cloud-hosted LLM API that your billing team queries with patient account numbers is a business associate, even if the vendor never views the raw data in plain text.
  • A wellness app a patient downloads independently, with no contract between the app and their provider, generally sits outside HIPAA.
  • An analytics vendor using properly de-identified data under the Safe Harbor or Expert Determination method may fall outside PHI rules entirely, but the de-identification has to be done correctly first.

Core safeguards every HIPAA-safe AI deployment needs

The Security Rule groups requirements into three buckets, and AI adds a fourth layer of practical detail on top.

  1. Administrative safeguards. Write down who owns vendor risk management, train your workforce on what they can and cannot paste into an AI tool, and keep a documented Security Risk Analysis specific to each AI use case rather than a generic, organization-wide one.
  2. Physical safeguards. Control which devices can access PHI-handling AI tools, and know where the underlying servers live, whether that’s a vendor’s cloud region or your own data center.
  3. Technical safeguards. Require encryption in transit and at rest, enforce role-based access control and multi-factor authentication, and keep audit logs that capture who queried what, when.
  4. AI-specific controls. Confirm whether the vendor retains your prompts or outputs for model training, what “modified retention” actually means in their configuration, and whether de-identification happens before data ever reaches a third-party model.

Encryption standards worth asking for by name include AES-256 for data at rest and TLS 1.2 or higher in transit. Key management matters too: who holds the keys, and can your organization revoke access independently of the vendor. Our own guidance on data governance for AI walks through how to structure these decisions before a single prompt touches patient data.

Pro Tip: Ask every AI vendor, in writing, whether enabling a specific feature (like browsing, plugins, or third-party connectors) pulls your data outside the scope of their BAA. Many HIPAA-eligible products lose that status the moment a non-covered feature gets turned on.

Where AI touches PHI most: priority use cases and their risk points

Not every AI use case carries the same exposure, so triage your attention toward where PHI actually flows.

  • Clinical documentation and note automation carries high risk because raw dictated audio and generated text both contain identifiable patient details, and a careless prompt or output log can leak PHI outside your controlled environment.
  • Imaging and diagnostic AI intersects with device regulation: the FDA’s guidance for AI-enabled device functions recommends lifecycle management, transparency about model limitations, and a predetermined change control plan, which matters whenever an AI tool is part of a regulated diagnostic workflow.
  • Patient-facing chatbots sit at the sharpest edge of the covered-entity boundary. The moment a chatbot is deployed on the provider’s behalf to triage symptoms or schedule care, it needs a BAA; a general consumer assistant a patient uses independently does not.
  • Revenue cycle and administrative automation (claims processing, eligibility checks, denial management) often integrates directly with EHR systems, which means every API connection point needs its own audit trail.
  • Population health analytics built on de-identified data looks safer on paper, but poorly executed de-identification creates real re-identification risk, especially with small sample sizes or rare conditions.

The vendor vetting checklist: what to demand before you sign

Treat every AI vendor the way you would treat any other business associate, with contract language and technical proof, not sales copy.

  1. Require an explicit BAA that names the specific AI service you’re using, not a generic master agreement, and confirms coverage of any subcontracted cloud providers.
  2. Ask for the cloud hosting chain. Under HHS guidance on cloud computing, a cloud service provider that creates, receives, maintains, or transmits ePHI is a business associate even if it never holds the decryption key, so every layer in the stack needs its own BAA coverage.
  3. Get written confirmation of encryption standards (AES-256 or equivalent), audit log retention periods, and RBAC and MFA enforcement, not a vague “enterprise-grade security” claim.
  4. Pin down data handling specifics: default retention period, deletion timelines, and whether your prompts or outputs are ever used to train the vendor’s models. Ask specifically about a no-retention or modified-retention configuration.
  5. Request operational proof: recent penetration test summaries, SOC 2 or ISO 27001 attestations, incident response SLAs, and confirmation of annual third-party audits.
  6. Walk away from any vendor that won’t sign a BAA, gives vague answers about training data use, or operates under a consumer-grade terms-of-service agreement rather than an enterprise contract.

A sample set of BAA provisions from HHS OCR is a useful starting template, though your legal team should tailor it to the specific AI workflow. For a deeper technical review of what’s actually happening inside a vendor’s black box, our breakdown of why AI applications need a professional code audit covers the gap between a vendor’s claims and its actual implementation.

How to roll out AI into clinical workflows without creating new risk

A phased approach protects you from the two most common failure modes: moving too fast and skipping the paperwork, or moving so cautiously nothing ever ships.

  1. Start with a narrow pilot. Pick one workflow, document exactly where PHI enters and exits the system, and involve the clinicians who will actually use the tool in scoping the use case.
  2. Conduct and formally record a Security Risk Analysis specific to that AI use case, not a reused template from a prior assessment.
  3. Execute the BAA and verify the vendor’s actual configuration options, whether that’s a locally hosted model, an API with modified retention enabled, or a fully managed on-premises deployment.
  4. Build out the technical deployment checklist: network segmentation separating the AI tool from unrelated systems, encryption at every hop, RBAC tied to job function, audit logging turned on from day one, and a clear key management policy.
  5. Train clinicians and staff on what the tool does and doesn’t do, stand up a monitoring dashboard for anomalous access or exports, and write a rollback plan before go-live, not after an incident.

Our AI agent security playbook covers the segmentation and access-control decisions in more depth if you’re deploying autonomous agents rather than a single-purpose tool.

Pro Tip: Gate go-live on a signed BAA and a completed Security Risk Analysis. If either is missing, the pilot isn’t ready for real patient data, no matter how good the demo looked.

Two safeguards required before AI pilot

Keeping AI HIPAA-safe after launch: monitoring, audits, and incident response

Compliance isn’t a one-time gate. It’s a cadence.

  • Re-run your Security Risk Analysis annually or whenever the AI use case materially changes, and reassess vendors on the same schedule, consistent with the direction of HHS’s proposed Security Rule updates, which would require annual written verification from a subject matter expert for every business associate.
  • Keep audit logs long enough to support both internal review and a potential OCR investigation, and set alerts for unusual PHI access patterns or bulk exports.
  • Monitor model performance over time, including subgroup checks for bias, since the FDA’s AI-enabled device guidance treats transparency and bias management as lifecycle requirements, not one-time checks at launch.
  • Have a written incident response plan specific to AI-related PHI exposure, including who coordinates breach notification and when OCR reporting thresholds are triggered.

AI-related breaches involving PHI still fall under HIPAA’s standard breach notification framework established by HHS, which means the clock for assessment and notification starts the moment exposure is discovered, not when the root cause is fully understood.

Our guide to AI observability covers the monitoring and drift-detection piece of this in more detail, particularly for teams running their own models rather than relying entirely on a vendor’s dashboard.

What a technical partner actually does in a HIPAA-safe AI rollout

We built our approach around the problem most healthcare teams hit: AI-generated or vendor-assembled code that looks production-ready but hides PHI handling mistakes a reviewer would catch in an afternoon. Our code audits and reviews catch exactly those gaps, whether a tool was built in-house, by a prior contractor, or by an AI coding assistant. We also build AI workflow integrations and local model deployments for teams that want PHI to stay inside their own infrastructure instead of crossing into a third-party API, and we set up monitoring, logging, and alerting so access anomalies surface before they become incidents. We have relied on this combination of senior engineering review and AI-assisted workflow design to get software across the finish line without cutting corners on security.

Build it in-house or bring in a partner?

Three questions settle this faster than most committees expect: how much PHI the tool will actually touch, how mature your internal security practice already is, and how fast you need to move. High PHI exposure paired with a thin internal security team is the clearest signal to bring in a managed partner rather than learning HIPAA’s technical safeguards through trial and error on live patient data. If you do build internally, staff for it properly: a security risk analysis is not a checkbox a developer fills out between sprints, and vendor contract review belongs with someone who reads BAAs for a living.

— Chad

Get your AI integration audited before it touches patient data

We review AI-generated and vendor-built applications for exactly the gaps that turn into HIPAA violations: missing encryption, loose access controls, and vague data retention settings.

Bowtie

Our AI services cover code audits, AI agent and workflow automation, and local model deployment for teams that need PHI to stay off third-party servers entirely. If you’re mid-build or already live and want a second set of eyes before an audit finds the problem for you, start with our AI workflow integration plan and get a straight answer on what needs fixing before launch.

This article is general information, not a substitute for advice from a qualified lawyer. Consult a qualified legal professional about your own circumstances before acting on anything here.

FAQ

Are there any AI platforms that are HIPAA compliant?

Some AI vendors offer HIPAA-eligible configurations, meaning specific features become compliant only when a signed BAA is in place and certain settings (like modified data retention) are enabled. A platform’s general marketing claim of being “HIPAA compliant” is not sufficient on its own; the actual contract and configuration determine coverage, and some features can fall outside the BAA entirely when a third-party connector is turned on.

Does HIPAA apply to AI?

HIPAA applies to AI tools when they process protected health information on behalf of a covered entity, which makes the AI vendor a business associate under the Security Rule. It does not apply when a patient uses a consumer AI tool independently, with no covered entity involved in that interaction.

What are the top AI applications in healthcare?

The highest-impact applications include clinical documentation and note automation, imaging and diagnostic support, patient-facing chatbots for triage and scheduling, revenue cycle and claims automation, and population health analytics built on de-identified data.

What is the 30% rule for AI?

Organizations should rely on documented Security Risk Analyses, signed BAAs, and the technical safeguards required by the Security Rule rather than informal percentage-based rules of thumb.

Sources