When AI Informs Diagnosis: Privacy, Consent, and Liability Considerations

Artificial intelligence (AI) tools are increasingly transforming the healthcare landscape, with the potential to improve accuracy, efficiency, and accessibility across a wide range of applications. From diagnostics to administrative tasks, AI may offer significant benefits for physicians, physician groups, and healthcare systems. This article focuses on concerns about patient privacy, consent, data governance, and bias raised by the rapid integration of these tools.

IMAGING AND DIAGNOSTIC TECHNOLOGY

AI-powered imaging and diagnostic technologies can analyze medical images to identify abnormalities and detect conditions such as cancer, stroke, or diabetic retinopathy with high accuracy. These systems are medical devices regulated by the US Food and Drug Administration (FDA). They are trained on large datasets of labeled images and learn to recognize subtle patterns that may elude human clinicians. Such datasets are derived from publicly available archives (e.g., The Cancer Imaging Archive or OpenNeuro) and from imaging data collected in clinical settings. Once trained, AI systems can prioritize cases for review or assist physicians by identifying areas of concern, making their review more efficient.

Considerations

These tools rely on large datasets, which may raise concerns about patient consent and authorization. The Health Insurance Portability and Accountability Act of 1996 (HIPAA) may permit use or disclosure of properly de-identified data without individual authorization, but organizations should document the de-identification method, assess actual-knowledge and re-identification risk, and use data-use terms that prohibit re-identification, unauthorized model training, and onward transfers. Even where identifiers are removed—such as by removing the 18 personal identifiers listed by HIPAA, 45 CFR § 164.514(b)(2)—depending on the data used, there remains a chance of re-identification.

The growing amount of personal data collected by companies increases the likelihood of enabling re-identification of data by combining multiple de-identified data points. HIPAA de-identification also does not end the inquiry; state consumer-health-data laws, human-subjects research rules, Institutional Review Board approvals or waivers, and contractual restrictions may still constrain AI training, validation, or commercialization.

The “black box” nature of some models makes it difficult for physicians and patients to understand how a diagnosis was reached, complicating informed consent and medical liability. Although FDA considers whether the intended user can understand the output of an AI device, the complexity of these models is such that when an AI model flags a lesion as suspicious or assigns a high probability of cancer, a human may not easily understand the underlying reasoning. This may make it more challenging for clinicians to validate results, reconcile results with other findings, or communicate findings to patients. These issues raise questions about whether and how to ensure patients are adequately informed about their care and able to consent.

Where AI output is used in clinical decision-making, organizations should distinguish between patient-facing informed consent, privacy notices, and internal clinical governance. Consent concerns should be tied to whether the patient is being asked to accept care informed by AI, whether identifiable or regulated data is being used, and whether the AI tool materially affects diagnosis, treatment, or prioritization. Patients have a right to understand the basis for their medical decisions and the risks involved in their choices. And if neither the physician nor the AI can fully explain a diagnosis, there are significant questions about the adequacy of the patient’s consent.

This opacity also raises liability questions: if a physician follows an AI recommendation that proves incorrect, who is responsible—the physician, the hospital, or even the AI developer? Conversely, if a physician overrides the AI and a negative outcome occurs, could that also lead to liability? Without a clear understanding of how AI models reason through imaging data, providers may face uncertainty regarding legal and ethical responsibilities. As regulators and courts begin to grapple with these questions, physicians in image-reliant specialties who use these AI tools should proactively assess their legal risks.

Moreover, if the data inputs used for training or validation lack demographic, socioeconomic, geographic, device-type, or clinical-setting diversity, the resulting models may underperform for certain populations, inserting unintended (and potentially unknown) bias into analytic results.

REMOTE PATIENT MONITORING

AI-integrated remote patient monitoring (RPM) platforms collect and analyze continuous data from wearable devices, such as heart rate monitors, glucose sensors, and sleep trackers. These tools use machine learning algorithms to detect anomalies in real-time and alert healthcare providers to potential issues before they escalate. They enhance chronic disease management and post-discharge care by enabling treatment in a patient’s home.

Considerations

The constant flow of sensitive biometric data introduces substantial privacy and regulatory risks. Patients may not fully understand the extent or purpose of data collection, and RPM platforms are not automatically outside HIPAA. HIPAA may apply when the tool is provided by or on behalf of a covered entity or business associate; direct-to-consumer apps and connected devices outside that relationship may instead face the Federal Trade Commission (FTC) Health Breach Notification Rule, state consumer-health-data laws, and general consumer protection requirements.

Smart devices that collect health-related or health-adjacent information now come in many forms, from bodily accessories like watches and rings to beds, scales, and more. Many of these devices collect a broad array of data, but privacy notices and data collection practices aren’t always easy to find. Some collection practices are disclosed with the product itself, on the manufacturer website, or embedded with an associated smartphone application. Entities selling these devices and providing these apps need to be mindful of how they are giving notice to consumers, explaining what data is collected, and how the consumers’ data is used.

Even with adequate notice and disclosures, other risks persist. Because of the opportunities and analytics AI can offer data processors, device manufacturers should be careful to not inadvertently enter the healthcare space, where they could face allegations of unauthorized practice of medicine. For instance, a watch that purports to diagnose heart attacks or high blood pressure could expose the manufacturer to liability. Medical diagnoses remain within the purview of professionally trained providers. Even so, patients over-reliant on device data could pursue negligence, misrepresentation, and other legal claims without adequate warnings and notice provisions.

Additionally, state-level regulatory requirements can create significant compliance challenges. States like Connecticut, Nevada, and Washington have specific consumer health data protection laws that regulate health-related and health-adjacent data (as distinct from healthcare and medical records). Providers and technology companies using RPM may need legal counsel to navigate specific requirements on collection, use, sharing, and consent. Broader consumer privacy laws like the California Consumer Privacy Act may also apply, granting consumers various rights including how their data is accessed, shared, and/or deleted.

Finally, many RPM systems also depend on third-party vendors for cloud storage, data analytics, or device integration. This may increase the risk of data breaches or unauthorized use. If a vendor misuses consumer health data or suffers a security lapse, the manufacturer could still be held accountable depending on applicable state laws and the contracts at issue. Without a clear legal framework or contractual protections, companies risk unintentionally violating state laws or consumer expectations—exposing themselves to regulatory penalties, reputational damage, and litigation.

Given the evolving legal and regulatory considerations in the increasingly intertwined AI, healthcare, and smart device spaces, interested persons should consider consulting a qualified privacy and healthcare attorney.

For more AI in healthcare news, subscribe to Health Law Scan and explore all articles in our AI in Healthcare series.