
epocrates
How the FDA may soon regulate the use of ChatGPT
July 6, 2023

While the novelty of ChatGPT and Google Bard are still under intense scrutiny, the FDA has not shied away from the approval of AI-driven medical devices. In fact, more than 500 AI-enabled or machine-learning (ML) devices have already been authorized by the FDA and many have undergone 510(k) clearance. In addition to AI-enabled and ML devices, several ChatGPT-enabled software and devices are also gaining momentum—and FDA approval—in the health care industry (Marr, 2023).
Some of the devices that have passed FDA clearance and are already in use are listed below, along with the design, performance, and safety measures that were carried out to attain FDA’s approval.
1. GE HealthCare Critical Care Suite
GE's Critical Care Suite is a collection of AI algorithms embedded in X-ray systems for quality control, automated measurements, and case prioritization, which can provide accurate and automated measurements of endotracheal tube position, detect nearly all cases of pneumothorax with high accuracy, and automatically analyze images without routing them to a server (GE HealthCare, 2023).
The following quality assurance measures were applied to the development and deployment of Critical Care Suite (U.S. Food and Drug Administration, 2019): risk analysis, requirement reviews, design reviews, testing on unit level (module verification), integration testing (system verification), performance testing (verification), safety testing (verification), and simulated use testing (validation). Furthermore, Critical Care Suite was evaluated on a dataset of 804 chest X-rays, with the algorithm predictions compared to actual readings provided by three independent US-board-certified radiologists.
2. eMurmur ID: AI-powered heart murmur detection
eMurmur ID uses advanced machine learning to identify and classify pathologic and innocent heart murmurs, heart rate, absence of a heart murmur, and S1/S2 markers. Accurately identifying and diagnosing a murmur as innocent vs. needing follow-up helps address unnecessary referrals, emotional stress, and more (U.S. Food and Drug Administration, 2022). eMurmur uses AI-based analytics, a mobile app, and a web portal to confirm the clinician’s diagnosis.
Performance data included software verification, validation testing, electromagnetic compatibility, electrical safety, wireless coexistence, and bench validation testing. The validation testing included 120 subjects, half with a pathological murmur (class I) and half with a confirmed innocent or no murmur (class III). Heart sounds were recorded and analyzed by eMurmur ID and compared to the clinical gold standard reference, as defined by an expert physician and independently verified by cardiac echocardiography (Food and Drug Administration, 2019).
3. GI Genius: Computer-aided polyp detection
This computer-assisted tool assists clinicians in detecting colonic mucosal lesions (polyps and adenomas) in real-time during standard colonoscopy examinations. The device has been trained to process colonoscopy images containing colorectal lesions (U.S. Food and Drug Administration, 2021).
The following non-clinical verification/validation activities were completed prior to approval: verification of the revised software at the system level, validation of the revised software at the user level, verification of protective measures identified by risk management, and completion of electromagnetic compatibility and electrical safety compliance tests. In addition, researchers carried out standalone performance testing to assess device performance using the same test protocol used for the predicate device. The GI Genius did not show any differences in performance and did not raise any questions of safety or effectiveness when compared to its predicate device, enabling the FDA to grant its approval.
4. BriefCase for iPE triage
This radiological computer-aided triage and notification software assists radiologists in workflow triage by flagging and communicating suspected positive cases of incidental pulmonary embolism (iPE) pathologies. The software uses an AI algorithm to analyze images and flag studies that have suspected findings.
A retrospective, blinded, multicenter study was conducted with the BriefCase software to evaluate the software’s performance prior to approval, with the sensitivity and specificity exceeding the 80% performance goal. The BriefCase time-to-notification for an iPE was 4.7 minutes, in contrast to the 220.9 minutes in cases where BriefCase was not in use (U.S. Food and Drug Administration, 2020).
How do current FDA regulations govern the use of AI by medical device developers, manufacturers, and clinicians?
The FDA recognizes that AI and ChatGPT are evolving at warp speed and must work to keep up with the new technology. As the FDA works to define broader LLM-specific regulations and rules for use, the medical industry will require its own set of regulations based on the intended use of the language model.
In addition to the benefits for clinicians, there are several changes to the regulatory oversight process on the horizon. Below is the FDA’s Center for Devices and Radiological Health (CDRH) annual list of guidance documents that may significantly impact AI/ML technologies. This list was released at the end of 2022 with the intention to publish in FY2023 (Buenafe et al., 2023):
- Cybersecurity in Medical Devices: Quality System Considerations and Content of Premarket Submissions
- Content of Premarket Submissions for Device Software Functions
- Transition Plan for Medical Devices That Fall Within Enforcement Policies Issued During the Coronavirus Disease 2019 Public Health Emergency
- Marketing Submission Recommendations for A Change Control Plan for Artificial Intelligence/Machine Learning (AI/ML)-Enabled Device Software Functions
What does the future hold for AI and LLMs in health care?
Implementing AI and ML can reveal exciting potential, but the medical industry and the FDA both agree that AI can be dangerous without regulatory oversight. The FDA is working to make these technologies safer, and as they change and evolve, so will the FDA’s regulatory policies, decisions, and supervision.
As the potential of LLMs and AI grows, figuring out how and where they benefit health care professionals and patients most is also evolving. Many developers in the medical device market are currently looking to leverage it in the following ways:
- Workflow optimization: AI can analyze workflows and identify areas for higher efficiency in health care settings, such as improving patient flow, optimizing scheduling, resource allocation, and giving time back to clinicians to focus on patient care planning, diagnosing, etc.
- Predictive analytics: Predictive analytics can analyze specific patient data and predict the likelihood of developing certain diseases or conditions. This early detection and identification of risk factors can improve patient outcomes and decrease health care costs.
- Decision support: AI software holds the power to analyze large amounts of data compared to what humans can do alone, including analyzing lab results, imaging, and other medical records. Analyzing this data can support effective treatment plans, diagnosis of certain conditions, and beyond.
- Intelligent automation: AI can help with tasks like data entry and other administrative duties that may take up critical time. Freeing up this time would allow clinicians to address higher-priority items.
- Personalized medicine: AI can support clinicians in personalizing treatment plans. AI uses data and algorithms to personalize therapies and preventative measures that are unique to each patient.
References
Baumann, J.. (2023 June 21). ChatGPT poses new regulatory questions for FDA, medical industry. Bloomberg Law. https://news.bloomberglaw.com/health-law-and-business/chatgpt-poses-new-regulatory-questions-for-fda-medical-industry?context=search&index=1.
Buenafe, M., Harper, J., & Gray, Andrew. (2023, March 3). The FDA regulatory landscape for AI in medical devices. Med Device Online. https://www.meddeviceonline.com/doc/the-fda-regulatory-landscape-for-ai-in-medical-devices-0001
Dave, T., Athaluri, S.A., & Singh, S. (2023, May 4). ChatGPT in medicine: an overview of its applications, advantages, limitations, future prospects, and ethical considerations. Frontier Artificial Intelligence. 6. doi: 10.3389/frai.2023.1169595. https://www.frontiersin.org/articles/10.3389/frai.2023.1169595/full
GE HealthCare. (2023). Critical care suite 2.0. https://www.gehealthcare.com/products/radiography/critical-care-suite.
Marr, B. (2023 Mar 2). Revolutionizing healthcare: The top 14 uses of chatGPT in medicine and wellness. Forbes. https://www.forbes.com/sites/bernardmarr/2023/03/02/revolutionizing-healthcare-the-top-14-uses-of-chatgpt-in-medicine-and-wellness/?sh=89c637b6e547.
University of Central Arkansas. (2023). Chat CPT: What is it?https://uca.edu/cetal/chat-gpt/.
U.S. Food and Drug Administration. (2020, August 26). BriefCase for iPE triage. https://www.accessdata.fda.gov/cdrh_docs/pdf21/K211951.pdf.
U.S. Food and Drug Administration. (2019, August 12). Critical care suite. https://www.accessdata.fda.gov/cdrh_docs/pdf18/K183182.pdf.
U.S. Food and Drug Administration. (2019 Mar 19). eMurmur ID. https://www.accessdata.fda.gov/cdrh_docs/pdf18/K181988.pdf.
U.S. Food and Drug Administration. (2021, Jul 23). GI genius. https://www.accessdata.fda.gov/cdrh_docs/pdf21/K211951.pdf.
U.S. Food and Drug Administration. (2022, May 31). eMurmur. https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn.cfm?ID=K220766.
Versaw, R. (2023, June 27). The transformative impact of artificial intelligence in medical tech. Forbes. https://www.forbes.com/sites/forbestechcouncil/2023/06/27/the-transformative-impact-of-artificial-intelligence-in-medical-tech/?sh=6ddc4f451ac9.
TRENDING THIS WEEK