At the HIMSS23 conference earlier this year, AI was a huge topic, to no one’s surprise. AI has taken the world by storm, and healthcare has been quick to jump on the bandwagon. 
However, the use of generative AI in healthcare has come under scrutiny, and ethical questions are being raised. Concerns include:
- Minimal, if any, regulation and oversight of generative AI technology
- Hasty implementation of the technology without a thoughtful review process
AI Uses in Healthcare
Some of the tasks in which AI is currently involved include:
- Rewriting patient conversations into an ordered format
- Simplification of explanation of benefit (EOB) forms
- Integrating into EHR for tasks such as drafting replies to patient messages, summarizing patient medical histories, translating patient information into different languages and varying reading levels
- Medical notetaking, relieving physicians of many hours of work each day.
Concerns with AI in Healthcare
Several issues have been raised about generative AI, particularly concerning accuracy, impartiality and accountability. In healthcare, it is vital to have a specific, authoritative source of information. But there are no footnotes or other annotations provided with AI-generated content. If the content source is unclear, can the content be trusted? Key point: AI does not deliberate or “think” – it simply predicts the next word group that seems to make sense.
With AI platforms that are large language models, answers that are incorrect or even irrelevant may be given. When biased data is used to train an algorithm or if biased applications are used on results, the results will be biased and that bias will be perpetuated.
Accountability comes into play because currently there is no stakeholder assigned responsibility for accuracy. Should the developer, the owner or the user have accountability? As this is a new and rapidly-developing technology, many stakeholders are urging the tech industry to apply a pause on training large AI systems to develop safety protocols and develop and implement a system of governance for AI.
Is your organization using AI in any form? What issues have you seen?