DermGPT Updates
The DermGPT team has had a wonderfully busy year and we have been amazed by the support from our dermatology community.
DermGPT, a groundbreaking language-based AI model designed specifically to assist dermatologists in their everyday clinical practice.
DermGPT was developed with the aim of enhancing clinical decision-making, improving patient care, and optimizing workflow efficiency. By leveraging the power of natural language processing and machine learning, DermGPT has the ability to analyze vast amounts of dermatological data, research papers, and clinical guidelines to provide accurate and up-to-date information at your fingertips.
Follow us here for product updates and reach out to us with any suggestions, collaborations, research ideas!
All the Best,
Team DermGPT
Articles included in the learned language model.
Dermatologists using DermGPT.
Queries answered by DermGPT.
Congratulations to Medical Students Payton Smith, Kathryn Haran, Chandler Johnson and Dr. Allison Kranyak on their poster presentation incorporating AI at the UCSF Annual Research Symposium. San Francisco, CA. May 2024.
Evaluating the Performance of DermGPT in Responding to Psoriasis-Specific Patient Inquiries: A Pilot Study
University of California
San Francisco
Payton Smith1, Kathryn Haran1, Chandler Johnson1, Allison Kranyak1, Tina Bhutani1, Faranak Kamangar1, Wilson Liao1 1University of California, San Francisco; 2Palo Alto Medical Foundation
Introduction
Artificial Intelligence encompasses a range of technologies, including Machine Learning, a domain that further branches into methodologies such as Supervised and Unsupervised Machine Learning, as seen in Figure 1.[1]
These branches are crucial to the development of large language models. Some of the most advanced instances in this category are models like GPT-3 and GPT-4, which have been designed to process vast amounts of data and have a high number of parameters, enhancing their ability to understand and generate human-like text.[2] These branches of Machine Learning are the foundation of DermGPT, a specialized adaptation of the generative pre-trained transformer (GPT) model (Figure 2.) that is fine-tuned to interpret dermatological inquiries based on >3,000 peer-reviewed publications. It serves healthcare professionals by offering evidence-based replies to patient questions, providing differential diagnoses, and assisting with administrative tasks such as drafting insurance denial letters.
While DermGPT does not have diagnostic capabilities, its strength lies in supporting healthcare providers with information that helps guide clinical decisions. One of the challenges faced by AI systems like DermGPT is the phenomenon of "artificial hallucinations," where the model might generate incorrect or misleading information based on its training data limitations. [3] However, the system combats this by continuously learning from peer-reviewed dermatological literature, ensuring its responses are up-to- date with the latest research and best practices. This commitment to ongoing learning helps maintain trust among healthcare providers, primary care professionals, and patients in the system's utility and accuracy.
Our research project, which focuses on psoriasis-specific patient inquiries, aims to evaluate DermGPT's performance. This will help determine how effectively DermGPT can support healthcare professionals dealing with psoriasis and potentially guide future improvements, ensuring the technology remains a valuable asset in dermatological practice.
Figure 1. A flowchart representing different branches within the field of Artificial Intelligence (AI).
Figure 2. Diagram of machine learning pipeline converging Generative Al and large language models like DermGPT.
Methods
Selection of Inquiries:
A total of 15 inquiries specific to psoriasis based on in-basket messages and common internet searches were selected by a panel of dermatological professionals. These inquiries encompass a range of topics commonly encountered in clinical practice, including symptom management, treatment options, and long- term care strategies (examples in Table 1).
Interaction with DermGPT:
The selected inquiries were input into DermGPT, and the responses were recorded verbatim. Each inquiry was presented in a standardized format to ensure that the responses could be fairly evaluated. To prevent any prior context bias, a new chat was initiated for each question. Evaluation Process and Methodology:
Responses were then evaluated as "appropriate" or "inappropriate" based on majority agreement among the dermatologic professionals. We selected three reviewers and three repetitions for each question.
Results
A total The effectiveness of DermPT's responses was evaluated by gauging the accuracy of the responses and their comprehensibility to the general public. DermGPT produced 87% (13/15) appropriate responses (as seen in Table 1.) and 13% (2/15) inappropriate responses.
Conclusion
DermGPT provided suitable draft answers to questions on psoriasis. and these responses show promise in streamlining physicians' workloads.
Table 1. Example questions and DermGPT
Which cream should I use for the psoriasis on my face? I have clobetasol, Vtama, and Dovonex.
Majority Rating: Appropriate
When it comes to choosing a cream for your face, it is important to consult with a dermatologist who can evaluate your specific condition and provide personalized recommendations. However, it is worth noting that clobetasol, VTAMA, and Dovonex have different properties and uses. Clobetasol is a high-potency topical corticosteroid. It is commonly used for the short-term treatment of inflammatory skin conditions, such as eczema and psoriasis. However, due to its potency, it is generally not recommended for long-term use on the face, as it may increase the risk of side effects, such as skin thinning and increased blood vessels. VTAMA (tapinarof) is a cream used for the topical treatment of plaque psoriasis in adults.
It is an aryl hydrocarbon receptor agonist. While VTAMA may be effective for psoriasis, its specific use for facial psoriasis may vary. It is important to consult with a dermatologist to determine if VTAMA is suitable for your facial condition. Dovonex (calcipotriene) is a topical medication that contains a synthetic form of vitamin D . It is commonly used for the treatment of plaque psoriasis. Dovonex may be suitable for use on the face, but the specific dosage and duration of treatment should be determined by a dermatologist. Again, it is crucial to consult with a dermatologist who can assess your condition and provide personalized recommendations for the best cream to use on your face.
How do I know if I have psoriatic arthritis?
Majority Rating: Appropriate
Psoriatic arthritis is a type of arthritis that can develop in individuals with psoriasis, a chronic skin condition. The symptoms of psoriatic arthritis can vary, but common signs include joint pain, stiffness, and swelling, along with skin symptoms such as red, scaly patches. If you suspect you may have psoriatic arthritis, it is important to consult with a healthcare provider who can evaluate your symptoms, perform a physical examination, and order any necessary tests, such as blood tests or imaging studies, to make a diagnosis. They will consider your medical history, family history, and the presence of both skin and joint symptoms to determine if you have psoriatic arthritis. Early diagnosis and treatment are crucial for managing the condition and preventing further joint damage.
Is there a cure for psoriasis?
Majority Rating: Appropriate
Currently, there is no known cure for psoriasis. Psoriasis is a chronic autoimmune skin condition characterized by red, scaly patches on the skin. While there is no cure, there are various treatment options available to manage the symptoms and achieve remission. These treatments include topical medications, phototherapy, systemic medications, and biologic therapies. The goal of treatment is to control inflammation, reduce symptoms, and improve the quality of life for individuals with psoriasis. It is important to work with a healthcare provider to develop a personalized treatment plan based on the severity of the condition and individual needs.
Future Directions
Future research could explore whether answers sourced from peer-reviewed materials surpass those provided by ChatGPT, which primarily utilizes non- peer-reviewed sources.
Strengths/Weaknesses
Strength: Provides peer-reviewed dermatologic-based references for all responses given. Weakness: DermGPT does not have access to real-time information or the ability to browse the internet. Therefore, it may have a limited amount of up-to- date and comprehensive knowledge beyond the documents on which the model is trained.
References
[1] T. Dave, S. A. Athaluri, and S. Singh, "ChatGPT in medicine: an overview of its applications, advantages, limitations, future prospects, and ethical considerations," Front Artif Intell, vol. 6, p. 1169595, 2023, doi: 10.3389/frai.2023.1169595. [2] J. Liu, C. Wang, and S. Liu, "Utility of ChatGPT in Clinical Practice," J Med Internet Res, vol. 25, p. e48568, Jun 28 2023, doi: 10.2196/48568. [3] H. Alkaissi and S. I. McFarlane, "Artificial Hallucinations in ChatGPT: Implications in Scientific Writing," Cureus, vol. 15, no. 2, p. e35179, Feb 2023, doi: 10.7759/cureus.35179.