Open AI’s ChatGPT was launched for the public in November 2022 and since then it has been a hot topic amongst healthcare professionals WORLDWIDE whether ChatGPT is ready for healthcare? These discussions inspired a team of doctors led by Dr. John Myers, vice chief of Innovation in the UC-San Diego School of Medicine Division of Infectious Disease and Global Public Health to see if chatbot could accurately and empathetically respond to patient queries related to their real health issues. In this recent study which is published in the April 2023 issue of JAMA Internal Medicine, scientists have explored how well ChatGPT can answer health-related questions from real patients and their result shows that ChatGPT outperformed human doctors, especially on ground of empathy when assessed by a panel of medical experts.
To carry out the study, the team randomly selected 195 question and answer exchanges posted last October on Reditt’s AskDocs which has a community of 481,000 members where experienced doctors respond to publicly asked medical questions. Then the same queries were posed to ChatGPT version 3.5 in December 2022. The answers were collected from the chatbot and presented to a panel of healthcare professionals to evaluate both the doctors’ and AI’s answers to the questions. To ensure that raters did not know which answers were from AI and which ones were from doctors, responses were randomly ordered and labelled as Response1 and Response 2 to blind evaluators to the identity to the author. On a scale of 5, the answers were rated for empathy and quality.
On tabulation of the ratings, the results were highly in favour of ChatGPT over doctors. The evaluators preferred the AI response 78.6% of the time, giving ChatGPT answers an average rating of 4.1/5 for quality, corresponding to ‘Good’. Doctors’ answers were given an average rating of 3.26 corresponding to ‘Acceptable’. For the empathy component, evaluators rated ChatGPT answers as ‘empathetic’ or ‘very empathetic’ 9.8 times more often than they did doctors’ responses. Bedside manners were found to be better with ChatGPT. It is to be noted here that ChatGPT gave lengthy responses with a use of average 211 words whereas doctors’ responses had a usage of average 52 words. But this can be attributed to the fact that doctors’ responses were drawn from a casual, anonymous online forum and not a professional setting so their answers to questions may not have been very thoughtfully worded as they could have been had they seen the patients’ in person. This above-mentioned point can be considered as a significant limitation of this study.
This clearly shows that ChatGPT cannot claim a conclusive victory over trained and experienced doctors. However, it clearly suggests that ChatGPT could satisfactorily assist medical professionals in responding to patient queries by drafting a message based on the query for doctors or support staff to edit. Conclusively one can say that while AI won’t replace your doctor but doctored working together with technologies like ChatGPT may revolutionise medicine and definitely AI-augmented care is the future of medicine.
Source: www.bigthink.com
