AI outclasses physician recommendations in virtual urgent care visits

Recommendations made by an artificial intelligence (AI)-assisted virtual urgent care clinic, Cedars-Sinai Connect, more frequently achieve better-quality ratings than those made by physicians, a recent study has shown.

A total of 461 physician-managed visits with AI recommendations of sufficient confidence and complete medical records for adults with respiratory, urinary, vaginal, eye, or dental symptoms from 12 June to 14 July 2024 were included in this retrospective cohort study.

The researchers assessed the concordance of diagnosis and management recommendations of initial AI recommendations and final physician recommendations. Adjudicators scored all nonconcordant and a sample of concordant recommendations as optimal, reasonable, inadequate, or potentially harmful.

More than half of the visits (n=262, 56.8 percent) showed concordance between the initial AI and final physician recommendations. However, among the 461 weighted visits, more AI recommendations were rated as optimal (77.1 percent, 95 percent confidence interval [CI], 72.7–80.9) compared with treating physician decisions (67.1 percent, 95 percent CI, 62.9–71.1).

In addition, quality scores were equal in 67.9 percent (95 percent CI, 64.8–70.9), better for AI in 20.8 percent (95 percent CI, 17.8–24.0), and better for treating physicians in 11.3 percent (95 percent CI, 9.0–14.2) of cases.

“[These] findings suggest that AI performed better in identifying critical red flags and supporting guideline-adherent care, whereas physicians were better at adapting recommendations to changing information during consultations,” the researchers said. “Thus, AI may have a role in assisting physician decision making in virtual urgent care.”