Just published: effects of disease prevalence and feedback on dermatological decision making

Kira Prentice, Head of Product - DiagnosUs
February 2, 2022

Imagine that you are a dermatologist. You spend the morning seeing patients who have been referred to you for suspicion of skin cancer. Many of them do, in fact, have skin lesions that require treatment. We would say that disease "prevalence" was high in this set of patients.

Suppose that you next spend the afternoon giving annual screening exams to members of the general population. Here disease prevalence will be low. Would your morning’s work influence your decisions about patients in the afternoon?

In collaboration with Centaur Labs, Jeremy M Wolfe, PhD, Professor of Ophthalmology & Radiology at Harvard Medical School conducted a study published this month in Cognitive Research: Principles and Implications (CRPI) that tackled this question. We know from other contexts that recent history can influence current decisions and we know that target prevalence has an impact on decisions.

In this study, Centaur Labs collected decisions about skin lesions from individuals with varying degrees of expertise using our proprietary medical imaging labeling app, DiagnosUs. Over 300,000 trials were collected in DiagnosUs over five days, with 803 participants in the study. This allowed the researcher to examine the effects of feedback history and prevalence in a single study.

This research finds that feedback educates observers, causing them to become more liberal when targets (diagnoses of melanoma) have been relatively common and more conservative when those targets are rare. The effects of a block of trials with feedback can last for days with those effects showing up when the observer takes up a similar task again. It may be possible to use the educational effects of feedback when it is desirable to shift an observer's criterion, especially if the subsequent task does not involve reliable feedback.

Read the full study at Cognitive Research: Principles and Implications (CRPI).

Related posts

February 18, 2021

AIMed interviews Centaur Labs CEO Erik Duhaime

Founder and CEO of Centaur Labs talks to AI Med magazine about the power of collective intelligence.

Continue reading →
August 31, 2022

9 most common types of medical text datasets

From SMS to insurance claims to pathology reports and scientific studies, in this post we dig into the most common type of medical text datasets leveraged for NLP in healthcare.

Continue reading →