Maithra Raghu

Chevron down

Direct Uncertainty Prediction with Applications to Healthcare

Large labeled datasets for supervised learning are frequently constructed by assigning each instance to multiple human evaluators, and this leads to disagreement in the labels associated with a single instance. Here we consider the question of predicting the level of disagreement for a given instance, and we find an interesting phenomenon: direct prediction of uncertainty performs better than the two-step process of training a classifier and then using the classifier outputs to derive an uncertainty. We show stronger performance for predicting disagreement via this direct method both in a synthetic setting whose parameters we can fully control, and in a paradigmatic healthcare application involving multiple labels assigned by medical domain experts. We further show implications for allocating additional labeling effort toward instances with the greatest levels of predicted disagreement.

Maithra Raghu is a PhD Candidate at Cornell University, and Research Scientist at Google Brain. She is interested in the Science of Deep Learning -- principled, reproducible and interpretable insights on deep neural network representations and the applications of these insights to healthcare.

Buttontwitter Buttonlinkedin

As Featured In

Original
Original
Original
Original
Original
Original

Partners & Attendees

Intel.001
Nvidia.001
Acc1.001
Ibm watson health 3.001
Rbc research.001
Mit tech review.001
Kd nuggets.001
Facebook.001
Graphcoreai.001
Maluuba 2017.001
Twentybn.001
Forbes.001
This website uses cookies to ensure you get the best experience. Learn more