Aggregating soft labels from crowd annotations improves uncertainty estimation under distribution shift

Dustin Wright; Isabelle Augenstein

doi:10.1371/journal.pone.0323064

Loading metrics

Open Access

Peer-reviewed

Research Article

Aggregating soft labels from crowd annotations improves uncertainty estimation under distribution shift

Dustin Wright ,

Roles Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Software, Validation, Visualization, Writing – original draft, Writing – review & editing

* E-mail: dw@di.ku.dk

Affiliation University of Copenhagen, Department of Computer Science, Copenhagen, Denmark

https://orcid.org/0000-0001-6514-8733

⨯
Isabelle Augenstein

Roles Conceptualization, Funding acquisition, Investigation, Methodology, Project administration, Resources, Supervision, Writing – original draft, Writing – review & editing

Affiliation University of Copenhagen, Department of Computer Science, Copenhagen, Denmark
⨯

Aggregating soft labels from crowd annotations improves uncertainty estimation under distribution shift

Dustin Wright,
Isabelle Augenstein

Published: June 9, 2025
https://doi.org/10.1371/journal.pone.0323064

About the Authors

Dustin Wright

Roles Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Software, Validation, Visualization, Writing – original draft, Writing – review & editing

* E-mail: dw@di.ku.dk

Affiliation University of Copenhagen, Department of Computer Science, Copenhagen, Denmark

https://orcid.org/0000-0001-6514-8733

Isabelle Augenstein

Roles Conceptualization, Funding acquisition, Investigation, Methodology, Project administration, Resources, Supervision, Writing – original draft, Writing – review & editing

Affiliation University of Copenhagen, Department of Computer Science, Copenhagen, Denmark

Competing Interests

The authors have declared that no competing interests exist.

Subject Areas
?

For more information about PLOS Subject Areas, click here.
We want your feedback. Do these Subject Areas make sense for this article? Click the target next to the incorrect Subject Area and let us know. Thanks for your help!

Toxicity
Is the Subject Area "Toxicity" applicable to this article?

Thanks for your feedback.
Imaging techniques
Is the Subject Area "Imaging techniques" applicable to this article?

Thanks for your feedback.
Probability distribution
Is the Subject Area "Probability distribution" applicable to this article?

Thanks for your feedback.
Statistical models
Is the Subject Area "Statistical models" applicable to this article?

Thanks for your feedback.
Statistical data
Is the Subject Area "Statistical data" applicable to this article?

Thanks for your feedback.
Statistical distributions
Is the Subject Area "Statistical distributions" applicable to this article?

Thanks for your feedback.
Bayesian method
Is the Subject Area "Bayesian method" applicable to this article?

Thanks for your feedback.
Machine learning
Is the Subject Area "Machine learning" applicable to this article?

Thanks for your feedback.