BRAVE: Block-wise Structural Regularization via Controlled Evidence Feedback for Reliable Label Aggregation under Sparse Crowdsourcing
Reliable label aggregation under sparse crowdsourcing by separating block-local and global posterior structure and controlling recursive evidence feedback.
What changes when evidence feeds back?
A regulated global signal informs the next block without becoming a substitute for new labels.
The reliability failure
Sparse crowdsourcing creates a subtle failure mode: once an aggregated label posterior is fed back as evidence, an early mistake can reinforce itself. The model may appear increasingly confident even though no genuinely new annotation evidence has arrived. We call this illusory evidence accumulation.
What I designed
BRAVE separates the posterior used inside each evidence block from the global posterior carried across blocks. A controlled feedback coefficient regulates how much global belief can influence the next local update. This preserves useful information transfer without letting recursive pseudo-evidence dominate the observed labels.
My contribution
As first author, I led the algorithm design, literature review, experimental design, code implementation, experimental deployment, and analysis. I built the study around both predictive performance and uncertainty quality, because reliable aggregation requires more than accuracy alone.
Evidence
Across 14 crowdsourcing benchmarks, BRAVE achieved the lowest negative log-likelihood on 5 datasets and the best or tied-best expected calibration error on 9. Its accuracy was within 0.03 of the strongest external baseline on 11 datasets. We also completed a downstream reward-model calibration transfer experiment.
Status: under review at TMLR. This page describes first-author work but does not imply acceptance. The linked public page omits Author Console parameters and this site does not host the submitted manuscript.