After a decent amount of data is annotated, we know we can train a good, accurate model. That model makes predictions that should be as good as, or better than humans, meaning we can use the model to verify human annotations.
After doing this suggested QA task, you have successfully corrected up to 10% of your documents by automatically detecting misannotations.
The end result?
Accurate training data with only 15% of the normal validation effort!