Our confidence in shared labels can be increased by inspecting the data