Writing great instructions
Rubrics, gold items, and calibration that hold quality.
Quality holds when the rubric is clear before the first hour is captured.
Write the operational rubric first
Describe the environment, the task, what “good” looks like, and what fails. Harbor reviewers score against that rubric — not self-assessment.
Include gold items
Gold-standard examples train both contributors and reviewers. Programmes may include calibration tasks, agreement checks, and spot audits.
Show fail cases
Rejected uploads usually miss the brief, break length or consent rules, or look staged. Say that in the invitation.
Keep AI off the task unless you allow it
Unless a programme brief says otherwise, contributors and experts must not use external AI to complete eval or capture work. Unassisted human judgment is the point.
Calibrate before production
People who fail calibration may get extra training, a smaller scope, or removal from that cohort. Repeated failure in the same domain lowers match priority.
See Quality and review.
Need a programme scoped? Talk to Harbor.