Lesson 3 of 11
Labeling data: write the rules first
In plain words: A model learns only what its labels teach it, and labels are only consistent when everyone follows the same written rules.
- Look for labels you already have: the team that solved a ticket, an approved refund, a failed inspection. They’re free.
- Keep the label set small and clear: labels that don’t overlap, plus “other” so nobody is forced to guess.
- Write guidelines: a definition and a real example for each label, and the agreed answer for every tricky case you find.
- Run a pilot: two people label the same 100 to 200 examples, then fix the guidelines wherever they disagreed.
- Choose who labels: experts for judgment calls such as medical or legal cases, trained labelers for volume, a model for drafts that people check.
- Use a labeling tool such as Label Studio or CVAT rather than a spreadsheet: it records who labeled what, and with which version of the rules.
Tip Each time two people disagree, add the case and its answer to the guidelines. That list of tricky cases is worth more than any definition.
See it in a real project
- How to label data for machine learning: a label set, guidelines and a pilot round for support tickets
- How to route support tickets to the right team: labels that already exist: the team that solved each ticket
- How to sort customer reviews by sentiment: star ratings as free labels
- How to detect fraudulent transactions: fraud labels from chargebacks and confirmed cases