Going from raw data to labeled data is not a magic black box. In fact, we’ve described our system in detail on the AI compiler page. Within super.AI’s service, we often need real people to label data by hand to ensure accuracy. And that means they need to understand how to label your data, which makes instructional clarity essential.

A set of well written instructions is the largest factor under your control that will determine the success of the data labeling process.

We’ve put together 10 rules that help create instructions that any human labeler anywhere in the world can understand.

1. Get writing and rework your instructions as you go

Great instructions are often the product of many iterations of tweaks and adjustments. Getting them perfect the first time is hard: for example, there are often edge cases that are hard to anticipate without actually labeling a lot of data first.

Fortunately, your first set of instructions don’t need to be perfect. With super.AI, you can create a draft and quickly improve it based on feedback from the labelers and analysis of where they make mistakes.

The best strategy is to get started quickly and refine. It’s worth being patient when trying to find an instruction set that provides the results you’re after.

Example: If you see images of wolves coming back labeled as dogs, expand your instructions to specify that you only want domestic dogs labeled as dogs.

2. Remember that you are writing for another person

While this seems obvious, bearing it in mind at all times will help focus your writing. It’s like speaking to an interviewer rather than rambling to yourself in the shower.

3. Labelers likely lack your domain knowledge

Do not assume that a labeler knows the difference between a Gibson ES-335 and a ES-355TD (if you happen to be working for Gibson). Be detailed, be specific, and provide everything necessary for someone below your level of expertise to get the job done.

Example: The ES-355TD model features a split-diamond inlay in the headstock and mother-of-pearl fingerboard inlays.

4. Keep it brief

While your instructions need to be detailed and specific, they also need to be understood in under 15 minutes. If a labeler can’t do that, there’s too much information there for them to bear in mind throughout the task.

5. Describe both the concept and the details

Provide an introductory overview, then zoom in on the fine details. This will help the labeler place the information you provide in the right context and make the task more meaningful.

Example:

6. Format content for clarity

Use bullet points, numbered lists, bolding, and other formatting to structure your content in the most digestible fashion.

It's also possible to use HTML formatting in your instructions.

Example: This list that you’re reading right now.

7. Provide label descriptions and clarify ambiguity

Whenever there’s room for ambiguity, it’s important to be specific. Remember, though, that you don’t need to include every nuance in your initial instruction set; you can add clarity as you go based on where labelers get confused. Any definitions you create do not need to be scientific; they just need to be consistent.

Example: The chair label should not be applied to a sofa.
Example: A city is any settlement with over 100,000 inhabitants.

8. Highlight any possible edge cases

This is one area where iteration is essential, as there will be things you cannot predict.

Example: Include any bicycles attached to a car within the car bounding box.
Example: If a speed limit sign shows two values, enter the lowest value.

9. If possible, provide examples of good and bad labels based on the instructions you’ve provided

Illustrations of what good and bad labels look like are perhaps the most powerful tool for improving labeling. In an instant, a labeler can grasp something that might be difficult to describe or understand.

10. Include links to any tools the user might need to complete the task

If your task requires more than just the super.AI labeling UI to complete, you need to provide access to additional tools in your instructions.

Example: Use a URL shortener (https://bitly.com/) for any links.