Creating a task
Set up a task in the dashboard once, then drive it through the API.
Tasks are created in the dashboard because they need decisions a human makes once: what to label, who should do it, and what good looks like. Everything below can be edited later from the task page.
1. Choose what you need done
Label images or data for your own datasets (classification, boxes, segmentation, attributes). The evaluation types (Compare answers, Grade answers, Find failures, Write reference answers) are for judging an AI model and take prompts or outputs as items.
2. Pick expert pools
Pools are curated groups by domain and role, each with its size and hourly price. Pick one or several. If nothing fits, request a new pool; the task is saved as a draft until we activate it.
3. Describe the task
| Section | What to fill in |
|---|---|
| Basics | A title experts will recognise and the domain. |
| Instructions | What to do with each item and what a good answer looks like. Add a short walkthrough video if you can; it cuts questions more than any text. |
| Data and labels | Upload a first batch or link to your data. Define the annotation type and the label classes. Upload 3 to 5 solved examples per class, including the tricky edge cases. Pick the output format. |
| Scope and quality | Whether the data is a one-off batch (with a deadline) or ongoing (daily pushes, with an expected daily volume, a turnaround target and an optional monthly budget cap). The item count is never typed in: it comes from your uploads and API pushes. Then the quality overlap (by default one rater per item and 10% of items get a second one to measure agreement; subjective tasks can send every item to 2 or 3 raters), seconds per item (for the effort estimate), what happens when raters disagree, and a reference set with gold labels. |
| Experts and pay | Required expertise level and the rate experts see before they apply. |
4. Provide a test set
The reference set (30 to 100 items you already labeled correctly) does two jobs: we use it to qualify experts before they touch your data, and we mix a share of it invisibly into the queue to keep quality stable over time. For classification, upload the files plus a CSV with two columns, filename and label. For bounding boxes and segmentation, upload a zip with the images and an annotations.json in COCO format (boxes as bbox, outlines as polygon or RLE segmentation); answers are scored by overlap (IoU) against it. Segmentation results come back as polygons or masks (COCO RLE, column-major counts with size: [height, width]). Under Qualification you choose whether candidates label the whole set or a random sample (default 50 items, different per candidate so answers cannot be shared), the pass mark (default 90%), the share of hidden checks in production (default 5%) and whether an expert is paused automatically when their accuracy drops.
5. Launch
Create and open task publishes it to the pools. Save draft keeps it private. From the task page you can edit everything, see pushed items and their status, and copy the task ID for the API.