Auto-Annotation as an Accelerator

How auto-annotation works

Auto-annotation is two passes:

1. Inference pass — run a model over every candidate frame; produce candidate boxes/masks/keypoints.
2. Human review pass — annotators see the predictions overlaid on each image and accept, edit, or reject them.

graph LR
    A[candidate frames] --> B[YOLO predict]
    B --> C[boxes overlay]
    C --> D{human review}
    D -- accept --> E[labeled dataset]
    D -- edit --> E
    D -- reject --> F[hand-label]
    F --> E

If the inference model is decent, the human is mostly clicking "accept" — which is 5–10× faster than drawing.

Pick the right inference model

Use the best model you have access to for your task. In order of preference:

1. Your latest fine-tune. If you already have a v1 model on similar data, this is the best auto-annotator for v2.
2. A YOLO trained on a similar dataset. Pose datasets, agricultural datasets, security datasets — pick one with overlapping classes. Open-vocabulary detectors like YOLO World can also bootstrap brand-new classes from text prompts.
3. A pretrained Ultralytics YOLO model on COCO/etc. Fine if your classes overlap with the 80 COCO classes. Useless otherwise.

The confidence threshold

Auto-annotation's confidence threshold is a tradeoff between overdraw (boxes the model wasn't sure about, that humans have to delete) and misses (real objects without a box, that humans have to draw).

0.5 is usually the sweet spot. You can tune it for your model and dataset by sampling 20 frames and counting average click cost per frame.

Active learning: route the right frames to humans

Once you have an auto-annotator and reviewers, the next optimization: route the uncertain frames to humans first. The model is most likely to be wrong where it's least confident.

A useful sampling order for review:

1. Frames where the model has detections at confidence 0.4–0.6 (borderline).
2. Frames with very high count of detections (likely to have NMS edge cases).
3. Frames where two classes had similar confidence on the same object.
4. Random sample of the rest, for coverage.

Platform's review queue can be sorted by these criteria.

Class-by-class fallback

Auto-annotation is great for classes the model knows. For new classes you're introducing, the model has nothing to offer:

• Bootstrap by labeling 100–200 examples by hand.
• Train a v0 model on those (it'll be bad — that's fine).
• Use v0 to auto-annotate the next batch; the bad-but-existing predictions still beat starting from scratch.
• Iterate. By v1 the auto-annotator is useful.

Reviewer metrics

Beyond throughput:

Edit rate trending up over time is the early warning that your auto-annotator's domain has drifted. If you'd rather offload the labeling UI entirely, the Roboflow integration and other tools listed in the integrations index plug straight back into Platform datasets, while the broader data collection and annotation guide covers the underlying class-imbalance tradeoffs.