Define the Dataset Specification

Where the spec sits#

The spec is the bridge between the objective and the camera. It drives collection, and it is the page you return to when the first model misses its target:

The loop goes back to the spec, not to the training settings: a missed target usually means a scenario the spec didn't cover. The steps of a CV project guide describes the same lifecycle.

Write scenarios as rows#

A scenario is one combination of capture conditions the model must handle. Rows with quotas make gaps visible before anyone picks up a camera:

A scenario you can't write down is one the model won't see in training, and it will be the one that fails in production.

Size the spec against the volume targets#

The Ultralytics tips for best training results give production rules of thumb:

These are the bar for a production model. A few hundred labeled objects per class is enough to start experimenting, and that first model is the fastest way to learn which scenarios need more data. To see how far a dataset is from the bar, count it. This script runs on COCO128, a 128-image slice of COCO that downloads automatically:

from collections import Counter
from pathlib import Path

from ultralytics.data.utils import IMG_FORMATS, check_det_dataset, img2label_paths

IMAGES_PER_CLASS, INSTANCES_PER_CLASS = 1500, 10_000  # Ultralytics production targets

data = check_det_dataset("coco128.yaml")  # downloads COCO128 (~7 MB) on first run; swap in your data YAML
images = [p for p in Path(data["train"]).rglob("*.*") if p.suffix[1:].lower() in IMG_FORMATS]
image_count, instance_count, background = Counter(), Counter(), 0
for label in map(Path, img2label_paths([str(p) for p in images])):
    classes = [int(r.split()[0]) for r in label.read_text().splitlines() if r.strip()] if label.exists() else []
    instance_count.update(classes)
    image_count.update(set(classes))
    background += not classes

print(f"{len(images)} images, {background} background ({background / len(images):.0%}, target 0-10%)")
print(f"{'class':>12} {'images':>7} {'instances':>10} {'of target':>10}")
ranked = instance_count.most_common()
for cls, n in ranked[:3] + ranked[-3:]:  # best and worst covered classes
    coverage = min(image_count[cls] / IMAGES_PER_CLASS, n / INSTANCES_PER_CLASS)
    print(f"{data['names'][cls]:>12} {image_count[cls]:>7} {n:>10} {coverage:>10.1%}")

never_labeled = [name for i, name in data["names"].items() if i not in instance_count]
print(f"{len(never_labeled)} classes in names with zero labels: {never_labeled[:4]} ...")

128 images, 2 background (2%, target 0-10%)
       class  images  instances  of target
      person      61        254       2.5%
         car      12         46       0.5%
         cup      10         36       0.4%
        bear       1          1       0.0%
      banana       1          1       0.0%
        skis       1          1       0.0%
9 classes in names with zero labels: ['fire hydrant', 'parking meter', 'sheep', 'cow'] ...

How to read it:

• Coverage is the smaller of the image ratio and the instance ratio, because a class needs both. Even person, the best-covered class, sits at 2.5% of the production bar.
• One-instance classes (bear, banana, skis) cannot be learned or measured. In your dataset, either collect them or drop them from v1.
• Classes with zero labels are listed in the YAML but never drawn. The model still spends capacity predicting them. Remove them from names, or collect them before training.

Cover the variance dimensions#

One frame already mixes near and far people, bicycles, a dog, and cars. A real spec has to plan for that range on purpose. For each dimension, write down whether the spec covers it or why it doesn't apply; empty rows are scenarios you'll discover in production instead:

Variance matters more than volume: 1,500 images spread across these rows are worth more than 5,000 from one camera at noon.

Plan background images and edge cases#

Background images are frames with no labeled objects, such as an empty dock. They teach the model that nothing is there and reduce false positives.

Edge cases are heavy occlusion, objects cut off at the frame edge, unusual orientations, stacked objects, and look-alikes such as a forklift mast next to a steel column. Give each one its own quota row.