The Seven Vision Tasks

The seven tasks at a glance#

Read the table by the output column — that's what the output line of your spec must match.

Two pairs trip people up. Instance vs semantic segmentation: instance separates every object (three cars, three masks); semantic labels pixels by class only (one "car" region, no count). Detection vs OBB: both return one box per object, but an OBB rotates with the object, which matters when objects lie at angles in top-down imagery.

One API, seven outputs#

Switching tasks doesn't mean switching tools. The weights file decides what comes back, and the Results object fills the matching attribute:

from ultralytics import YOLO

image = "https://ultralytics.com/images/bus.jpg"
outputs = ["probs", "boxes", "masks", "semantic_mask", "depth", "keypoints", "obb"]

for suffix in ["-cls", "", "-seg", "-sem", "-depth", "-pose", "-obb"]:
    weights = f"yolo26n{suffix}.pt"
    model = YOLO(weights)
    result = model(image, verbose=False)[0]
    filled = [name for name in outputs if getattr(result, name) is not None]
    print(f"{weights:18} task={model.task:9} -> {', '.join(filled)}")

yolo26n-cls.pt     task=classify  -> probs
yolo26n.pt         task=detect    -> boxes
yolo26n-seg.pt     task=segment   -> boxes, masks
yolo26n-sem.pt     task=semantic  -> semantic_mask
yolo26n-depth.pt   task=depth     -> depth
yolo26n-pose.pt    task=pose      -> boxes, keypoints
yolo26n-obb.pt     task=obb       -> obb

Read the right-hand column: segmentation and pose also return boxes, because they are detection plus one more thing. Semantic segmentation and depth return no per-object output at all — they describe every pixel instead. Your downstream code reads exactly one of these attributes, so the output line of your spec should name it.

Choosing in practice#

Start from what one prediction must describe, then take the cheapest task that answers the decision line:

1. One label for the whole image → classification.
2. A class for every pixel, with no need to separate objects → semantic segmentation.
3. A distance for every pixel → depth estimation.
4. Something about each object:
   • where it is, to count, locate, or filter it → detection (add tracking for identity across frames);
   • its exact shape, boundary, or area → instance segmentation;
   • its posture → pose;
   • its rotation in top-down imagery → OBB.

When two tasks both fit, choose the cheaper one to label and run. Detection is the default for most products: boxes are fast to draw and review, and with tracking on top they already power counting, zone alerts, and traffic analytics like this highway view:

Depth: the task that answers "how far"#

Depth estimation returns a map the size of the image with one distance, in meters, per pixel — no classes and no boxes. Use it when distance drives the decision: a robot keeping clear of obstacles, a vehicle slowing when something is close, a rough volume estimate of a pile.

from ultralytics import YOLO

model = YOLO("yolo26n-depth.pt")
result = model("https://ultralytics.com/images/bus.jpg", verbose=False)[0]

depth = result.depth.data  # meters, one value per pixel
print(f"depth map {tuple(depth.shape)}, nearest {float(depth.min()):.1f} m, farthest {float(depth.max()):.1f} m")

depth map (1080, 810), nearest 2.0 m, farthest 14.9 m

A single camera infers depth from learned cues such as perspective and object size, so measure its error on your own scenes before using it for safety-critical distances. Depth pairs well with detection: detect the object, then read the depth map inside its box.

Pitfalls#