Beyond Detection: Segment, Pose, OBB, and Classify·Segment·Lesson 3/10

Lessonintermediate

Use Masks in Code: Area, Overlap, Outlines, Cutouts

Read masks.data and masks.xy correctly, measure object area and fill, test overlap with masks instead of boxes, and cut objects out of the frame.

A segmentation model pays for itself only when code turns its masks into a decision. On the bus photo, each pedestrian's box overlaps the bus's box by 56–74%, while their masks overlap the bus by 0% — a "person inside the vehicle zone" alert built on boxes would fire on all four people standing on the sidewalk. Masks get these answers right, but their default views have size and format traps that produce plausible, wrong numbers.

Outcome

Run scripts that report each object's mask area, fill ratio, outline size, and overlap with another object, and save a transparent cutout per object — using the right mask view for each.

Fast Track

If you already know your way around, here's the short version.

Predict with retina_masks=True whenever you measure or crop, so masks match the original image size.
Use masks.data (N × H × W) for areas, overlaps, and cutouts; use masks.xy (pixel polygons) for drawing and exporting outlines.
Area is mask.sum(); fill is mask area ÷ box area; overlap is (mask_a & mask_b).sum().
Check result.masks is None before reading masks: it is None when nothing is found.

Hands-on

Two views of the same mask#

A person isolated with an Ultralytics YOLO segmentation mask, left in place in the full frame on a black background

result.masks holds every instance mask in two representations. Pick by what the next line of code does; the Results reference lists every attribute.

Attribute	Shape	Coordinates	Use it for
`masks.data`	N × H × W tensor of 0/1	inference size by default; original size with `retina_masks=True`	pixel areas, overlaps between masks, cutouts
`masks.xy`	list of N arrays, each P × 2	original image pixels	drawing outlines, exporting polygons
`masks.xyn`	list of N arrays, each P × 2	normalized 0–1	writing YOLO polygon labels back out

masks.data defaults to the inference size

On the 1080 × 810 bus image, masks.data comes back as (5, 640, 480) — the letterboxed inference size — so pixel counts are about 2.85× too small and the mask cannot be laid over the original image. retina_masks=True upsamples masks to the original resolution at a small postprocessing cost. Use it whenever you measure or crop.

Measure and cut out every object#

This script reads both views for each object: pixel area from masks.data, outline size from masks.xy, and a transparent PNG cutout that uses the mask as an alpha channel. The isolating segmentation objects guide covers more cutout variants.

import cv2
import numpy as np

from ultralytics import YOLO

model = YOLO("yolo26n-seg.pt")
result = model("https://ultralytics.com/images/bus.jpg", retina_masks=True)[0]  # masks at original resolution

print("masks.data:", tuple(result.masks.data.shape), "orig_shape:", result.orig_shape)
for i, box in enumerate(result.boxes):
    name = result.names[int(box.cls)]
    mask = result.masks.data[i].cpu().numpy().astype(np.uint8)  # H x W, 1 = object
    polygon = result.masks.xy[i]  # P x 2 outline in original pixels
    x1, y1, x2, y2 = box.xyxy[0].int().tolist()
    mask_area = int(mask.sum())
    box_area = (x2 - x1) * (y2 - y1)
    print(f"{name:6s} conf={float(box.conf):.2f} mask={mask_area:7d}px box={box_area:7d}px fill={mask_area / box_area:.2f} polygon={len(polygon)} pts")

    # Cut the object out on a transparent background
    cutout = np.dstack([result.orig_img, mask * 255])[y1:y2, x1:x2]
    cv2.imwrite(f"{i}_{name}.png", cutout)

masks.data: (5, 1080, 810) orig_shape: (1080, 810)
person conf=0.88 mask=  49331px box=  97219px fill=0.51 polygon=575 pts
person conf=0.86 mask=  21316px box=  67760px fill=0.31 polygon=408 pts
bus    conf=0.86 mask= 255265px box= 409822px fill=0.62 polygon=1172 pts
person conf=0.81 mask=  33524px box=  55842px fill=0.60 polygon=350 pts
person conf=0.64 mask=  11109px box=  23828px fill=0.47 polygon=211 pts

What to read in it:

masks.data: (5, 1080, 810) matches orig_shape, so retina_masks=True did its job and every pixel count is in original pixels.
Index i links the two. result.masks.data[i] belongs to result.boxes[i], so names and confidences come from the box and pixels from the mask.
Fill tells a story a detector cannot: the bus fills 62% of its box, while the second person, cut off by the right edge of the frame and standing against the bus, fills only 31%.
The cutouts are BGRA PNGs containing the person or the bus and nothing else — ready for a catalog, a second model, or a review queue.

The isolation guide's cropped variant uses black instead of a transparent background:

The same person cropped to its bounding box, with every pixel outside the mask set to black

Overlap: where boxes lie#

Rules such as "a person is inside the restricted zone" or "product covers the conveyor" are overlap tests. Run the same test with boxes and with masks:

import numpy as np

from ultralytics import YOLO

result = YOLO("yolo26n-seg.pt")("https://ultralytics.com/images/bus.jpg", retina_masks=True, verbose=False)[0]
names = [result.names[int(c)] for c in result.boxes.cls]
masks = result.masks.data.cpu().numpy().astype(bool)  # N x H x W at original resolution
boxes = np.zeros_like(masks)
for i, (x1, y1, x2, y2) in enumerate(result.boxes.xyxy.int().tolist()):
    boxes[i, y1:y2, x1:x2] = True  # the same objects as filled rectangles

bus = names.index("bus")
for i, name in enumerate(names):
    if name == "person":
        by_box = (boxes[i] & boxes[bus]).sum() / boxes[i].sum()
        by_mask = (masks[i] & masks[bus]).sum() / masks[i].sum()
        print(f"person {i}: {by_box:4.0%} of box overlaps the bus box, {by_mask:4.0%} of mask overlaps the bus mask")

person 0:  67% of box overlaps the bus box,   0% of mask overlaps the bus mask
person 1:  69% of box overlaps the bus box,   0% of mask overlaps the bus mask
person 3:  74% of box overlaps the bus box,   0% of mask overlaps the bus mask
person 4:  56% of box overlaps the bus box,   0% of mask overlaps the bus mask

The people stand in front of the bus, so the bus mask stops at each person's outline and the masks share no pixels. The boxes are rectangles drawn around both, so they overlap heavily. The same calculation works for any region you draw yourself: fill a polygon into a zero array with cv2.fillPoly and use it in place of the bus mask.

Decision rule

Any rule phrased as inside, touching, covering, or blocking is an overlap test. Run it on masks. Boxes are fine for near and left of, where a few pixels of slack do not change the answer.

From pixels to real-world answers#

Question	Computation	Watch out for
How big is it?	`mask.sum()` × (cm² per pixel)	Calibrate cm² per pixel once with a known object. On an angled camera, area changes with distance: calibrate per zone or measure only in one region.
How much of the region is covered?	class-mask pixels ÷ region pixels	Decide whether overlapping instances count once or twice.
Is it inside the zone?	`(mask & zone).sum() / mask.sum()`	Draw the zone at the original image size.
What does its outline look like?	`masks.xy[i]`	See the split-object warning below before computing area from it.

Pitfalls#

Reading masks when nothing was found

Symptom: AttributeError: 'NoneType' object has no attribute 'xy' on frames with no objects. Cause: result.masks is None when the model finds nothing, not an empty object. Fix: loop over result.boxes (empty, never None, for segment models) and index masks inside the loop, as masks.py does, or check result.masks is None first.

Polygon area on split objects

Symptom: areas that are wildly wrong for occluded objects. Cause: when an occluder splits an object, masks.xy joins every contour into one array, and cv2.contourArea on that array measures a shape that does not exist. Fix: compute areas from masks.data pixel counts; keep polygons for drawing.

Mixing resolutions

Symptom: areas from two runs differ by a factor of about 2.85. Cause: one run used retina_masks=False, so its areas are in inference pixels. Fix: measure with retina_masks=True everywhere, and on CUDA call .cpu() before .numpy().

Try It

Run masks.py on an image of your own with at least three objects and report the object with the lowest fill and why its box overstates its size. Rerun without retina_masks=True and note how masks.data.shape and the areas change. Then adapt overlap.py to test whether any object in your image overlaps a rectangle you draw over the bottom third of the frame.

Done When

You've finished the lesson when all of these are true.

You have one transparent PNG cutout per detected object in your image.
You can state the lowest fill ratio and explain the difference between masks.data with and without retina_masks=True.
Your zone test prints a box-based and a mask-based overlap for every object, and you can explain any object where they disagree.

Show solution

Without retina_masks=True, the bus image returns masks.data with shape (5, 640, 480): every pixel area shrinks by (1080 × 810) / (640 × 480) ≈ 2.85, and the mask no longer lines up with orig_img for cutouts. When the image's aspect ratio differs from the inference shape, letterbox padding adds a second misalignment.

The lowest fill on the bus image is the second person (0.31): cut off by the right edge of the frame and standing against the bus, so most of the box is background and bus.

For the zone, build it once at the original size and reuse the mask test:

import cv2

h, w = result.orig_shape
zone = np.zeros((h, w), np.uint8)
cv2.fillPoly(zone, [np.array([[0, 2 * h // 3], [w, 2 * h // 3], [w, h], [0, h]])], 1)
zone = zone.astype(bool)
for i, name in enumerate(names):
    print(f"{name} {i}: box {(boxes[i] & zone).sum() / boxes[i].sum():.0%}, mask {(masks[i] & zone).sum() / masks[i].sum():.0%}")

On the bus image:

person 0: box 36%, mask 29%
person 1: box 33%, mask 37%
bus 2: box 4%, mask 0%
person 3: box 31%, mask 21%
person 4: box 47%, mask 53%

The people's legs really are in the bottom third, so box and mask agree within about ten points. The bus is the false alarm: its box dips 4% into the zone, its mask not at all. With a "more than 0% inside" rule, boxes would flag the bus and masks would not.

What's next

Next, we drop instances altogether and label every pixel with semantic segmentation.

Get Started