Object Tracking with ByteTrack and BoT-SORT

Link to this sectionWhat a tracker actually does#

A tracker maintains a set of "tracks" — hypotheses about real-world objects. Each frame's detections are matched to tracks based on:

1. Motion prediction — where the track should be this frame (Kalman filter).
2. Spatial overlap — IoU between predicted track location and detection.
3. Appearance (BoT-SORT only) — visual features for re-identification, matching the detection back to its track even after occlusion.

Tracks that don't get matched for a few frames are deleted. New unmatched detections become new tracks.

Link to this sectionByteTrack: fast and robust#

from ultralytics import YOLO

model = YOLO("yolo26n.pt")
results = model.track(
    "input.mp4",
    persist=True,
    tracker="bytetrack.yaml",
    conf=0.25,
)

ByteTrack's trick: it uses low-confidence detections for matching, then drops them from the output. This recovers tracks during partial occlusion when confidence dips temporarily.

Link to this sectionBoT-SORT: appearance-aware#

results = model.track(
    "input.mp4",
    persist=True,
    tracker="botsort.yaml",
)

BoT-SORT extracts an appearance embedding per detection and uses it for re-identification when objects briefly disappear (behind a tree, pole, another object). It's slower — sometimes 30%+ — but if your scene has frequent occlusions, the difference in ID consistency is dramatic.

Link to this sectionUsing IDs in a pipeline#

from collections import defaultdict
from ultralytics import YOLO

model = YOLO("yolo26n.pt")

counts = defaultdict(set)

for r in model.track("input.mp4", stream=True, persist=True, tracker="bytetrack.yaml"):
    for box in r.boxes:
        if box.id is None:
            continue
        cls = r.names[int(box.cls)]
        counts[cls].add(int(box.id))

for cls, ids in counts.items():
    print(f"{cls}: {len(ids)} unique objects")

That's the whole pipeline. Tracking gives you stable IDs; counting unique IDs gives you accurate counts even on long videos. Built-in solutions like region counting, heatmaps, and distance calculation are all thin layers on top of this loop.

Link to this sectionTuning the tracker#

The tracker config files (bytetrack.yaml, botsort.yaml) live in the Ultralytics package. Common knobs:

• track_buffer — how many frames a track survives without a match (default ~30 frames). Larger = more robust to occlusion, more zombie tracks.
• match_thresh — IoU threshold for matching detection to track. Lower if your motion is fast and the box jumps a lot.
• new_track_thresh — confidence floor for starting a new track. Raise it if you're getting too many short-lived false tracks.

Link to this sectionStream sources#

Same API for live cameras — the Predict mode docs cover every supported source type:

for r in model.track("rtsp://camera.local/stream", stream=True, persist=True):
    for box in r.boxes:
        # do something with box.id, box.cls, box.xyxy
        pass

Just remember: stream=True is mandatory for live or long sources, otherwise memory grows unbounded.