Ultralytics YOLO in Production·Choose a Runtime·Lesson 2/9

Lessonintermediate

Export to ONNX (and Verify Parity)

Export a portable ONNX model, then prove it matches PyTorch on boxes, confidences, and mAP before anything ships.

An export that shifts confidences by 0.1 does not crash. It quietly moves detections across your conf threshold on every device you ship to, and nobody notices until the counts are wrong. ONNX is the portable starting point for deployment, and five minutes of parity checking is the cheapest insurance in this course.

Outcome

Export to ONNX, prove with numbers that it matches PyTorch on boxes, confidences, and mAP, and explain any gap you find.

Fast Track

If you already know your way around, here's the short version.

yolo export model=yolo26n.pt format=onnx dynamic=True.
Run one image through both files; compare box count, classes, confidences, and coordinates, not just the count.
Run yolo val on both with the same batch; mAP50-95 should be within 0.5 points.
A gap? Check input shape (static vs dynamic), your own preprocessing, and which head (nms) you exported.

Hands-on

Export#

from ultralytics import YOLO

model = YOLO("yolo26n.pt")  # or runs/detect/pills_v1/weights/best.pt
model.export(format="onnx", dynamic=True)  # writes yolo26n.onnx next to the weights

The log shows the opset it picked (opset 18 with onnx 1.23) and that onnxslim simplified the graph. The options that change behavior, from Export mode:

Option	Default	Change it when
`imgsz`	`640`	Production runs at another size; a static export is fixed to it
`dynamic`	`False`	Batch size or image shape varies at inference
`batch`	`1`	You batch at inference (the maximum batch for a dynamic export)
`opset`	latest supported	An older runtime rejects the model; lower it
`quantize`	`None` (FP32)	`16` for FP16; `8` for INT8 with calibration `data`
`nms`	`None`	`True` embeds NMS in the graph; `False` selects YOLO26's NMS-free head

With the default nms=None, the export returns raw predictions and the Ultralytics wrapper runs NMS, exactly like PyTorch.

Check Parity on a Known Image#

Run the same image through both models and compare everything:

from ultralytics import YOLO

pt = YOLO("yolo26n.pt")
onnx = YOLO(pt.export(format="onnx", dynamic=True))

img = "https://ultralytics.com/images/bus.jpg"
a = pt(img, verbose=False)[0].boxes
b = onnx(img, verbose=False)[0].boxes

ia, ib = a.conf.argsort(descending=True), b.conf.argsort(descending=True)
print(f"boxes:         PyTorch {len(a)}  ONNX {len(b)}")
print(f"same classes:  {a.cls[ia].tolist() == b.cls[ib].tolist()}")
print(f"max conf diff: {(a.conf[ia] - b.conf[ib]).abs().max():.4f}")
print(f"max xyxy diff: {(a.xyxy[ia] - b.xyxy[ib]).abs().max():.2f} px")

Sorting both sets by confidence pairs up the same objects, so the last two lines compare each detection with its twin. Now run it twice, once as shown and once with dynamic=True removed:

Export	Boxes	Same classes	Max conf diff	Max box diff
`dynamic=True`	5 vs 5	yes	0.0000	0.00 px
static (default)	5 vs 5	yes	0.0977	10.11 px

The static export is not broken. PyTorch letterboxes the portrait bus.jpg to the smallest stride-aligned shape, 640×480 (height × width), while a static 640×640 graph must pad it to a square with gray bars. Different padding gives slightly different predictions. To compare like with like, either export with dynamic=True, or run the PyTorch side with rect=False so it pads to the full square the way the static export does (see fixed shape vs minimum rectangle); with rect=False the static export also matches to 0.0000.

Validate Both Files#

One image catches gross errors; validation catches subtle ones. Use the same batch for both so preprocessing matches:

yolo val model=yolo26n.pt data=coco8.yaml batch=1
yolo val model=yolo26n.onnx data=coco8.yaml batch=1

Both report mAP50-95 0.654 here. COCO8 proves the pipeline, not the model; on your own validation set, investigate any gap above 0.5 points. The whole check:

Call ONNX Runtime Directly#

Production services often skip the Ultralytics wrapper and call the inference engine themselves. Then preprocessing is your job, and it must match Ultralytics exactly. With nms=False the YOLO26 export returns final detections, so no NMS code is needed:

import cv2
import numpy as np
import onnxruntime as ort

from ultralytics import YOLO
from ultralytics.utils.downloads import safe_download

path = YOLO("yolo26n.pt").export(format="onnx", nms=False)  # NMS-free head: output (1, 300, 6)
session = ort.InferenceSession(path, providers=["CPUExecutionProvider"])  # put CUDAExecutionProvider first on GPU
name = session.get_inputs()[0].name

# Preprocess like Ultralytics: letterbox to 640x640 with gray (114) padding, BGR->RGB, 0-1, NCHW
im = cv2.imread(safe_download("https://ultralytics.com/images/bus.jpg"))
h, w = im.shape[:2]
r = 640 / max(h, w)
nh, nw = round(h * r), round(w * r)
top, left = (640 - nh) // 2, (640 - nw) // 2
canvas = np.full((640, 640, 3), 114, np.uint8)
canvas[top : top + nh, left : left + nw] = cv2.resize(im, (nw, nh), interpolation=cv2.INTER_LINEAR)
x = canvas[..., ::-1].transpose(2, 0, 1)[None].astype(np.float32) / 255

(out,) = session.run(None, {name: x})
det = out[0][out[0][:, 4] > 0.25]  # rows: x1, y1, x2, y2, score, class
det[:, [0, 2]] = ((det[:, [0, 2]] - left) / r).clip(0, w)  # undo padding and scale
det[:, [1, 3]] = ((det[:, [1, 3]] - top) / r).clip(0, h)
for x1, y1, x2, y2, score, cls in det:
    print(f"class {int(cls):2d}  score {score:.2f}  box [{x1:.0f}, {y1:.0f}, {x2:.0f}, {y2:.0f}]")

class  5  score 0.93  box [6, 229, 807, 749]
class  0  score 0.92  box [47, 399, 237, 902]
class  0  score 0.90  box [223, 405, 345, 863]
class  0  score 0.86  box [669, 391, 810, 880]
class  0  score 0.52  box [0, 554, 65, 874]

One bus (class 5) and four people (class 0), matching YOLO(path)(img) to the pixel. The bus scores 0.93 here and 0.88 with the default export because nms=False selects a different head; compare files exported with the same nms value.

Pitfalls#

Comparing a static export with rectangular PyTorch inference

Symptom: same boxes, confidences off by up to 0.1, worst on non-square images. Fix: check parity with dynamic=True, or with rect=False on the PyTorch side, or judge the static export by validation mAP.

Hand-written preprocessing that differs from Ultralytics

Symptom: your own ONNX Runtime code finds fewer or shifted boxes than YOLO("model.onnx"). Fix: letterbox (don't stretch), pad with 114, convert BGR to RGB, divide by 255, and undo the padding and scale on the boxes, exactly as ort_raw.py does.

Mixing heads

Symptom: every confidence is slightly different, even with matching shapes. Fix: compare an nms=False (NMS-free) export only against another nms=False export. With nms=True, also check that the conf and iou baked into the graph match what you use elsewhere.

Try It

Export your model (or yolo26n.pt) to ONNX twice, static and dynamic=True, and run parity.py against each. Then run yolo val on the .pt and the dynamic .onnx with the same batch and write a three-line parity report: max confidence difference for each export, and the mAP50-95 gap.

Done When

You've finished the lesson when all of these are true.

You have a .onnx file next to your .pt and a parity printout for both the static and dynamic export.
You can explain the static-export confidence gap and show the dynamic=True or rect=False fix removing it.
Validation mAP50-95 on the ONNX file is within 0.5 points of the PyTorch model, or the gap is explained.

Show solution

from ultralytics import YOLO

if __name__ == "__main__":  # validation starts worker processes on macOS and Windows
    weights = "yolo26n.pt"  # or runs/detect/pills_v1/weights/best.pt
    onnx_path = YOLO(weights).export(format="onnx", dynamic=True)

    maps = {}
    for w in (weights, onnx_path):
        maps[w] = YOLO(w).val(data="coco8.yaml", batch=1, plots=False).box.map  # or my_dataset/data.yaml
        print(f"{w}: mAP50-95 = {maps[w]:.4f}")
    print(f"gap = {abs(maps[weights] - maps[onnx_path]) * 100:.2f} points")

yolo26n.pt: mAP50-95 = 0.6540
yolo26n.onnx: mAP50-95 = 0.6540
gap = 0.00 points

A parity report for yolo26n on bus.jpg reads: static export max confidence difference 0.0977 (letterbox shape, removed by rect=False), dynamic export 0.0000, mAP50-95 gap 0.00 points. Ship the dynamic export.

What's next

On NVIDIA hardware the next step is TensorRT, where FP16 and INT8 trade a little accuracy for a lot of speed.

Get Started