Choose a Deployment Target

Start From the Device#

The runtime follows from where inference runs, not from habit. Walk the tree from the top and stop at the first match:

Each runtime maps to one format= value in Export mode:

On recent Apple platforms you can also export Apple's newer Core AI format (format=coreai, exported on macOS 26+ Apple silicon or x86-64 Linux). New to exporting? The Export for Deployment lesson covers the basics.

If you would rather not run inference servers at all, Ultralytics Platform deploys a model as a dedicated endpoint in one of 42 regions, and your profile's Deployments tab maps where each one runs:

That path is covered in Deploy and Call an Endpoint. The rest of this course is for when you run the model yourself.

Turn the FPS Target Into a Millisecond Budget#

"Real time" is not a requirement; a number is. Work it out before you benchmark anything:

Then apply these decision rules:

• If YOLO26n needs more than your model budget on the device's CPU, move to an accelerator (GPU, iGPU, NPU) or shrink imgsz before anything else; the YOLO26 benchmarks list YOLO26n at 38.9 ms on CPU ONNX and 1.7 ms on a T4 with TensorRT.
• If one device must serve several cameras, divide the model budget by the camera count.
• If the fleet has more than a handful of device types, prefer runtimes that do not need per-device builds, or plan a build per device class.
• If the calling code is Swift, Kotlin, or C++, check the runtime's SDK for that language before committing.

Benchmark the Candidates on the Device#

Spec sheets and other people's benchmarks are a starting point. The decision is made on the hardware you will deploy, and by the slow frames, not the average: a runtime that is fast at the median but stalls on one frame in twenty will still drop frames. This script exports each candidate, times 100 inferences on a real frame, and judges each by its 95th percentile (p95):

from ultralytics import YOLO
from ultralytics.utils.downloads import safe_download

BUDGET_MS = 30  # the model's share of a 20 FPS frame budget
CANDIDATES = ("onnx", "openvino")  # runtimes the decision tree allows on this machine

if __name__ == "__main__":  # exports and inference can start worker processes on macOS and Windows
    frame = str(safe_download("https://ultralytics.com/images/bus.jpg"))  # use a real frame from your camera
    for fmt in CANDIDATES:
        model = YOLO(YOLO("yolo26n.pt").export(format=fmt))
        for _ in range(5):  # warm up: the first calls include one-time setup
            model(frame, verbose=False)
        runs = sorted(model(frame, verbose=False)[0].speed["inference"] for _ in range(100))
        p50, p95 = runs[49], runs[94]
        verdict = "meets" if p95 <= BUDGET_MS else "misses"
        print(f"{fmt:9s} p50 {p50:5.1f} ms  p95 {p95:5.1f} ms  {verdict} the {BUDGET_MS} ms budget")

On the Apple silicon laptop used to test this course, while other jobs shared the CPU:

onnx      p50  30.0 ms  p95  46.4 ms  misses the 30 ms budget
openvino  p50  22.8 ms  p95 113.5 ms  misses the 30 ms budget

Read it line by line:

• speed["inference"] is the model's own time for that frame, without decoding or drawing, so it compares runtimes fairly.
• OpenVINO wins the median and loses the p95 by a factor of more than two. Judged on averages, you would ship the runtime that stalls on busy moments; judged on p95, ONNX Runtime is the steadier choice on this machine.
• Both miss the budget, so the next move here is a smaller imgsz, an accelerator, or fewer cameras per device, not another CPU runtime.

For a quick accuracy-and-speed sweep across formats, Benchmark mode exports, validates, and times each one:

yolo benchmark model=yolo26n.pt data=coco8.yaml imgsz=640 format=onnx

Drop format= to sweep every format the machine supports. COCO8 has only four validation images, so treat its mAP column as a smoke test and use your own dataset YAML for accuracy decisions.

Pitfalls#

Edge Devices Add Four Questions#

The Raspberry Pi, NVIDIA Jetson, and DeepStream on Jetson guides cover the common targets in detail. Before you commit to an edge box, answer:

• Cold start: where is the TensorRT engine built, and who waits for it?
• Power and heat: can the enclosure sustain the model at full FPS all day?
• Memory: is there room for frame buffers and tracker state next to the model? INT8 halves the model's share.
• Updates: how does a new best.pt reach 1,000 devices? A Docker image keeps them consistent.