Deploy and Call an Endpoint

Shared Inference or a Dedicated Endpoint#

Every model can already be called through Platform's shared inference API, which suits testing. Production traffic belongs on a dedicated endpoint:

Create and Size the Endpoint#

Open the model's Deploy tab. Your browser measures latency to all 42 regions (14 in the Americas, 13 in Europe, 12 in Asia-Pacific, 3 in the Middle East and Africa) and sorts the table by it:

Click Deploy on the row closest to your callers, review CPU, memory, price, and the generated name (such as yolo26n-iowa), and click Create Deployment. You can also start from New Deployment on your profile's Deployments tab:

Size by traffic pattern:

Plans allow 3 (Free), 10 (Pro), or unlimited (Enterprise) deployments, and stopped endpoints still count. The endpoint accepts traffic once its badge reads Ready:

Worked Example: A Client That Survives Production#

Each endpoint has a URL like https://predict-<id>-<hash>-<region>.a.run.app serving /predict (POST), /health, and an interactive /docs page. It accepts only the API key bound at creation; workspace owners see that key pre-filled in the deployment's Predict tab under Docs, next to Python, JavaScript, and cURL snippets. The snippets make one request; production code must also survive a cold start and a busy instance:

import os
import time
from collections import Counter

import httpx

URL = "https://YOUR_DEPLOYMENT_URL.run.app/predict"
HEADERS = {"Authorization": f"Bearer {os.environ['ULTRALYTICS_API_KEY']}"}  # the deployment's bound key, server-side only

def predict(client, path, conf=0.25, attempts=4):
    """POST one image; retry timeouts, 429, and 5xx with backoff; fail fast on other errors."""
    for attempt in range(attempts):
        try:
            with open(path, "rb") as f:
                r = client.post(URL, headers=HEADERS, data={"conf": conf}, files={"file": f}, timeout=90)
            if r.status_code < 400:
                return r.json()
            if r.status_code != 429 and r.status_code < 500:
                r.raise_for_status()  # 401 wrong key, 413 too large: retrying won't help
            wait, reason = float(r.headers.get("Retry-After", 2**attempt)), r.status_code
        except httpx.TimeoutException:  # a scaled-to-zero endpoint can take up to a minute to wake
            wait, reason = 2**attempt, "timeout"
        print(f"attempt {attempt + 1}: {reason}, retrying in {wait:.0f} s")
        time.sleep(wait)
    raise RuntimeError(f"endpoint still unavailable after {attempts} attempts")

with httpx.Client() as client:
    response = predict(client, "bus.jpg")
for image in response["images"]:
    counts = Counter(d["name"] for d in image["results"])
    print(f"{sum(counts.values())} objects {dict(counts)} in {image['speed']['inference']:.1f} ms")

We tested it against a local stub that answers 429 once and then returns a real YOLO26n result for bus.jpg in the endpoint's response format:

attempt 1: 429, retrying in 2 s
5 objects {'bus': 1, 'person': 4} in 34.9 ms

Line by line, the decisions that matter:

• timeout=90 covers a cold start of up to about a minute; the default timeouts of most HTTP clients are far shorter.
• 429 and 5xx are retried, honoring Retry-After when the endpoint sends it. Each endpoint serves from a single instance and sheds load with 429 when busy.
• Other 4xx errors fail fast: 401 means the wrong key and 413 a body over 32 MB, and neither fixes itself.
• The response holds one entry per image (or per video frame) in images, each with results (class, name, confidence, box with x1, y1, x2, y2) and speed in milliseconds.

Optional form fields are conf (default 0.25), iou (0.7), imgsz (the model's training size), vid_stride for video, and normalize=true for 0–1 coordinates. The cURL equivalent is one line per field:

curl -X POST "https://YOUR_DEPLOYMENT_URL.run.app/predict" \
  -H "Authorization: Bearer $ULTRALYTICS_API_KEY" \
  -F "file=@bus.jpg" -F "conf=0.25" -F "iou=0.7" -F "imgsz=640"

Ship New Versions Without Changing the URL#

Replace model on the deployment page swaps in another completed model from the same workspace. The old model keeps serving until the new one is ready, the URL, region, and key stay the same, and if the rollout fails the previous model stays live. Rolling back is the same action pointed at the previous model, so keep it in the project. Update configuration resizes in place, and Stop pauses billing and keeps the URL.

Some changes need a new endpoint: regions are fixed (moving means delete and recreate, with a new URL), and custom domains aren't supported, so put your own gateway in front if callers need a stable branded hostname. To run outside Platform, export the model (Export for Deployment) or self-host it behind Triton Inference Server.

Pitfalls#