Local Inference Not Working on JetPack 6.2.2 (YOLOv8 Workflow + RTSP Stream)

[NgocTrang]

Hi Roboflow Support Team,

I’m on the Basic Plan and I’m having issues running my workflow locally on a Jetson Orin Nano (JetPack 6.2.2) using a YOLOv8 detection model.

Workspace Details:

• api_key=[removed by mo d]

• workspace_name=“kawada rd”

• workflow_id=“ch eck”

What Wo rk:

• The cloud API works perf ectly.

• I was also able to process video streams (RTSP link from an OAK camera) using my workflow locally (Deploy → Video → Live Video → Run Locally). Both worked succes sfully:

  • My Windo ws 11 PC

  • Jetson Orin Nano running JetP ack 5.1.1

I upgraded to JetPack 6.2.2 because I want to use the Detections Combine block in my workflow.

• I am able to successfully connect to my inference server using the “Choose an inference server => Other => Connect => Locally => RTSP => Runworkflow” option with Jetpack 6.2.2 in the Roboflow dashboard.

What Doe sn’t Work:

• Processing video streams (RTSP link from OAK camera) with my workflow locally on Jetson Orin Nano running J etPack 6.2.2.

• Code used: (from Deploy → Video → Live Video → Run Locally)

pipeline = InferencePipeline.init _with_workflow(

api_key=“[removed by mod]”,

workspace _name=“kawadard”,

wo rkflow_id=“check”,

video_reference=“rtsp://localho s t:8554/mystream” ,

max_fps=30,

o n_ pr e diction=my_sink

)

Environment:

• Jetpack: 6.2.2

• Ubuntu: 22.04

• Python: 3.10.12

• Inference packag e: 0.64.7

Issues Details:

After waiting for a while, the system stopped automatically without displaying anything:

[ WARN:308@339.152] global cap_ffmpeg_impl.hpp:453 _opencv_ffmpeg_interrupt_callback Stream timeout triggered after 30007.662934 ms
[ WARN:309@379.845] global cap_ffmpeg_impl.hpp:453 _opencv_ffmpeg_interrupt_callback Stream timeout triggered after 30028.735226 ms
[ WARN:309@380.083] global cap_ffmpeg_impl.hpp:453 _opencv_ffmpeg_interrupt_callback Stream timeout triggered after 30288.40 5730 ms
Killed

Questions:

Can you help me troubleshoot why l ocal inference isn’t wor king?

Thank you very much!

• Project Type: Object detection
• Operating System & Browser: Linux
• Project Universe Link or Workspace/Project ID: workspace_name=“kawadard”,
  workflow_id=“check”, Project ID=”eq_detection-py6oi/7”
• Do you grant Roboflow Support permission to access your Workspace for troubleshooting? (Yes/No): Yes

[erik_roboflow]

Hi @NgocTrang ,

I’ll try to repro the issue, but I suspect it’s because you’re running InferencePipeline, which will use locally installed inference package for inferencing, so it will run on cpu instead of gpu. You should runinference server in docker container (gpu-enabled) and InferenceHttpClient (.run_workflow() or .start_inference_pipeline_with_workflow() ) and point the http client to your server (localhost:9001 by default) that’s running in docker container.

I noticed you shared your API key, I removed that, but you should rotate it (remove old one and create a new one) ASAP.

Thanks, Erik

[NgocTrang]

Hello @erik_roboflow ,

Thank you for your response. Perhaps your guess about the cause of this error is correct, but I think it is more likely due to a compatibility issue. Because:

Code:

def my_sink(result, video_frame) :

if result.get(“output_image” ):

cv2.imshow(“Workflow Image”, result[“output_image”].numpy_image)

cv2.waitKey(1)

When I deleted these commands that related cv2 library, the code above (InferencePipeline) was able to return the workflow’s prediction results:

When I use cv2.imshow() to display the video output, the system freezes at that point.

This indicates that the system is getting stuck at the display step. I run cv2.imshow() separately for testing, and it successfully displays a single image. My Jetson has OpenCV v4.10.0 installed with CUDA.

So, could you help me check where the error is really coming from?

Regarding the method you mentioned earlier (InferenceHttpClient), I’m very interested in using it. However, I’ve only been able to find examples using single images. So if you don’t mind, could you share a sample code or documentation that deploys with live video input (Example: RTSP link)?

Thank you also for reminding me about the API key issue, I really hadn’t noticed it. I’ve already created a new one, so it’s fine. I would really appreciate it if you could respond as soon as possible. Thank you very much!

[erik_roboflow]

Hi @NgocTrang ,

Could you try the code below?

import cv2
from inference_sdk import InferenceHTTPClient
from inference_sdk.webrtc import RTSPSource, StreamConfig, VideoMetadata

# Initialize client
client = InferenceHTTPClient.init(
    api_url="http://localhost:9001",
    api_key="MY_API_KEY"
)

# Configure video source (RTSP stream)
source = RTSPSource("rtsp://localhost:8554/mystream")

# Configure streaming options
config = StreamConfig(
    stream_output=["image"],
    data_output=["infor"]      # Get prediction data via datachannel
)

# Create streaming session
session = client.webrtc.stream(
    source=source,
    workflow="check",
    workspace="kawadard",
    image_input="image",
    config=config
)

# Handle incoming video frames
@session.on_frame
def show_frame(frame, metadata):
    cv2.imshow("Workflow Output", frame)
    if cv2.waitKey(1) & 0xFF == ord("q"):
        session.close()

# Handle prediction data via datachannel
@session.on_data()
def on_data(data: dict, metadata: VideoMetadata):
    print(f"Frame {metadata.frame_id}: {data}")

# Run the session (blocks until closed)
session.run()

[erik_roboflow]

This worked for me, I used webcam on my macbook as I don’t have rtsp camera on hand (just set your API key):

import cv2
from inference_sdk import InferenceHTTPClient
from inference_sdk.webrtc import WebcamSource, RTSPSource, StreamConfig, VideoMetadata

# Initialize client
client = InferenceHTTPClient.init(
    api_url="http://localhost:9001",
    api_key="8iE***"
)

# Configure video source (RTSP stream)
source = WebcamSource(resolution=(1280, 720))

# Configure streaming options
config = StreamConfig(
    stream_output=["image"],
    data_output=["infor"]      # Get prediction data via datachannel
)

# Create streaming session
session = client.webrtc.stream(
    source=source,
    workflow="check",
    workspace="kawadard",
    image_input="image",
    config=config
)

# Handle incoming video frames
@session.on_frame
def show_frame(frame, metadata):
    cv2.imshow("Workflow Output", frame)
    if cv2.waitKey(1) & 0xFF == ord("q"):
        session.close()

# Handle prediction data via datachannel
@session.on_data()
def on_data(data: dict, metadata: VideoMetadata):
    print(f"Frame {metadata.frame_id}: {data}")

# Run the session (blocks until closed)
session.run()

[NgocTrang]

hi @erik_roboflow ,
Thank you for the quick reply!
I tried running your code. At first, it threw a ‘500 server error… Internal error.’ I believe this happened because the inference‑server container couldn’t access the host’s RTSP server. So I changed the line -p 9001:9001 \ to --network host \ and ran it again. Luckily, it ran successfully:

However, it still doesn’t display any image:

To clarify, after adding the code to save each frame as an image and to print frame.shape, unexpectedly only the very first frame was saved/printed:

Meanwhile, I can still do the same thing with RTSP link and OpenCV in a simple way as follows. Therefore, I don’t think this error is caused by my OpenCV package:

[erik_roboflow]

Hi @NgocTrang , can you share full code (without API Key) so I can try to repro this on my end 1:1?

[NgocTrang]

Hi @erik_roboflow,

I just made a few changes to the def show_frame to see if it actually worked.

import cv2
from inference_sdk import InferenceHTTPClient
from inference_sdk.webrtc import WebcamSource, RTSPSource, StreamConfig, VideoMetadata
import os

os.makedirs("image", exist_ok=True)

# Initialize client
client = InferenceHTTPClient.init(
    api_url="http://localhost:9001",
    api_key="key"
)

# Configure video source (RTSP stream)
source = RTSPSource("rtsp://localhost:8554/mystream")


# Configure streaming options
config = StreamConfig(
    stream_output=["image"],
    data_output=["info"]      # Get prediction data via datachannel
)

# Create streaming session
session = client.webrtc.stream(
    source=source,
    workflow="check",
    workspace="kawadard",
    image_input="image",
    config=config
)

# Handle incoming video frames
@session.on_frame
def show_frame(frame, metadata):
    print(frame.shape)
    if metadata.frame_id % 10 == 0:
        cv2.imwrite(f"image/frame_{metadata.frame_id}.jpg", frame)
    cv2.imshow("Workflow Output", frame)
    if cv2.waitKey(1) & 0xFF == ord("q"):
        session.close()

# Handle prediction data via datachannel
@session.on_data()
def on_data(data: dict, metadata: VideoMetadata):
    print(f"Frame {metadata.frame_id}: {data}")

# Run the session (blocks until closed)
session.run()

To be more certain, you can use the exact project that I have uploaded to GitHub here github

[erik_roboflow]

Hi @NgocTrang ,

The code provided works on my end. I mocked rtsp stream, but it works as expected, so my assumption is your local server. Could you check container logs? Docker deskotp helps a ton here, you can just click on the container to check logs.

In your case you’d need to share both ports 9001 (so your py script can connect to docker) and 8554 (so server can access rtsp cam).

[NgocTrang]

Hi @erik_roboflow ,

Thanks for the pointers — I’ve got it working. What fixed it for me was upgrading to OpenCV 4.12.0 (CUDA).

However, I don’t think the issue was only the OpenCV package.

• With OpenCV 4.10.0 (CUDA) I can still use imshow() for single images or videos without any problem, so it wasn’t purely an OpenCV bug.
• The failure only happened in my full pipeline (Roboflow Inference + CUDA build) when running inside my container/host environment: After I simply import the inference-sdk library, this code failed to display an image:

So it looks like it was a combination issue — OpenCV build + GUI backend + container/display setup — rather than OpenCV alone.

Anyway, thank you very much for supporting me so wholeheartedly during this time!

[NgocTrang]

Update: I think I have identified the exact root cause of the issue.

After reinstalling inference v1.0.1, the system again failed to display anything. I was able to fix this by removing the bundled opencv-python dependency and downgrading NumPy to 1.26.4.

pip install numpy==1.26.4
pip uninstall opencv-python

This indicates an incompatibility between the pip opencv-python wheel included with the Inference package (tested across multiple versions) and the system OpenCV with CUDA on JetPack 6.2.2.