Low latency video streaming for real time computer vision

🤝 Community Help
1

has anyone integrated a low latency WebRTC stream with Roboflow project. I am exploring a setup where live camera/live video feeds are streamed with low latency and sent to a computer vision model for real time processing and i am looking for WebRTC solutions like Red5 Has anyone used a similar setup with Roboflow ?

2

Hey @achanta_abhay :waving_hand: I’m Balthasar, an engineer at Roboflow working on our cloud stream processing solution. This sounds like a great fit for what we’re building!

We already offer live WebRTC streaming for real-time computer vision, and we’re working on an improved v2 focused on lower latency, better video quality, and reliable, long-running streams, including automatic reconnection and recovery. We’re also making v2 much more cost-efficient! We already have several customers running streams 24/7, but it’s also useful for shorter processing jobs.

You can connect RTSP cameras, RTMP feeds, USB cameras, and webcams to the cloud and run your models and Roboflow Workflows there, without having to manage your own GPU infrastructure.

I’d love to hear more about your application! What are you building, what cameras are you using, and what latency are you aiming for? Send me a quick email at balthasar [at] roboflow [dot] com with your use case, and we can get you early access and schedule a call to help you get set up. Excited to see what you’re working on! :slightly_smiling_face:

3

I’ve played around with Luxonis’ OAK cameras for real time streaming (stream can be displayed in a browser too with WebRTC) and they have some examples of integrating with roboflow workflows. You can do inference right on the camera, and now that doing local inference costs no credits it may be more economically viable if you chose to use the local roboflow inference server.

I didn’t use their inference server because I needed total offline access. I used RF-DETR (object detection nano) because of its free commercial license but had to convert it to the camera platform architecture which was a bit of a pain but in the end latency seemed alright with a slow moving object in my environment (I don’t have metrics, but it was running at 24 FPS, not sure about model performance though in terms of latency).