Enable local inference, RF-DETR

🤝 Community Help
1

Hi Roboflow Support Team,

I’m on the Core Plan and having issues with local inference for my fine-tuned RF-DETR model. The cloud API works perfectly, but local inference finds 0 detections on the same images.

Project Details:

  • Workspace: woodvision

  • Project: rv-c2vdq

  • Model: Version 10 (RF-DETR Medium)

  • API Key: UyXXXXXXXXXXXXXXX

  • Plan: Core ($79/mo)

What Works:

  • Using Roboflow cloud API with the roboflow Python library - detections work perfectly

  • Processing time: ~1500ms per image

What Doesn’t Work:

  • Using local inference with inference-gpu library

  • Same model, same images = 0 detections found

  • Code: model = get_model("rv-c2vdq/10", api_key=API_KEY) then model.infer(image_path, confidence=40)

Environment:

  • Python 3.11

  • inference-gpu installed

  • Windows with NVIDIA Quadro P520 GPU

  • The model loads successfully (no errors), but predictions list is always empty

Use Case: I’m building a production lumber defect detection system that requires <500ms inference time, which is why I need local inference instead of cloud API.

Questions:

  1. Is there a deployment step I’m missing to enable local inference for RF-DETR models?

  2. Does my Core Plan actually support local inference for fine-tuned RF-DETR models?

  3. Are there any special configuration steps needed to make local inference work properly?

The console shows the model loads successfully but then finds 0 detections on images that clearly have defects (verified via cloud API).

Can you help me troubleshoot why local inference isn’t working?

Thanks, Erik

2

Hi @EE_2026 Thanks for the detailed report—this is really helpful for troubleshooting. I noticed a potential issue in your code that’s likely the root cause:

You have: model.infer(image_path, confidence=40)

The confidence parameter expects a decimal (0-1), not a percentage. confidence=40 means 4000%, which would filter out everything.

Try this instead: model.infer(image_path, confidence=0.40)

Let me know if the confidence fix resolves it, otherwise I have some troubleshooting steps we can go through!