# Inference workflows process-video ... is always falling back to CPU, not using the GPU **URL:** \ **Category:** 🤝 Community Help\ **Created:** [October 31, 2025, 12:10am UTC](https://discuss.roboflow.com/t/inference-workflows-process-video-is-always-falling-back-to-cpu-not-using-the-gpu/11401 "2025-10-31T00:10:51Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![Thomas](https://avatars.discourse-cdn.com/v4/letter/t/e36b37/32.png) [@Thomas](https://discuss.roboflow.com/u/Thomas)\ **Post date:** [October 31, 2025, 12:10am UTC](https://discuss.roboflow.com/t/inference-workflows-process-video-is-always-falling-back-to-cpu-not-using-the-gpu/11401/1 "2025-10-31T00:10:51Z")
## Hi Friends, I am having issues using from a CLI workflow with video processing my server GPU. I am alway falling back to CPU, which is toooo. I spent a day on trying docu and chatgtp, but haven’t got any success. Any ideas or how-to’s would be greatly welcomed. thx and cheers, Thomas Infrastructure: inference version: v0.59.0 inference-sdk version: v0.59.0 inference-cli version: v0.59.0 running on ubuntu linux server, within docker starting with “inference server start” as well as with a shell script. Same effects. I have verified that GPU is working can be reached thru CUDA from the container here is my script ### # 1\_aves\_del\_pozo\_detection\_nusakan # inference workflows process-images-directory --help # script is verified to work, but seems not to use GPU # updated inference on 27th of Oct. to v50 # updated /etc/docker/daemon.json on 27th of Oct. to include “default-runtime”: “nvidia” # script shall be run on nusakan server import os import subprocess video\_path = “/nunki/magellanes/uploads/cam4/2025/04/14/magellanes-cam4\_00\_20250414153636.mp4” output\_dir = “/nunki/yatina/output/video\_output/” workspace\_name = “yatinanet” workflow\_id = “aves-time-in-zone” api\_key = “mykey” processing\_target = “api” # api\_url = “[http://inference.yatina.net:9001](http://inference.yatina.net:9001)” # installed on Hermann’s nusakan / Docker, verified to run # aggregation\_format = “jsonl” maxfps = “25” # threads =“1” import os output\_path = os.path.abspath(output\_dir) print(f"Roboflow is writing files to: {output\_path}") # — force workflow to use GPU — os.environ[“ROBOFLOW\_DEVICE”] = “cuda” os.environ[“ROBOFLOW\_TARGET”] = “api” # — debug: check GPU before running — gpu\_test\_cmd = [ “inference”, “workflows”, “process-video”, “–help” ] # Run help command just to ensure CLI can see environment result = subprocess.run(gpu\_test\_cmd, check=False, capture\_output=True, text=True, env=os.environ) if “cuda” in result.stdout.lower() or “cuda” in result.stderr.lower(): print(“GPU detected by inference CLI ✅”) else: print(“⚠ Warning: GPU may not be detected by inference CLI”) print(“STDOUT:”, result.stdout) print(“STDERR:”, result.stderr) command = [ “inference”, “workflows”, “process-video”, “-v”, video\_path, “-o”, output\_dir, “-pt”, processing\_target, “–workspace\_name”, workspace\_name, “–workflow\_id”, workflow\_id, “–api-key”, api\_key, # “–api\_url”, api\_url, # “–aggregate”, # “–output\_file\_type”, aggregation\_format, ```auto "--max_fps", maxfps, "--save_out_video", "--allow_override", "--debug_mode", ``` # “–threads”, threads ] print(“Executing command:”) print(" ".join(command)) # Prints the full command as a string # Print standard output and errors result = subprocess.run(command, check=False, capture\_output=True, text=True) print(“STDOUT:\n”, result.stdout) print(“STDERR:\n”, result.stderr) # subprocess.run(command, check=False) * * * ### ![image](https://canada1.discourse-cdn.com/flex029/uploads/roboflow1/original/2X/7/734a1ef6a248c6925c78208a6b2fa35e8795fc7c.png) ---
**Author:** ![Ford](https://yyz1.discourse-cdn.com/flex029/user_avatar/discuss.roboflow.com/ford/32/7747_2.png) [@Ford](https://discuss.roboflow.com/u/Ford)\ **Post date:** [October 31, 2025, 2:23pm UTC](https://discuss.roboflow.com/t/inference-workflows-process-video-is-always-falling-back-to-cpu-not-using-the-gpu/11401/2 "2025-10-31T14:23:54Z")
Hi @Thomas, Great question and happy to help here! To start, can you confirm that you have installed inference for GPU using `pip install inference-gpu`? Here is a link to our [documentation](https://inference.roboflow.com/quickstart/run_a_model/) that walks through this process. Next, please verify that the GPU is properly attached to your container. NVIDIA’s [Container Toolkit Configuration guide](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html)walks through this configuration process. Finally, please verify that CUDA is available inside your container (see [CUDA verification](https://docs.pytorch.org/docs/stable/generated/torch.cuda.is_available.html)), and the ONNX `CUDAExecutionProvider` is listed ([guide](https://onnxruntime.ai/docs/execution-providers/)). ---
**Author:** ![Thomas](https://avatars.discourse-cdn.com/v4/letter/t/e36b37/32.png) [@Thomas](https://discuss.roboflow.com/u/Thomas)\ **Post date:** [October 31, 2025, 6:01pm UTC](https://discuss.roboflow.com/t/inference-workflows-process-video-is-always-falling-back-to-cpu-not-using-the-gpu/11401/3 "2025-10-31T18:01:36Z")
Hi Ford, thanks a lot for offering help and guidance. Here is my reply/status on your questions: 1. I have done pip install inference-gpu 2. container toolkit 3. CUDA verification 4. ONNX verification…that could be the issue what do you suggest me to do to fix that, please? Thank you very much indeed cheers Thomas ---
**Author:** ![Thomas](https://avatars.discourse-cdn.com/v4/letter/t/e36b37/32.png) [@Thomas](https://discuss.roboflow.com/u/Thomas)\ **Post date:** [November 1, 2025, 5:54pm UTC](https://discuss.roboflow.com/t/inference-workflows-process-video-is-always-falling-back-to-cpu-not-using-the-gpu/11401/4 "2025-11-01T17:54:37Z")
Meanwhile I found the following: there is a command difference inference-process-images-directory (which does use my GPU) versus inference process-video (which apparently DOES NOT use my GPU) nference-process-images-directory uses my server GPU processing\_target = “api” api\_url = [“http://localhost:9001”](http://localhost:9001) # installed on my Docker / Ubuntu Linux, verified to run with CUDA (see screenshot below) inference process-video does NOT use my server GPU, it only uses CPU, which is inefficient for video For whatever reason, your CLI comand **inference process-video** does not allow me to direct the processing target to an API , which means I do not see, where it is processing/infering the video file. I just see that only CPU is being consumed, and that it takes quite long. Observations: - The command **inference process-video** does not have a --pt or --api\_url option, so it sends the video analysos somewhere per default…possibly you can help me how to direct this to [“http://localhost:9001”](http://localhost:9001) - I tried to use in the command **inference process-video** the same --pt and --api\_url as in the other command inference-process-images-directory, but it still remains on CPU…and I do not get any related errors. ## Here is the script, I am using, and the below screenshots * * * # script is verified to work, but seems not to use GPU # updated inference on 27th of Oct. to v50.1 # updated /etc/docker/daemon.json on 27th of Oct. to include “default-runtime”: “nvidia” # verified that workflows process-images-directory does use GPU # script shall be run on nusakan server # Run help command just to ensure CLI can see environment import os import subprocess # — Config — video\_path = “/nunki/magellanes/uploads/cam4/2025/04/14/magellanes-cam4\_00\_20250414153636.mp4” output\_dir = “/nunki/yatina/output/video\_output/” workspace\_name = “yatinanet” workflow\_id = “aves-time-in-zone” api\_key = “edited” processing\_target = “api” # api\_url = “[http://localhost:9001](http://localhost:9001)” # GPU Docker server api\_url = “[http://0.0.0.0:9001](http://0.0.0.0:9001)” # GPU Docker server # api\_url = “[http://172.17.0.2:9001](http://172.17.0.2:9001)” # GPU Docker server IP fount out with docker inspect -f ‘{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}’ inference-server-yatina maxfps = 25 # — Force GPU — os.environ[“ROBOFLOW\_DEVICE”] = “cuda” os.environ[“ROBOFLOW\_TARGET”] = “api” # — Ensure output directory exists — os.makedirs(output\_dir, exist\_ok=True) print(f"Roboflow is writing files to: {os.path.abspath(output\_dir)}") # — Debug GPU detection — gpu\_test\_cmd = [“inference”, “workflows”, “process-video”, “–help”] result = subprocess.run(gpu\_test\_cmd, check=False, capture\_output=True, text=True, env=os.environ) stdout\_lower = result.stdout.lower() stderr\_lower = result.stderr.lower() if “cuda” in stdout\_lower or “cuda” in stderr\_lower or “tensorrt” in stdout\_lower or “tensorrt” in stderr\_lower: print(“GPU detected by inference CLI ✅”) else: print(“⚠ Warning: GPU may not be detected by inference CLI”) print(“STDOUT:”, result.stdout) print(“STDERR:”, result.stderr) # — Build and run workflow command — command = [ “inference”, “workflows”, “process-video”, “-v”, video\_path, “-o”, output\_dir, “-pt”, processing\_target, “–workspace\_name”, workspace\_name, “–workflow\_id”, workflow\_id, “–api-key”, api\_key, “–api\_url”, api\_url, “–max\_fps”, str(maxfps), “–save\_out\_video”, “–allow\_override”, “–debug\_mode” ] print(“Executing command:”) print(" ".join(command)) result = subprocess.run(command, check=False, capture\_output=True, text=True, env=os.environ) print(“STDOUT:\n”, result.stdout) print(“STDERR:\n”, result.stderr) if result.returncode != 0: print(f"Workflow failed with return code {result.returncode}") else: print(“Workflow completed successfully!”) * * * * * * * * * ![gyIjgMfvy0KYRYBP](https://canada1.discourse-cdn.com/flex029/uploads/roboflow1/original/2X/0/08dee3da1dbbc541a8ee2a3918f1a2a48fd1e9a5.png) ![i6AGqHtEgZjmhVHK](https://canada1.discourse-cdn.com/flex029/uploads/roboflow1/original/2X/0/08cde67d9919065d711f1ff31fe6be2823c304aa.png) ---
**Author:** ![Thomas](https://avatars.discourse-cdn.com/v4/letter/t/e36b37/32.png) [@Thomas](https://discuss.roboflow.com/u/Thomas)\ **Post date:** [November 1, 2025, 8:47pm UTC](https://discuss.roboflow.com/t/inference-workflows-process-video-is-always-falling-back-to-cpu-not-using-the-gpu/11401/5 "2025-11-01T20:47:01Z")
good news, I finally got it working to use inference workflows process-videos on my linux server with GPU. I needed to run the process inside the container, which can be done with a shell script and mounting the necessary directories. Hopefully that result helps other folks suffering the same issues to waste less time than I did. #!/bin/bash docker exec -it inference-server-yatina inference workflows process-video [and the rest of your options] Have fun! ---
**Author:** ![Ford](https://yyz1.discourse-cdn.com/flex029/user_avatar/discuss.roboflow.com/ford/32/7747_2.png) [@Ford](https://discuss.roboflow.com/u/Ford)\ **Post date:** [November 4, 2025, 3:30am UTC](https://discuss.roboflow.com/t/inference-workflows-process-video-is-always-falling-back-to-cpu-not-using-the-gpu/11401/6 "2025-11-04T03:30:10Z")
Good evening @Thomas! This is fantastic, thank you very much for sharing to help the community at large! We greatly appreciate your contribution here, will certainly be a valuable resource for many other Roboflow community members. Happy building! ---
**Author:** ![system](https://canada1.discourse-cdn.com/flex029/uploads/roboflow1/original/1X/bb6edbfa56d0cf2ee603f061e9e09103502cc8cc.png) [@system](https://discuss.roboflow.com/u/system)\ **Post date:** [November 11, 2025, 3:30am UTC](https://discuss.roboflow.com/t/inference-workflows-process-video-is-always-falling-back-to-cpu-not-using-the-gpu/11401/7 "2025-11-11T03:30:34Z")
This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.