Instance segmentation model with InferencePipeline doesn't work

emmajarvinen · April 3, 2024, 11:00am

Hi!

I’m trying to deploy an instance segmentation model I trained on Roboflow. I succeeded in making predictions on a single image, but when I’m trying to use InferencePipeline, I get a mysterious error. I have a MacBook M1 Pro with macOS Venture 13.3.1, roboflow version 1.1.26 and inference version 0.9.20.

This is the code:

from inference import InferencePipeline
from inference.core.interfaces.stream.sinks import render_boxes

API_KEY = "api-key"

MODEL_ID = "model-id"
VIDEO_FILE = "test-video.mp4"
MAX_FPS = 2
CONFIDENCE = 0.75

pipeline = InferencePipeline.init(
    model_id=MODEL_ID,
    video_reference=VIDEO_FILE,
    on_prediction=render_boxes,
    api_key=API_KEY,
    max_fps=MAX_FPS,
    confidence=CONFIDENCE,
)

pipeline.start()
pipeline.join()

And here are the error logs:

2024-04-03 13:41:51.985053 [E:onnxruntime:, sequential_executor.cc:514 ExecuteKernel] Non-zero status code returned while running Concat node. Name:'/model.11/Concat' Status Message: concat.cc:157 PrepareForCompute Non concat axis dimensions must match: Axis 3 has mismatched dimensions of 67 and 68
[04/03/24 13:41:51] ERROR    Unable to validate model artifacts, clearing cache: Unable to run test inference. Cause: [ONNXRuntimeError] : 1 : FAIL : Non-zero status code returned     roboflow.py:612
                             while running Concat node. Name:'/model.11/Concat' Status Message: concat.cc:157 PrepareForCompute Non concat axis dimensions must match: Axis 3 has                      
                             mismatched dimensions of 67 and 68                                                                                                                                        
Traceback (most recent call last):
  File "/.venv/lib/python3.10/site-packages/inference/core/models/roboflow.py", line 652, in validate_model
    self.run_test_inference()
  File "/.venv/lib/python3.10/site-packages/inference/core/models/roboflow.py", line 666, in run_test_inference
    result = self.infer(test_image)
  File "/.venv/lib/python3.10/site-packages/inference/core/models/instance_segmentation_base.py", line 97, in infer
    return super().infer(
  File "/.venv/lib/python3.10/site-packages/inference/core/models/roboflow.py", line 624, in infer
    return super().infer(image, **kwargs)
  File "/.venv/lib/python3.10/site-packages/inference/core/models/base.py", line 27, in infer
    predicted_arrays = self.predict(preproc_image, **kwargs)
  File "/.venv/lib/python3.10/site-packages/inference/models/yolov8/yolov8_instance_segmentation.py", line 41, in predict
    predictions = self.onnx_session.run(None, {self.input_name: img_in})
  File "/.venv/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 217, in run
    return self._sess.run(output_names, input_feed, run_options)
onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Non-zero status code returned while running Concat node. Name:'/model.11/Concat' Status Message: concat.cc:157 PrepareForCompute Non concat axis dimensions must match: Axis 3 has mismatched dimensions of 67 and 68

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/.venv/lib/python3.10/site-packages/inference/core/models/roboflow.py", line 610, in __init__
    self.validate_model()
  File "/.venv/lib/python3.10/site-packages/inference/core/models/roboflow.py", line 654, in validate_model
    raise ModelArtefactError(f"Unable to run test inference. Cause: {e}") from e
inference.core.exceptions.ModelArtefactError: Unable to run test inference. Cause: [ONNXRuntimeError] : 1 : FAIL : Non-zero status code returned while running Concat node. Name:'/model.11/Concat' Status Message: concat.cc:157 PrepareForCompute Non concat axis dimensions must match: Axis 3 has mismatched dimensions of 67 and 68

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/predict-on-video.py", line 12, in <module>
    pipeline = InferencePipeline.init(
  File "/.venv/lib/python3.10/site-packages/inference/core/interfaces/stream/inference_pipeline.py", line 229, in init
    model = get_model(model_id=model_id, api_key=api_key)
  File "/.venv/lib/python3.10/site-packages/inference/models/utils.py", line 195, in get_model
    return ROBOFLOW_MODEL_TYPES[(task, model)](model_id, api_key=api_key, **kwargs)
  File "/.venv/lib/python3.10/site-packages/inference/core/models/roboflow.py", line 614, in __init__
    raise ModelArtefactError from e
inference.core.exceptions.ModelArtefactError

Last week I trained an object detection model and I was able to use it effortlessly with InferencePipeline. However, now I’m wondering why I get the ModelArtefactError. It seems that the “validate_model” function terminates when trying to run “run_test_inference”. Both these functions are located in .venv/lib/python3.10/site-packages/inference/core/models/roboflow.py.

Jacob_Witt · April 3, 2024, 12:06pm

Hi @emmajarvinen - thanks for posting!

There seems to be an issue with the model. We’re going to investigate and keep you updated on this thread.

emmajarvinen · April 3, 2024, 12:18pm

Hi! Thanks for the quick reply.

p.peczek · April 3, 2024, 5:55pm

Hi there. @emmajarvinen, as you pointed out - the problem is with model package, not with pipeline itself. Could you tell us what model do you use?

emmajarvinen · April 3, 2024, 6:53pm

Model Type: Roboflow 3.0 Instance Segmentation (Accurate)
Checkpoint: COCOs-seg

p.peczek · April 3, 2024, 8:53pm

Yeah, but which model exactly we are talking about.

p.peczek · April 3, 2024, 8:53pm

Could you provide project name and version?

p.peczek · April 3, 2024, 8:54pm

Are you able to run normal inference from that model?

emmajarvinen · April 3, 2024, 9:52pm

Hi! Sorry for misunderstanding. And yes, I am able to run the model on normal inference. (I’ve only tested it on a couple of images).

I have a confidential project, but I can share the details in 1-on-1 conversation if that’s possible.

p.peczek · April 4, 2024, 3:13pm

Yeah, I am happy to have a call - interesting bug in general.

Before we have a call. Could u test the inference pipeline with the following env variable exported:
export MODEL_VALIDATION_DISABLED=True

p.peczek · April 5, 2024, 8:15am

@emmajarvinen Ok, Fixed the bug with model. Should work now.
Thanks for reporting the problem, it will drive internal discussions to prevent this situation from happening in the future.
Also, sorry for inconvenience

p.peczek · April 5, 2024, 8:17am

@emmajarvinen
To purge cache from faulty model:
rm -r /tmp/cache/ruusunlehti-segmentation/1/

emmajarvinen · April 5, 2024, 9:36am

Hi! It works now. Thank you!

p.peczek · April 5, 2024, 9:39am

Ok, glad it does. Once again, sorry for inconvenience

system · April 12, 2024, 9:39am

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Can't run semantic segmentation model on Nvidia Jetson Community Help bugs	4	109	April 25, 2024
Deploy Model on Raspberry Pi Community Help bugs	4	122	March 3, 2024
Deploying instance segmentation model to MacOS Community Help	6	185	August 11, 2023
Raspberry (Node.js code): 'No trained model was found for this model version' Community Help	5	242	August 15, 2023
I cannot run code that "detects in real time" the model I created. Community Help bugs , feature-request , segmentation	1	116	October 20, 2023

Instance segmentation model with InferencePipeline doesn't work

Related Topics