Error when inferring my uploaded custom model (yolov8)

Our uploaded model has error: Sign in to Roboflow , it says {“message”:“Model type not supported: (‘object-detection’, ‘yolov8’)”}. What happened?

  • Project type: object detection
  • Type of my model: yolov8

@narugo1992 i wonder if it’s related to this: Live inference using webcam doesn't work - #2 by Mohamed

If you can submit a bug report, as outlined below, we can look at the issue you’re facing.

  • the current workaround is to train with a Roboflow Train Credit, and Deploy that way, or retrain your YOLOv8 model and upload the weights

If it is an error with model weights upload, and inability to test with any deployments, please file a bug here, following this process.

Additionally, if there is information you do not want to post publicly, please send the bug report information to help@roboflow.com

We’ll need more information such as:

  1. the link to your project
  2. the link to your training notebook if you still have it
  3. the method you tested inference or deployment with when you noticed the error
  4. a screen recording of the error being triggered, if possible (this is very helpful for us)
  5. a copy of the file you tested model inference with when you noticed the failure, if you used a specific image or video file

For others that see this — we found the training was completed with ultralytics==8.0.42 in the training environment - but in the deploy env it was 8.0.89.

This means you need to retrain your model with the install for Ultralytics set as:

pip install ultralytics==8.0.20

pip install ultralytics==8.0.28

Okay, fine, I just ask this question.

Another question is, is there a way to remove the deployed model from Roboflow, in case there are issues with the current deployment? I saw in the documentation that attempting to redeploy the same model can cause errors. I may need to redeploy the model later.

I encountered a new problem just now: when exporting the dataset in YOLOv8 format from Roboflow, there is an error when using the 8.0.20 version of Ultralytics, which reads “FileNotFoundError: Dataset ‘/root/autodl-tmp/aniface/data.yaml’ not found :warning:, missing paths [‘/root/autodl-tmp/datasets/valid/images’]”. It seems that the format required by the old version of Ultralytics is not compatible with the format exported by Roboflow.

Based on my initial analysis, this might be caused by the relative paths in the exported dataset’s configuration file, which is not supported by the old version and leads to path errors.

@Mohamed Finally I uploaded this model (trained and deployed with 8.0.20), but it still says Model type not supported: ('object-detection', 'yolov8') :pensive:

Here is the deployed weight: anime_detection_demo Dataset and Pre-Trained Model by narugo1992

This is the structure of the weights directory

test_exp/person_detect/default_m_20/
├── args.yaml
├── confusion_matrix.png
├── events.out.tfevents.1682780598.autodl-container-a26611893c-9a9f1a50.411016.0
├── events.out.tfevents.1682817855.autodl-container-a26611893c-9a9f1a50.106739.0
├── events.out.tfevents.1682819492.autodl-container-a26611893c-9a9f1a50.138701.0
├── F1_curve.png
├── model_artifacts.json
├── P_curve.png
├── PR_curve.png
├── R_curve.png
├── results.csv
├── results.png
├── roboflow_deploy.zip
├── state_dict.pt
├── train_batch0.jpg
├── train_batch1.jpg
├── train_batch2.jpg
├── val_batch0_labels.jpg
├── val_batch0_pred.jpg
├── val_batch1_labels.jpg
├── val_batch1_pred.jpg
├── val_batch2_labels.jpg
├── val_batch2_pred.jpg
└── weights
    ├── best.pt
    └── last.pt

This is the code I used to deploy

from roboflow import Roboflow

rf = Roboflow(api_key="xxxxxxxxxxxxxxxxxxxxxxx")
project = rf.workspace("narugo1992-ynxbj").project("anime_detection_demo")
version = project.version(2)

print(version)

version.deploy("yolov8", "test_exp/person_detect/default_m_20/")

@Mohamed This is the screenshot of the weights page

I’m quite sure that the this error isn’t related to Live inference using webcam doesn't work - #2 by Mohamed

This is the full object in console

{
    "message": "Request failed with status code 500",
    "name": "AxiosError",
    "config": {
        "transitional": {
            "silentJSONParsing": true,
            "forcedJSONParsing": true,
            "clarifyTimeoutError": false
        },
        "transformRequest": [
            null
        ],
        "transformResponse": [
            null
        ],
        "timeout": 0,
        "xsrfCookieName": "XSRF-TOKEN",
        "xsrfHeaderName": "X-XSRF-TOKEN",
        "maxContentLength": -1,
        "maxBodyLength": -1,
        "env": {
            "FormData": null
        },
        "headers": {
            "Accept": "application/json, text/plain, */*"
        },
        "method": "post",
        "url": "/query/inferenceproxy",
        "params": {
            "api_key": "<hidden>",
            "confidence": 0.01,
            "image": "https://source.roboflow.com/I3coJGncvGUIq2vUQ0eaInuQhOd2/Y1JV95N0XolV3aibVO5J/original.jpg",
            "endpoint": "https://detect.roboflow.com/anime_detection_demo/2",
            "setDefault": true
        }
    },
    "code": "ERR_BAD_RESPONSE",
    "status": 500
}

and this is the response of this 500:

{"message":"Model type not supported: ('object-detection', 'yolov8')"}

I may have found the key to the problem. I found that when I trained the model locally and did not interrupt and resume during the training process (args.yaml with model set to yolo8m.yaml instead of a custom pt file), it can be uploaded normally and can also return results normally during actual use. Therefore, I suggest that Roboflow can improve this.

We run conversions for optimization of weights once uploaded.

To keep weights in their “base” form, for your own customizations, you’ll want to save your weights after training: How to Save and Load Model Weights in Google Colab

Roboflow Deploy with your uploaded weights, for successfully converted weights, should allow you to both deploy, and re-deploy, however:

@narugo1992 what is the suggestion for improvement?

It is not intended to run the notebook or training with arbitrary start/stoppage of the training process? I’m wondering if you followed the exact tutorial process, rather than the custom process with start/stop, and saw issues?

Our tutorials and the flow is meant to follow the base example, but you can train from a checkpoint of weights, and still successfully upload. I’ve done it before, myself.

What command did you use to train your model? And did your training stop early due to the patience settings / mAP not increasing?

This is my training code: imgutils/person.py at 0b76ec536ecb061f2de66c1429aa09f4ce8ab042 · deepghs/imgutils · GitHub

import os.path

from ultralytics import YOLO

from ..base import _TRAIN_DIR as _GLOBAL_TRAIN_DIR

_TRAIN_DIR = os.path.join(_GLOBAL_TRAIN_DIR, 'person_detect')


def train(train_cfg: str, session_name: str, level: str = 's',
          max_epochs: int = 300, save_per_epoch: int = 10, **kwargs):
    # Load a pretrained YOLO model (recommended for training)
    _last_pt = os.path.join(_TRAIN_DIR, session_name, 'weights', 'last.pt')
    if os.path.exists(_last_pt):
        model, resume = YOLO(_last_pt), True
    else:
        model, resume = YOLO(f'yolov8{level}.pt'), False

    # Train the model using the 'coco128.yaml' dataset for 3 epochs
    model.train(
        data=train_cfg, epochs=max_epochs,
        name=session_name, project=_TRAIN_DIR,
        save=True, save_period=save_per_epoch, plots=True,
        exist_ok=True, resume=resume,
        **kwargs
    )

Therefore, the issue lies in the fact that if the training process is interrupted and then resumed, the model parameter in args.yaml will be set to a .pt file instead of the original yolov8x.yaml. Based on existing tests, this causes problems. Hence, my suggestion is to perform a structural check on the uploaded models on the Roboflow server because different specifications of yolov8 models have distinct differences in structure and file size. The specific model type (n/s/m/l/x) should be obtained through the check instead of relying solely on the configuration in args.yaml.