Detectron2 Video inferencing shows category IDs instead of class names

I have uploaded a labelled Image data using roboflow and extracted it as a COCO format.

I’m using detectron2 to do object detection on a video after training a custom dataset. The output video does the detection, but it indicates category ids of either 1 or 2. But I want my detection to show the class names of either ‘weed’ or ‘crop’ instead.


I have downloaded the config files and the model weights for my custom training and used them as shown below;

%run detectron2/demo/ --config-file /content/config2.yaml --video-input /content/crop-weed.mp4 --confidence-threshold 0.6 --output output_colab.mp4 \
--opts MODEL.WEIGHTS /content/output/model_final.pth

Expected behavior

I expected detectron2 to do the video inference and show the class names of either ‘weed’ or ‘crop’ around the bounding box of the object detected.

I need help, please.

Hi @Kamal_Moha

Sorry to hear about the trouble. Could you share the code you are using?

Was this based off of one of our guides or notebooks and if so, could you please share the link with us so that we can help you solve this faster.

Thanks @leo for getting back. I was following the getting started notebook from detectron2 documentation. It worked when I run their documentation notebook, but when a similar code on a custom dataset, it failed to work as expected. The video inference code for my custom dataset is as below;

%run detectron2/demo/ --config-file /content/config2.yaml --video-input /content/crop-weed.mp4 --confidence-threshold 0.6 --output output_colab2.mp4 \
--opts MODEL.WEIGHTS /content/output/model_final.pth

Though with more deep research I have been able to find a solution using the below code;

import detectron2
from detectron2.utils.logger import setup_logger
# import some common libraries
import numpy as np
import tqdm
import cv2
# import some common detectron2 utilities
from detectron2 import model_zoo
from detectron2.engine import DefaultPredictor
from detectron2.config import get_cfg
from detectron2.utils.video_visualizer import VideoVisualizer
from detectron2.utils.visualizer import ColorMode, Visualizer
from import MetadataCatalog
import time

# Extract video properties
video = cv2.VideoCapture('/content/crop-weed.mp4')
width = int(video.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(video.get(cv2.CAP_PROP_FRAME_HEIGHT))
frames_per_second = video.get(cv2.CAP_PROP_FPS)
num_frames = int(video.get(cv2.CAP_PROP_FRAME_COUNT))

# Initialize video writer
video_writer =  cv2.VideoWriter('inst-seg.mp4', fourcc=cv2.VideoWriter_fourcc(*"mp4v"), fps=float(frames_per_second),
                                frameSize=(width, height), isColor=True)

# Initialize visualizer
v = VideoVisualizer(MetadataCatalog.get("my_dataset_train"), ColorMode.IMAGE)

def runOnVideo(video, maxFrames):
    """ Runs the predictor on every frame in the video (unless maxFrames is given),
    and returns the frame with the predictions drawn.

    readFrames = 0
    while True:
        hasFrame, frame =
        if not hasFrame:

        # Get prediction results for this frame
        outputs = predictor(frame)

        # Make sure the frame is colored
        frame = cv2.cvtColor(frame, cv2.COLOR_RGB2BGR)

        # Draw a visualization of the predictions using the video visualizer
        visualization = v.draw_instance_predictions(frame, outputs["instances"].to("cpu"))

        # Convert Matplotlib RGB format to OpenCV BGR format
        visualization = cv2.cvtColor(visualization.get_image(), cv2.COLOR_RGB2BGR)

        yield visualization

        readFrames += 1
        if readFrames > maxFrames:

# Enumerate the frames of the video
for visualization in tqdm.tqdm(runOnVideo(video, num_frames), total=num_frames):

    # Write test image
    # cv2.imwrite('/content/33514.jpg', visualization)

    # Write to video file

# Release resources

The above code works as expected, but it’s very long & not the best.

It would be really nice to make the first code work and that’s something I’m searching. I would love your help on that

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.