Using Slicer with tracking only annotates first frame

[akella]

Hi! I’m new, so i admit i might be just doing a simple mistake. Still i would be grateful for you guidance!
TASK: I wanted to track birds in the sky.
Here is my input 1920x1080
Here is how small they are:

Took tracking example from repository
supervision/tree/develop/examples/tracking
Because objects are small, i used Slicer, and adapted example in such a way:

import argparse
import os

from inference.models.utils import get_roboflow_model
from tqdm import tqdm
from inference import get_model

import supervision as sv
model = get_model(model_id="yolov8n-640")

def slicer_callback(slice) -> sv.Detections:
    result = model.infer(slice)[0]
    detections = sv.Detections.from_inference(result)
    return detections


def process_video(
    roboflow_api_key: str,
    model_id: str,
    source_video_path: str,
    target_video_path: str,
    confidence_threshold: float = 0.01,
    iou_threshold: float = 0.5,
) -> None:
    model = get_roboflow_model(model_id=model_id, api_key=roboflow_api_key)

    tracker = sv.ByteTrack()
    box_annotator = sv.BoundingBoxAnnotator()
    label_annotator = sv.LabelAnnotator()
    frame_generator = sv.get_video_frames_generator(source_path=source_video_path)
    video_info = sv.VideoInfo.from_video_path(video_path=source_video_path)
    slicer = sv.InferenceSlicer(
        callback=slicer_callback,
        slice_wh=(256, 256),
        overlap_ratio_wh=(0.2, 0.2),
    )

    with sv.VideoSink(target_path=target_video_path, video_info=video_info) as sink:
        for frame in tqdm(frame_generator, total=video_info.total_frames):
            # results = model.infer(
            #     frame, confidence=confidence_threshold, iou_threshold=iou_threshold
            # )[0]
            # detections = sv.Detections.from_inference(results)
            
            detections = slicer(frame)


            detections = tracker.update_with_detections(detections)

            annotated_frame = box_annotator.annotate(
                scene=frame.copy(), detections=detections
            )

            annotated_labeled_frame = label_annotator.annotate(
                scene=annotated_frame, detections=detections
            )

            sink.write_frame(frame=annotated_labeled_frame)

I tried this code on walking people video, it tracked people across the whole video.
Yet when i try it on my birds video, i only get first frame annotated, and then nothing.
Is it due to object size? Or could there be any other reason?
Here is what i get as an annotated result Imgur: The magic of the Internet

OS: Mac OS
I ran it with:

python inference_example.py     --roboflow_api_key ROBOFLOW_KEY     --source_video_path input.mp4     --target_video_path tracking_result.mp4

p.s: also not sure if this considered a roboflow app, sorry if not.

[akella]

So, i went a bit further, i trained model on similar sky images.
when i put my video into Visualize interface, it tracks birds successfully. Or rather - detects. Here is video
But when i try to do tracking with same script, it only annotates first frame again.

def process_video(
    roboflow_api_key: str,
    model_id: str,
    source_video_path: str,
    target_video_path: str,
    confidence_threshold: float = 0.01,
    iou_threshold: float = 0.5,
) -> None:
    model = get_roboflow_model(model_id="birds-nqymw/2", api_key=roboflow_api_key)

    tracker = sv.ByteTrack()
    box_annotator = sv.BoundingBoxAnnotator()
    label_annotator = sv.LabelAnnotator()
    frame_generator = sv.get_video_frames_generator(source_path=source_video_path)
    video_info = sv.VideoInfo.from_video_path(video_path=source_video_path)
    

    with sv.VideoSink(target_path=target_video_path, video_info=video_info) as sink:
        for frame in tqdm(frame_generator, total=video_info.total_frames):
            slicer = sv.InferenceSlicer(
                callback=slicer_callback,
                slice_wh=(512, 512),
                overlap_ratio_wh=(0.5, 0.5),
            )
            detections = slicer(frame)


            detections = tracker.update_with_detections(detections)

            annotated_frame = box_annotator.annotate(
                scene=frame.copy(), detections=detections
            )

            annotated_labeled_frame = label_annotator.annotate(
                scene=annotated_frame, detections=detections
            )

            sink.write_frame(frame=annotated_labeled_frame)

As im a little beginner in this, i would be grateful in just general directions of what could be wrong to see such thing. Low confidence in detection? Too small? Some script error?

[akella]

For anyone having same issues, i found root of my problems
it is described in this issue:

Problems with tracker.update_with_detections(detections)

Basically for small fast objects there is no overlap, so tracker fails to track them continuously, so i either need a slowmo video, or slow birds, or fat birds =)