Identifying multiple class predictions the same smart polygon area in json response for instance segmentation model

[kedar]

I’m running an Instance Segmentation Model to identify the color of walnut kernels.

The json response contains multiple classes predicted for the same smart polygon areas in addition to unique predictions. See attached image reference generated by your web app.

I would like to understand, how I can programmatically identify the smart polygons that have multiple class predictions from the json response.

I’m not including the json response here as it is too big at attach.

[Jacob_Witt]

@kedar This is a great question!

I would approach this the following way:

1. for each object detected by your model, use the x and y values as a centroid, use width and height to create a bounding box.
2. For each object i detected, loop through all other objects to see whether the other object’s centroid falls within the bounding box of i.
3. Group the results by calculating the mean centroid, width, and height, and append all of the classes.

Here is the code ChatGPT returned - I can’t vouch that it runs, but should be directionally helpful.

import json

def centroid_inside_object(centroid, obj):
    return (obj["x"] <= centroid["x"] <= obj["x"] + obj["width"]) and (obj["y"] <= centroid["y"] <= obj["y"] + obj["height"])

def find_containing_objects(data):
    results = []

    # For each object in the data
    for i, obj in enumerate(data):
        centroid = {"x": obj["x"], "y": obj["y"]}
        classes = []

        # Check every other object to see if our current centroid is inside it
        for j, other_obj in enumerate(data):
            if i != j and centroid_inside_object(centroid, other_obj):
                classes.append(other_obj["class"])

        # If the centroid of the object is inside any other objects, save the results
        if classes:
            results.append({
                "x": centroid["x"],
                "y": centroid["y"],
                "classes": classes
            })

    return results