Issues annotations for unity perception not showing

Community Help
1

Hello,
I’m using Unity Perception to generate synthetic data, and I see that dragging the generated JSON into the upload section doesn’t result in an annotated image. I have attached the JSON below in case the formatting is incorrect.

{
  "frame": 4,
  "sequence": 0,
  "step": 2,
  "timestamp": 0.0498,
  "captures": [
    {
      "@type": "type.unity.com/unity.solo.RGBCamera",
      "id": "camera",
      "description": "",
      "position": [
        -0.64,
        1.11,
        -10.0
      ],
      "rotation": [
        0.0,
        0.0,
        0.0,
        1.0
      ],
      "velocity": [
        0.0,
        0.0,
        0.0
      ],
      "acceleration": [
        0.0,
        0.0,
        0.0
      ],
      "filename": "step2.camera.png",
      "imageFormat": "Png",
      "dimension": [
        500.0,
        500.0
      ],
      "projection": "Perspective",
      "matrix": [
        3.569568,
        0.0,
        0.0,
        0.0,
        3.569568,
        0.0,
        0.0,
        0.0,
        -1.0006001
      ],
      "annotations": [
        {
          "@type": "type.unity.com/unity.solo.BoundingBox3DAnnotation",
          "id": "bounding box 3D",
          "sensorId": "camera",
          "description": "Produces 3D bounding box ground truth data for all visible objects that bear a label defined in this labeler's associated label configuration.",
          "values": [
            {
              "instanceId": 2,
              "labelId": 1,
              "labelName": "starPillow",
              "translation": [
                0.216937542,
                0.248326883,
                6.639356
              ],
              "size": [
                1.87891459,
                0.815005064,
                1.775292
              ],
              "rotation": [
                -0.7071068,
                0.0,
                0.0,
                0.7071067
              ],
              "velocity": [
                0.0,
                0.0,
                0.0
              ],
              "acceleration": [
                0.0,
                0.0,
                0.0
              ]
            },
            {
              "instanceId": 3,
              "labelId": 1,
              "labelName": "starPillow",
              "translation": [
                -1.51000047,
                0.248326883,
                10.739357
              ],
              "size": [
                1.87891459,
                0.815005064,
                1.775292
              ],
              "rotation": [
                -0.7071068,
                0.0,
                0.0,
                0.7071067
              ],
              "velocity": [
                0.0,
                0.0,
                0.0
              ],
              "acceleration": [
                0.0,
                0.0,
                0.0
              ]
            }
          ]
        },
        {
          "@type": "type.unity.com/unity.solo.BoundingBox2DAnnotation",
          "id": "bounding box",
          "sensorId": "camera",
          "description": "Produces 2D bounding box annotations for all visible objects that bear a label defined in this labeler's associated label configuration.",
          "values": [
            {
              "instanceId": 2,
              "labelId": 1,
              "labelName": "starPillow",
              "origin": [
                152.0,
                98.0
              ],
              "dimension": [
                255.0,
                238.0
              ]
            },
            {
              "instanceId": 3,
              "labelId": 1,
              "labelName": "starPillow",
              "origin": [
                45.0,
                156.0
              ],
              "dimension": [
                139.0,
                147.0
              ]
            }
          ]
        },
        {
          "@type": "type.unity.com/unity.solo.SemanticSegmentationAnnotation",
          "id": "semantic segmentation",
          "sensorId": "camera",
          "description": "Generates a semantic segmentation image for each captured frame. Each object is rendered to the semantic segmentation image using the color associated with it based on this labeler's associated semantic segmentation label configuration. Semantic segmentation images are saved to the dataset in PNG format. Please note that only one SemanticSegmentationLabeler can render at once across all cameras.",
          "imageFormat": "Png",
          "dimension": [
            500.0,
            500.0
          ],
          "filename": "step2.camera.semantic segmentation.png",
          "instances": [
            {
              "labelName": "starPillow",
              "pixelValue": [
                0,
                0,
                255,
                255
              ]
            }
          ]
        }
      ]
    }
  ]
}
2

Hi @chaitaes

Our supported import formats are listed at roboflow.com/formats. Unity Preception is a supported import format, and while I’m not familiar with Unity Preception, comparing it with our example format does show some differences in format.

3

Oh, okay thanks for narrowing it down for me. :grin: I think Unity Perception updated their JSON format, so it is no longer supported. I’ll figure out another way to get everything running.

4

Hi @chaitaes

I took a quick look at the Unity Perception package docs, and it looks like it says that “The schema of the value object is specific to tasks”, which is specified in the annotation specs. The annotation spec for a 2D bounding box should be in the format listed in our example import file.

Again, I don’t have experience with Unity Perception, but it looks like the one you posted is for the 3D bounding box. If there is a way to change your dataset annotations to a 2D bounding box, I think you should be able to import into Roboflow without a problem

5

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.