Use Claude for Visual Classification

[Sarah_A]

I would like to use Claude 3.5 Sonnet for objuect classification. In my specific case, I would like to detect whether a hand is present or not in the image. I added a Claude Block to my workflow and used the below code block. However, it does not work and I know there’s something I’m missing.

Is there a way to make this process easier?

{
  "type": "roboflow_core/anthropic_claude@v1",
  "name": "hand_detection",
  "images": "$inputs.image",
  "prompt": "Identify the hands in this images ",
  "task_type": "classification",
  "classes": [
    "hand",
    "no hand"
  ]
}

[Grzegorz]

Hi @Sarah_A,

It seems you are missing model_version, I created sample workflow with Claude 3.5 Sonnet for classification, here is my workflow:

{
  "version": "1.0",
  "inputs": [
    {
      "type": "InferenceImage",
      "name": "image"
    }
  ],
  "steps": [
    {
      "type": "roboflow_core/anthropic_claude@v1",
      "name": "anthropic_claude",
      "images": "$inputs.image",
      "api_key": "INSERT-YOUR-API-KEY",
      "task_type": "classification",
      "model_version": "claude-3-5-sonnet",
      "classes": [
        "car",
        "lorry"
      ]
    }
  ],
  "outputs": [
    {
      "type": "JsonField",
      "name": "anthropic_claude",
      "coordinates_system": "own",
      "selector": "$steps.anthropic_claude.*"
    }
  ]
}

I tested this workflow and got below output:

[
  {
    "anthropic_claude": {
      "output": "{\"class_name\": \"car\", \"confidence\": 0.99}",
      "classes": [
        "car",
        "lorry"
      ]
    }
  }
]

You can also use our workflows editor to build your workflows, here is my example workflow

Thanks,
Grzegorz