Best practices for developing custom python blocks?

[sbroberg]

New to roboflow here. I’m a software developer helping out my non-programming son who has been tasked with integrating an automated workflow into a university project. After a number of false starts we finally got things working, but there’s part of the process that seems really awkward to me: writing and debugging custom python blocks.

We use the website as the dev environment and run inference locally on our machine (MacOS), but that means when you run a workflow (or even use the “test block” option in the browser), if there’s a problem with your python code, you get extremely unhelpful error messages, sometimes as absurd as just “x”.

This is because the web UI (and what appears in the console output) seems to be just printing out the last error produced by the python interpreter, and not the backtrace, so there are no line numbers that tell you where the problem is (and there are none in the online editor anyway). Since a block will depend on inputs such as a source image (WorkflowImageData) and model output (sv.Detections), it’s nontrivial to set up a test harness so you can unit test a block.

Is there a standard way people are using to run/test the python blocks outside of roboflow, or at least outside the browser UI that more closely resembles normal python development?

[Ford]

Hi @sbroberg!
Welcome to the Roboflow Community! This is a fantastic question! In talking with some fellow Roboflowers, we like to think about the custom python blocks as standalone functions that should be kept as small and isolated as possible. Its important to not try to everything at once within a singular block.

For example use three small, decoupled blocks rather than one large block. Also, only use limited inputs and return a compatible output.

Again, welcome to the Roboflow community and happy building!!

[sbroberg]

Thanks, @Ford. I definitely get that blocks should be as atomic as possible; general good software engineering should follow the “do one thing” paradigm. I was asking this question because I was wonder if there were tools or techniques associated with Roboflow for doing what I was asking, and maybe the answer, based on what you said, is “no”. Let me be a little more explicit in the issue we faced.

I assume that Roboflow is meant to allow people who don’t want to get hung up on writing their own workflow infrastructure logic, which means a lot of the customers might be people who are not particularly adept at python programming - this certainly describes my son. So they’re going to make a lot of coding errors, even in simple blocks that are less than 40 lines long.

Here’s an example of the problem he faced. He was trying to get some basic info out of the Detections object, but because he was unfamiliar with the data structure, he didn’t know the precise command, so he was trying to grab an ‘x’ parameter out of the data element:

    x1 = pred.data['x']

This produced the error message:

Since the block referenced x in several other sections, without line numbers the only way for him to track down what was wrong was to comment out lines, then progressively uncomment them until he figured out what was causing the problem.

The immediate solution to this would be to just get better error reporting in the UI (unless this already exists, and you can help me out in showing how to make this happen).

But it also points to a larger issue in terms of integrating block code we want to maintain outside the context of the Roboflow app. Ideally, we would want all our custom python blocks to be checked into a code repository, along with a unit test for each block. But we need a way to run the block outside of roboflow for this to work.

My dream feature: In addition to having the “Test Block” button in the UI, have an “export test harness” button that would create a python script that could invoke the block in the normal python interpreter (along with the data context that is created the when you run the test in the UI, such as the specific input media and outputs of any upstream blocks). This script wouldn’t run the upstream modules, it would just set up the data environment (hardcoded into the script) from the output of the test that you just ran. It wouldn’t even need visibility to Roboflow libraries (other than definitions of data structures).

This exported test harness could be checked into the repository where you work on the python block and used in CI pipelines (so python blocks could be tested for validity prior to integration into a workflow), as well as allowing you to code your block in something like PyCharm which would let you use all the useful python IDE features, including command completion and data inspection to more easily work with the context data in the workflow.

[Ford]

@sbroberg This is a fantastic feature request, thank you so much for sharing this with the community!! I will certainly share this request with the team.

@Automatez What do you think? Would love to hear your opinion.

[Automatez]

I agree - the custom python block is really handy but can be tricky to implement. I certainly like what @sbroberg is suggesting for an option. In the past I have used the Deploy button on a workflow without a custom python block and then just wrote out the rest of the Python script in whatever form I like and stayed in Python to execute all. But I would prefer to be able to have it all live in the Workflow.

My biggest difficulties have been around data input to and output from the Python block. I could imagine the implementation most friendly to non-coders being a “mini-Deploy” from the Custom Python Block that generates something like below. Which I believe would also be handy for someone to paste into an LLM for some coding ideas dropped directly in place.

Sample Mini-Deploy result for Custom Python Block:

# Preceding blocks in your Workflow "compare_item_counts" from Workspace "my_images-p4ttm":
#
# Object Detection Model
#         V
# Property Definition (Extract Property > Class Name))
#         V
# output = 
# list from "property_definition": 
list_of_classNames = [
      "car",
      "car",
      "car",
      "traffic light",
      "traffic light",
      "car"
    ]

# Custom Python Block
#    Note: after completing below, copy all of this def into Custom Python Block in your workflow
def run(self, list_of_classNames) -> Previous_Block_Result: 

    #enter your python code here

    return { 

        'outputName': {'test','test2'} 

    } 

#some code here that makes sure the output is in the right format

# (workflow continues)
#            V
#   Output Block

[sbroberg]

@Automatez, yes, basically this. Ideally, you’d want your code block to be in a separate file so that it can be maintained separately from the harness (which you may also have multiple instances of to test different use cases).