Flower extraction: Basic project viability and approach

Community Help
1

Hi everyone, I am posting with a very fundamental question. I am looking through the sort of naive lens of a biologist, and I am hoping for feedback on the feasibility of my proposed project through roboflow and on the specific approach I should be taking. I don’t know if this is the exact proper forum for this kind of question but seemed like the obvious place to start! I understand that resources exist for answering this, but a bunch of it is over my head at the moment… without a proper background in computer vision I am a bit overwhelmed by the breadth of information here. Plus, it’s been difficult for me to tell what pre-trained options already exist and would be applicable for my purpose.

Here’s my goal: to be able to input an image that contains flowers (e.g. here: https://inaturalist-open-data.s3.amazonaws.com/photos/221500346/original.jpg) and for a pipeline to return the pixels correspond specifically to the flowers. For now, at least, all of the flowers are from closely related plants and so there wouldn’t be much diversity in shape/size/color. I would like for this to be able to scale up to tens of thousands of images. I have plenty of sample images to choose from, I could pre-screen them for flower presence (e.g. with an LLM), and I can handle upstream/downstream work in python, etc.

Am I right that this is a segmentation-specific problem? Is manually training a small dataset the most sensible path forward? Should I be annotating my dataset with polygons specifically, rather than bounding boxes? Are there customization options I should have on my radar or is there a specific out-of-the-box model I should be training?

Anything advice helps – thanks so much!!!
Patrick

  • Project Type: Segmentation (?)
  • Operating System & Browser: macOS, Google Chrome
2

Hey Patrick! I think you’re on the right track with wanting instance segmentation - that will help get you pixel-level identification. I’ll throw out some thoughts, but if this doesn’t get you started or you have more questions, I would also suggest including in future conversation here what you plan to do with the pixel-level representations as that could trigger other ideas/suggestions from the community.

Roboflow has resources for really robust processes around segmentation, but it tends to be useful to start with the quick version of things and then go more complex if you need it.

I would recommend reviewing this blog post. It explains the tools and process to annotate a bunch of images (that is, you upload a bunch of sample images and it helps you specifically outline flowers in images that are similar to what you need a model to guess at later). After you have done your annotation, you train the model to understand those flowers and then in theory you can run your tens of thousands of images through the model and it will identify the flowers by pixel now that it knows what to look for.

Some notes on this - I would start with at least 100 of your own images loaded to Roboflow and then annotated. More is always better, but you can go back later and add more if you need increase accuracy. The 100 should at least let you get through the steps to see if you want to invest some more time. (As an FYI, the effort here will be hours, not days or weeks.) Also, there are a few outdated screenshots in these instructions, but once you upload images and annotate, just follow the prompts to train a model and you should be good.

Finally, to kind of glimpse into the future of what you will have built once you are done, you can look at this existing flower segmentation dataset and model on Roboflow Universe. It will give you an idea of what you are building. And as you’ll see, these public models have a nice feature where you can drop in your own image and see if it works on the image. (I did try your flower example and you can as well. What I found is your flower image is too different from the existing flower dataset and so it does not “see” your flower. Although if you drop the confidence slider to almost zero it starts to find parts of flowers.)

And yes, if this all works well for you and the model is identifying your flowers well, there are all sorts of options in Roboflow for rolling this out at a larger scale without needing to drop in one image at a time. :slight_smile:

Bonus info if you’re brave: this video from Roboflow walks through the latest SAM 2.1 model inference process via a Python Notebook. If the above process within Roboflow does not get as accurate as you had hoped, but it IS headed in the right direction, fine-tuning SAM 2.1 could be your next step. And the nice thing is, all the work you did above to annotate your specific flower images can all be used in this more advanced model training. But certainly check back first with your progress after the first step, as sometimes the answer is to just get more images, not necessarily start switching models.

Best of luck!

3

Hi, thanks so much for this thorough response! Fundamentally it’s just hugely helpful just to know that I’m potentially in the right place for accomplishing this. And on top of that, these resources seem like a great starting point. I really appreciate the effort you put into getting me started!

1 Like