Setting up a video-based underwater fauna detection workflow - a few questions before I commit

🤝 Community Help
1

Hi all — I’m working on a marine monitoring project that analyses underwater video from fixed cameras, and I’m evaluating whether Roboflow fits before scaling up. I’ve tried sales for the commercial/privacy side and only got auto-replies pointing me here, so I’ll ask the technical parts of the community and chase sales separately for the rest.

The setup. Several fixed underwater cameras, each recording short clips (30 s) on a schedule, over a long deployment — so a large archive of many thousands of clips. Conditions are hard: turbidity, backscatter, low light, some night footage, biofouling, so a good fraction is partly unusable. Footage currently lives in Google Drive.

What I’m trying to build. A two-stage workflow:

  1. a detection model that finds mobile fauna in coarse groups (fish, cephalopods, crustaceans, echinoderms), meant to be reused across sites;
  2. a species classifier that runs on those detections, with a region-specific species list — and ideally able to fall back to genus/family or an “unidentified” class when a confident species call isn’t possible.

Downstream analysis I do myself in R; from the platform I need annotation, training, inference across the whole archive, and clean export of the raw detections.

Questions I’m hoping the community can help with:

  1. Video annotation with tracking — can I annotate on video with persistent track IDs and interpolation between keyframes, rather than labelling scattered extracted frames? A per-individual track across frames is essential for my metric. How does this work in practice in Roboflow?

  2. Two-stage (detector → classifier) — does anyone run a detection model feeding a separate classifier on the crops? Is that supported natively (e.g. Workflows), or do people chain two projects manually? Any examples appreciated.

  3. Detection export — can I export raw per-detection outputs (per clip and frame: class, confidence, bounding box, track ID) as CSV/JSON? I need the detections themselves, not just aggregate counts.

  4. Google Drive at scale — has anyone pulled a large video archive (hundreds of GB) from Google Drive into Roboflow? Native integration, or manual upload? What worked at volume?

  5. Model-assisted labelling — how well does pre-labelling speed things up on difficult (low-visibility) footage, and do corrections feed back into training?

Any pointers, gotchas, or “we do exactly this” experiences would be hugely helpful. Thanks!
Marc

2

Hey Marc,

this sounds like a super interesting project and a great fit.

On video annotation with tracking: native video annotation with persistent track IDs is something we’re actively working on, so it’s coming up. Today labeling happens on frames, so the fastest path right now is to sample frames and auto-label them (e.g. with SAM 3) to bootstrap your detector. Model-assisted labeling helps a lot here, even on tricky low-visibility footage: you correct the predictions, the corrections become training data, and each round your own model gets better at pre-labeling the next batch.

The two-stage setup sounds like a good idea, especially in a case like yours where the classes are hard to differentiate. Detecting the coarse group first and then classifying the crop makes perfect sense, and it’s a perfect use case for Workflows and natively supported there: you chain your detection model, a dynamic crop, and a classification model on the crops in one Workflow, no manual glue between two projects needed.

For getting the archive in and the detections out: there’s no native Google Drive connector today. We have an S3 bucket sync and are working on more ingestion methods (possibly Google Drive in the future), so for now you’d move the clips over via CLI, API, or browser upload, all of which hold up fine at volume. Once they’re in, running your Workflow across the whole archive is exactly what Batch Processing is for: it spins up the infrastructure, runs over your videos, and hands you the raw per-detection outputs as JSON. Docs: Batch Processing | Roboflow Docs

If you want, shoot me a quick email at balthasar @roboflow.com so we can stay in touch on the project, and happy to jump on a call if you need help getting set up. Thanks a lot!

Balthasar