How to correctly fine tune a model?

Hi guys. I have a project but I want to share with you a doubt I have. I annotated my dataset through Roboflow, and this dataset has only 2 classes. Now I have to create a system that utilize as object detector YOLOv8 (pre-trained on the COCO dataset), and as classifier some network such as ResNet or some other network. My question is, do I need to retrain YOLO too or I just train the network for the classifier?