Abstract: With the continuous development of smart cities, defect detection in sewer pipes has become increasingly crucial. However, current generic object detection methods are primarily tailored to recognition scenarios featuring substantial differences among objects, failing to effectively address the challenges present in defect detection within sewer pipes scenario, characterized by easily confusable defect categories and minimal disparities in defect levels. In light of this, this paper first explores the limitations of generic detection methods in the task of sewer pipes defect detection. Beginning with enhancing the fine-grained classification performance of models, a unified joint learning algorithm under the method of multiscale fine-grained enhancement is proposed, aiming to simultaneously enhance the defect categories classification and defect level classification capabilities of the model. Extensive experiments conducted on two self-constructed datasets, Sewer-Complete and Sewer-Part, validate the effectiveness and generalization ability of the proposed method. Comparative experiments and visual analysis against multiple existing detection methods further validate the superiority of this approach.