Resumen de Nephrops norvegicus burrows detection and classification from underwater videos using Deep Learning techniques
Atif Naseer
Problems faced by marine scientists during the assessment of Nephrops norvegicus species during underwater TV surveys have been addressed in this thesis. One of the main contributions of the work has been the study of the behavior of deep learning algorithms on the complex underwater dataset.
Currently, the Nephrops data are collected through the UWTV surveys and are reviewed manually by trained experts. Burrows systems are quantified following the protocol established by ICES.
Our first contribution is to develop the dataset for the deep learning models. No such dataset exists that someone can use to validate the results. After many revisions, the current work selected a few videos for annotation (the videos are selected with Marine experts based on the Nephrops burrows densities). The Marine expert validates each annotation before adding it to the dataset. After validating each annotation, a curated dataset is used for training and testing the model.
Different types of deep learning-based models have been finetuned and applied to the created dataset. The work proposed five different neural networks: MobileNet, Inception, ResNet50, ResNet101, and YOLOv3. All the models are trained and tested with the different combinations of datasets. A complete methodology is proposed for automatically detecting Nephrops burrows. The automatic detection algorithms could replace the human review of data, with the promise of better accuracy, coverage of more significant areas and higher consistency in the assessment.
Deep learning algorithms performed very well in identifying the burrows. Generic CNN-based object detectors still face challenges in underwater object detection. These challenges include image blurring, texture distortion, color shift, and scale variation, which result in low precision and recall rates. This thesis contributes by developing a Novel Detection Refinement Technique for Accurate Identification of Nephrops norvegicus burrows. The proposed technique is based on each detection's spatial temporal value. When integrated with any detector, the proposed method consistently increased the performance. The performance was calculated using mAP.
Another contribution lies in the tracking and counting burrows. Multiple OpenCV tracking algorithms are applied to that task, but due to three significant challenges, these tracking algorithms fail to track the Nephrops burrow. The first challenge is the camera's movement. The second challenge is the characteristics and size of burrows that are not fixed. The third challenge is the angle/opening of the burrow. The traditional object-tracking mechanism is not very effective. We proposed the tracking and counting of burrows using the spatial-temporal values of each burrow. The unique burrows are counted using the intersection values of detected burrows in consecutive frames.
From an experimental point of view, our contribution lies in comparing burrows detection with different models, the deep analytics and application of detection refinement algorithm by calculating the precision, recall and F1 score. The proposed tracking algorithm is also compared with the OpenCV tracking algorithms. All these experiments were performed for the different combinations of datasets and different levels of parameters. Results show that our approach has better results regarding burrows detections, refinements, tracking and counting of burrows.
