Full Text Available
Note: Clicking the button above will open the full text document at the original institutional repository in a new window.
Using Siamese neural networks to identify individual animals
The ability to identify individual animals in ecology is essential for monitoring [Schneider et al., 2018]. It allows a researcher to determine whether an animal being observed is new to the researcher or has previously been observed. This in turn allows estimation of ecological metrics such as popu...
Saved in:
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Language: | English |
| Published: |
Department of Statistical Sciences
2023
|
| Subjects: | |
| Tags: |
No Tags, Be the first to tag this record!
|
| _version_ | 1867613299266289664 |
|---|---|
| access_status_str | Open Access |
| author | Madzingira, Tinotenda |
| author2 | Durbach, Ian |
| author_browse | Durbach, Ian Madzingira, Tinotenda |
| author_facet | Durbach, Ian Madzingira, Tinotenda |
| author_sort | Madzingira, Tinotenda |
| collection | Thesis |
| description | The ability to identify individual animals in ecology is essential for monitoring [Schneider et al., 2018]. It allows a researcher to determine whether an animal being observed is new to the researcher or has previously been observed. This in turn allows estimation of ecological metrics such as population density [Schneider et al., 2018]. Traditionally, this was done by capturing and physically tagging animals [Cross et al., 2014a]. Increasingly, animal observation is being conducted by photographic means creating the need to be able to identify individual animals from images [Weinstein, 2018]. The dissertation answers whether machine learning can determine if the animals in a pair of images come from the same individual animal. To answer this question, the animal of interest in each image must be (1) located and isolated and (2) compared to the other image in the pair. Mask-Regional Convolution Neural Networks (Mask - RCNN) [He et al., 2017] are used for the object detection and instance segmentation which answers (1). This is a modern deep learning approach which has been used for tasks such as identifying breast cancer tumors [Chiao et al., 2019] outside of ecology and to measure the size of whales [Gray et al., 2019b] in ecology. In addition to classifying the object and proposing a bounding box, Mask RCNN also proposes a mask for each object. The “mask” is a selection of pixels that belong to the object of interest which are highlighted in processed images. A ResNet-101 model with the Feature Pyramid Network (FPN) that has been trained on the MS Coco dataset [Lin et al., 2014] is used with v Abstract vi transfer learning. Finally, a Siamese Neural Network (SNN) is used to measure the similarity between the objects in each pair of images, which answers (2). A SNN is a pair of identical neural networks that share the same weights and whose outputs are connected to a distance computing function. SNNs are used to identify subtle differences in the feature space of its inputs. Each of the neural networks is given one of the images in the pair as an input and a distance measure is computed on their outputs which allows the inputs to be classified as similar or dissimilar based on a given threshold of the distance metric. The structure of the SNNs applied are inspired by the one proposed in [Dey et al., 2017] and the weights are trained separately on each dataset. The proposed approach is evaluated on two datasets: a set of approximately 25,000 images of 5,000 humpback whale individuals taken a cross a variety of locations, and a set of approximately 13,000 images of 300 individual harbor seals from 3 locations on the west coast of Scotland. The object detection models are evaluated using the intersection over union (IOU) approach while the SNNs are evaluated on F1 score, the harmonic mean of precision and recall. The proposed method is tested on unseen images for each dataset. The SNN models achieved F1 scores of 63.9%, 66.7% and 68.4% for the Humpback whale dataset, the right and the left fins of the bottlenose dolphins respectively. The object detection model achieved an intersection over union of 88.6% and 74.3% for the Humpback whale dataset and the bottlenose dolphin dataset respectively. Lastly the orientation model achieved an F1 score of 94.7%. All quoted results are evaluated on an unseen test dataset. |
| format | Thesis |
| id | oai:open.uct.ac.za:11427/37459 |
| institution | University of Cape Town (South Africa) |
| language | eng |
| last_indexed | 2026-06-10T12:33:55.830Z |
| license_str | Not specified — see source repository |
| provenance_str_mv | Harvested via OAI-PMH from UCTD — University of Cape Town Open Access Repository |
| publishDate | 2023 |
| publishDateRange | 2023 |
| publishDateSort | 2023 |
| publisher | Department of Statistical Sciences |
| publisherStr | Department of Statistical Sciences |
| record_format | dspace |
| source_str | UCTD — University of Cape Town Open Access Repository |
| spelling | oai:open.uct.ac.za:11427/37459 Using Siamese neural networks to identify individual animals Madzingira, Tinotenda Durbach, Ian statistical sciences The ability to identify individual animals in ecology is essential for monitoring [Schneider et al., 2018]. It allows a researcher to determine whether an animal being observed is new to the researcher or has previously been observed. This in turn allows estimation of ecological metrics such as population density [Schneider et al., 2018]. Traditionally, this was done by capturing and physically tagging animals [Cross et al., 2014a]. Increasingly, animal observation is being conducted by photographic means creating the need to be able to identify individual animals from images [Weinstein, 2018]. The dissertation answers whether machine learning can determine if the animals in a pair of images come from the same individual animal. To answer this question, the animal of interest in each image must be (1) located and isolated and (2) compared to the other image in the pair. Mask-Regional Convolution Neural Networks (Mask - RCNN) [He et al., 2017] are used for the object detection and instance segmentation which answers (1). This is a modern deep learning approach which has been used for tasks such as identifying breast cancer tumors [Chiao et al., 2019] outside of ecology and to measure the size of whales [Gray et al., 2019b] in ecology. In addition to classifying the object and proposing a bounding box, Mask RCNN also proposes a mask for each object. The “mask” is a selection of pixels that belong to the object of interest which are highlighted in processed images. A ResNet-101 model with the Feature Pyramid Network (FPN) that has been trained on the MS Coco dataset [Lin et al., 2014] is used with v Abstract vi transfer learning. Finally, a Siamese Neural Network (SNN) is used to measure the similarity between the objects in each pair of images, which answers (2). A SNN is a pair of identical neural networks that share the same weights and whose outputs are connected to a distance computing function. SNNs are used to identify subtle differences in the feature space of its inputs. Each of the neural networks is given one of the images in the pair as an input and a distance measure is computed on their outputs which allows the inputs to be classified as similar or dissimilar based on a given threshold of the distance metric. The structure of the SNNs applied are inspired by the one proposed in [Dey et al., 2017] and the weights are trained separately on each dataset. The proposed approach is evaluated on two datasets: a set of approximately 25,000 images of 5,000 humpback whale individuals taken a cross a variety of locations, and a set of approximately 13,000 images of 300 individual harbor seals from 3 locations on the west coast of Scotland. The object detection models are evaluated using the intersection over union (IOU) approach while the SNNs are evaluated on F1 score, the harmonic mean of precision and recall. The proposed method is tested on unseen images for each dataset. The SNN models achieved F1 scores of 63.9%, 66.7% and 68.4% for the Humpback whale dataset, the right and the left fins of the bottlenose dolphins respectively. The object detection model achieved an intersection over union of 88.6% and 74.3% for the Humpback whale dataset and the bottlenose dolphin dataset respectively. Lastly the orientation model achieved an F1 score of 94.7%. All quoted results are evaluated on an unseen test dataset. 2023-03-15T13:20:06Z 2023-03-15T13:20:06Z 2022 2023-03-15T13:19:43Z Master Thesis Masters MSc http://hdl.handle.net/11427/37459 eng application/pdf Department of Statistical Sciences Faculty of Science |
| spellingShingle | statistical sciences Madzingira, Tinotenda Using Siamese neural networks to identify individual animals |
| thesis_degree_str | Master's |
| title | Using Siamese neural networks to identify individual animals |
| title_full | Using Siamese neural networks to identify individual animals |
| title_fullStr | Using Siamese neural networks to identify individual animals |
| title_full_unstemmed | Using Siamese neural networks to identify individual animals |
| title_short | Using Siamese neural networks to identify individual animals |
| title_sort | using siamese neural networks to identify individual animals |
| topic | statistical sciences |
| url | http://hdl.handle.net/11427/37459 |
| work_keys_str_mv | AT madzingiratinotenda usingsiameseneuralnetworkstoidentifyindividualanimals |