Full Text Available
Note: Clicking the button above will open the full text document at the original institutional repository in a new window.
Deep adaptive anomaly detection using an active learning framework
Anomaly detection is the process of finding unusual events in a given dataset. Anomaly detection is often performed on datasets with a fixed set of predefined features. As a result of this, if the normal features bear a close resemblance to the anomalous features, most anomaly detection algorithms e...
Saved in:
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Language: | English |
| Published: |
Department of Mathematics and Applied Mathematics
2023
|
| Subjects: | |
| Tags: |
No Tags, Be the first to tag this record!
|
| _version_ | 1867613242928398336 |
|---|---|
| access_status_str | Open Access |
| author | Sekyi, Emmanuel |
| author2 | Bassett, Bruce |
| author_browse | Bassett, Bruce Sekyi, Emmanuel |
| author_facet | Bassett, Bruce Sekyi, Emmanuel |
| author_sort | Sekyi, Emmanuel |
| collection | Thesis |
| description | Anomaly detection is the process of finding unusual events in a given dataset. Anomaly detection is often performed on datasets with a fixed set of predefined features. As a result of this, if the normal features bear a close resemblance to the anomalous features, most anomaly detection algorithms exhibit poor performance. This work seeks to answer the question, can we deform these features so as to make the anomalies standout and hence improve the anomaly detection outcome? We employ a Deep Learning and an Active Learning framework to learn features for anomaly detection. In Active Learning, an Oracle (usually a domain expert) labels a small amount of data over a series of training rounds. The deep neural network is trained after each round to incorporate the feedback from the Oracle into the model. Results on the MNIST, CIFAR-10 and Galaxy Zoo datasets show that our algorithm, Ahunt, significantly outperforms other anomaly detection algorithms used on a fixed, static, set of features. Ahunt can therefore overcome a poor choice of features that happen to be suboptimal for detecting anomalies in the data, learning more appropriate features. We also explore the role of the loss function and Active Learning query strategy, showing these are important, especially when there is a significant variation in the anomalies. |
| format | Thesis |
| id | oai:open.uct.ac.za:11427/37767 |
| institution | University of Cape Town (South Africa) |
| language | eng |
| last_indexed | 2026-06-10T12:33:01.081Z |
| license_str | Not specified — see source repository |
| provenance_str_mv | Harvested via OAI-PMH from UCTD — University of Cape Town Open Access Repository |
| publishDate | 2023 |
| publishDateRange | 2023 |
| publishDateSort | 2023 |
| publisher | Department of Mathematics and Applied Mathematics |
| publisherStr | Department of Mathematics and Applied Mathematics |
| record_format | dspace |
| source_str | UCTD — University of Cape Town Open Access Repository |
| spelling | oai:open.uct.ac.za:11427/37767 Deep adaptive anomaly detection using an active learning framework Sekyi, Emmanuel Bassett, Bruce anomaly detection deep learning active learning Anomaly detection is the process of finding unusual events in a given dataset. Anomaly detection is often performed on datasets with a fixed set of predefined features. As a result of this, if the normal features bear a close resemblance to the anomalous features, most anomaly detection algorithms exhibit poor performance. This work seeks to answer the question, can we deform these features so as to make the anomalies standout and hence improve the anomaly detection outcome? We employ a Deep Learning and an Active Learning framework to learn features for anomaly detection. In Active Learning, an Oracle (usually a domain expert) labels a small amount of data over a series of training rounds. The deep neural network is trained after each round to incorporate the feedback from the Oracle into the model. Results on the MNIST, CIFAR-10 and Galaxy Zoo datasets show that our algorithm, Ahunt, significantly outperforms other anomaly detection algorithms used on a fixed, static, set of features. Ahunt can therefore overcome a poor choice of features that happen to be suboptimal for detecting anomalies in the data, learning more appropriate features. We also explore the role of the loss function and Active Learning query strategy, showing these are important, especially when there is a significant variation in the anomalies. 2023-04-18T18:47:35Z 2023-04-18T18:47:35Z 2022 2023-04-18T18:47:19Z Master Thesis Masters MSc http://hdl.handle.net/11427/37767 eng application/pdf Department of Mathematics and Applied Mathematics Faculty of Science |
| spellingShingle | anomaly detection deep learning active learning Sekyi, Emmanuel Deep adaptive anomaly detection using an active learning framework |
| thesis_degree_str | Master's |
| title | Deep adaptive anomaly detection using an active learning framework |
| title_full | Deep adaptive anomaly detection using an active learning framework |
| title_fullStr | Deep adaptive anomaly detection using an active learning framework |
| title_full_unstemmed | Deep adaptive anomaly detection using an active learning framework |
| title_short | Deep adaptive anomaly detection using an active learning framework |
| title_sort | deep adaptive anomaly detection using an active learning framework |
| topic | anomaly detection deep learning active learning |
| url | http://hdl.handle.net/11427/37767 |
| work_keys_str_mv | AT sekyiemmanuel deepadaptiveanomalydetectionusinganactivelearningframework |