Full Text Available
Note: Clicking the button above will open the full text document at the original institutional repository in a new window.
Analysis of catastrophic interference with application to spline neural architectures
Dissertation (MSc(Computer Science))--University of Pretoria,2024
Saved in:
| Other Authors: | |
|---|---|
| Format: | Thesis |
| Language: | English |
| Published: |
University of Pretoria
2024
|
| Subjects: | |
| Tags: |
No Tags, Be the first to tag this record!
|
| _version_ | 1867613501977001984 |
|---|---|
| access_status_str | Open Access |
| author2 | Bosman, Anna Sergeevna |
| author_browse | Bosman, Anna Sergeevna |
| author_facet | Bosman, Anna Sergeevna |
| collection | Thesis |
| dc_rights_str_mv | © 2023 University of Pretoria. All rights reserved. The copyright in this work vests in the University of Pretoria. No part of this work may be reproduced or transmitted in any form or by any means, without the prior written permission of the University of Pretoria. |
| description | Dissertation (MSc(Computer Science))--University of Pretoria,2024 |
| format | Thesis |
| id | oai:repository.up.ac.za:2263/95024 |
| institution | University of Pretoria (South Africa) |
| language | English |
| last_indexed | 2026-06-10T12:37:09.691Z |
| license_str | Other — see source repository |
| provenance_str_mv | Harvested via OAI-PMH from UPSpace — University of Pretoria Institutional Repository |
| publishDate | 2024 |
| publishDateRange | 2024 |
| publishDateSort | 2024 |
| publisher | University of Pretoria |
| publisherStr | University of Pretoria |
| record_format | dspace |
| source_str | UPSpace — University of Pretoria Institutional Repository |
| spelling | oai:repository.up.ac.za:2263/95024 Analysis of catastrophic interference with application to spline neural architectures Bosman, Anna Sergeevna heinrich.vandeventer@outlook.com Van Deventer, Heinrich Pieter UCTD Machine learning Continual learning Catastrophic forgetting Catastrophic interference Overlapping representation Sparse distributed representation Regression Spline Artificial neural network (ANN) Universal function approximation Dissertation (MSc(Computer Science))--University of Pretoria,2024 Continual learning is the sequential learning of different tasks by a machine learning model. Continual learning is known to be hindered by catastrophic interference or forgetting, i.e. rapid unlearning of earlier learned tasks when new tasks are learned. Despite their practical success, artificial neural networks (ANNs) are prone to catastrophic interference. This study analyses how gradient descent and overlapping representations between distant input points lead to distal interference and catastrophic interference. Distal interference refers to the phenomenon where training a model on a subset of the domain leads to non-local changes on other subsets of the domain. This study shows that uniformly trainable models without distal interference must be exponentially large. A novel antisymmetric bounded exponential layer B-spline ANN architecture named ABEL-Spline is proposed that can approximate any continuous function, is uniformly trainable, has polynomial computational complexity, and provides some guarantees for distal interference. Experiments are presented to demonstrate the theoretical properties of ABEL-Splines. ABEL-Splines are also evaluated on benchmark regression problems. It is concluded that the weaker distal interference guarantees in ABEL-Splines are insufficient for model-only continual learning. It is conjectured that continual learning with polynomial complexity models requires augmentation of the training data or algorithm. Computing resources provided by the South African Centre for High-Performance Computing (CHPC). Supported by the National Research Foundation (NRF) of South Africa Thuthuka Grant Number 13819413/TTK210316590115. Computer Science MSc (Computer Science) Unrestricted Faculty of Engineering, Built Environment and Information Technology SDG-09: Industry, innovation and infrastructure 2024-03-01T10:56:26Z 2024-03-01T10:56:26Z 2024-05-13 2024-02-14 Dissertation * A2024 http://hdl.handle.net/2263/95024 https://doi.org/10.25403/UPresearchdata.25260349 en © 2023 University of Pretoria. All rights reserved. The copyright in this work vests in the University of Pretoria. No part of this work may be reproduced or transmitted in any form or by any means, without the prior written permission of the University of Pretoria. application/pdf University of Pretoria |
| spellingShingle | UCTD Machine learning Continual learning Catastrophic forgetting Catastrophic interference Overlapping representation Sparse distributed representation Regression Spline Artificial neural network (ANN) Universal function approximation Analysis of catastrophic interference with application to spline neural architectures |
| title | Analysis of catastrophic interference with application to spline neural architectures |
| title_full | Analysis of catastrophic interference with application to spline neural architectures |
| title_fullStr | Analysis of catastrophic interference with application to spline neural architectures |
| title_full_unstemmed | Analysis of catastrophic interference with application to spline neural architectures |
| title_short | Analysis of catastrophic interference with application to spline neural architectures |
| title_sort | analysis of catastrophic interference with application to spline neural architectures |
| topic | UCTD Machine learning Continual learning Catastrophic forgetting Catastrophic interference Overlapping representation Sparse distributed representation Regression Spline Artificial neural network (ANN) Universal function approximation |
| url | http://hdl.handle.net/2263/95024 https://doi.org/10.25403/UPresearchdata.25260349 |