Full Text Available
Note: Clicking the button above will open the full text document at the original institutional repository in a new window.
Retention Length and Memory Capacity of Recurrent Neural Networks
Dissertation (MSc (Computer Science))--University of Pretoria, 2020.
Saved in:
| Other Authors: | |
|---|---|
| Format: | Thesis |
| Language: | English |
| Published: |
University of Pretoria
2021
|
| Subjects: | |
| Tags: |
No Tags, Be the first to tag this record!
|
| _version_ | 1867613556871004160 |
|---|---|
| access_status_str | Open Access |
| author2 | Cleghorn, Christopher W. |
| author_browse | Cleghorn, Christopher W. |
| author_facet | Cleghorn, Christopher W. |
| collection | Thesis |
| dc_rights_str_mv | © 2019 University of Pretoria. All rights reserved. The copyright in this work vests in the University of Pretoria. No part of this work may be reproduced or transmitted in any form or by any means, without the prior written permission of the University of Pretoria. |
| description | Dissertation (MSc (Computer Science))--University of Pretoria, 2020. |
| format | Thesis |
| id | oai:repository.up.ac.za:2263/78061 |
| institution | University of Pretoria (South Africa) |
| language | English |
| last_indexed | 2026-06-10T12:38:02.052Z |
| license_str | Other — see source repository |
| provenance_str_mv | Harvested via OAI-PMH from UPSpace — University of Pretoria Institutional Repository |
| publishDate | 2021 |
| publishDateRange | 2021 |
| publishDateSort | 2021 |
| publisher | University of Pretoria |
| publisherStr | University of Pretoria |
| record_format | dspace |
| source_str | UPSpace — University of Pretoria Institutional Repository |
| spelling | oai:repository.up.ac.za:2263/78061 Retention Length and Memory Capacity of Recurrent Neural Networks Cleghorn, Christopher W. u12022404@tuks.co.za Pretorius, Abraham Daniel Recurrent Neural Networks Time Series Memory Capacity Memory Retention Temporal Series UCTD Engineering, built environment and information technology theses SDG-08 Engineering, built environment and information technology theses SDG-09 Dissertation (MSc (Computer Science))--University of Pretoria, 2020. Recurrent Neural Networks (RNNs) are variants of Neural Networks that are able to learn temporal relationships between sequences presented to the neural network. RNNs are often employed to learn underlying relationships in time series and sequential data. This dissertation examines the extent of RNN’s memory retention and how it is influenced by different activation functions, network structures and recurrent network types. To investigate memory retention, three approaches (and variants thereof) are used. First the number of patterns each network is able to retain is measured. Thereafter the length of retention is investigated. Lastly the previous experiments are combined to measure the retention of patterns over time. During each investigation, the effect of using different activation functions and network structures are considered to determine the configurations’ effect on memory retention. The dissertation concludes that memory retention of a network is not necessarily improved when adding more parameters to a network. Activation functions have a large effect on the performance of RNNs when retaining patterns, especially temporal patterns. Deeper network structures have the trade-off of less memory retention per parameter in favour of the ability to model more complex relationships. bs2026 Computer Science MSc (Computer Science) Unrestricted SDG-08: Decent work and economic growth SDG-09: Industry, innovation and infrastructure 2021-01-20T07:32:24Z 2021-01-20T07:32:24Z 2021-04-01 2020 Dissertation * S2021 http://hdl.handle.net/2263/78061 en © 2019 University of Pretoria. All rights reserved. The copyright in this work vests in the University of Pretoria. No part of this work may be reproduced or transmitted in any form or by any means, without the prior written permission of the University of Pretoria. application/pdf University of Pretoria |
| spellingShingle | Recurrent Neural Networks Time Series Memory Capacity Memory Retention Temporal Series UCTD Engineering, built environment and information technology theses SDG-08 Engineering, built environment and information technology theses SDG-09 Retention Length and Memory Capacity of Recurrent Neural Networks |
| title | Retention Length and Memory Capacity of Recurrent Neural Networks |
| title_full | Retention Length and Memory Capacity of Recurrent Neural Networks |
| title_fullStr | Retention Length and Memory Capacity of Recurrent Neural Networks |
| title_full_unstemmed | Retention Length and Memory Capacity of Recurrent Neural Networks |
| title_short | Retention Length and Memory Capacity of Recurrent Neural Networks |
| title_sort | retention length and memory capacity of recurrent neural networks |
| topic | Recurrent Neural Networks Time Series Memory Capacity Memory Retention Temporal Series UCTD Engineering, built environment and information technology theses SDG-08 Engineering, built environment and information technology theses SDG-09 |
| url | http://hdl.handle.net/2263/78061 |