Full Text Available
Note: Clicking the button above will open the full text document at the original institutional repository in a new window.
Optimising the Optimiser: Meta NeuroEvolution for Artificial Intelligence Problems
Since reinforcement learning algorithms have to fully solve a task in order to evaluate a set of hyperparameter values, conventional hyperparameter tuning methods can be highly sample inefficient and computationally expensive. Many widely used reinforcement learning architectures originate from scie...
Saved in:
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Language: | English |
| Published: |
Department of Statistical Sciences
2022
|
| Subjects: | |
| Tags: |
No Tags, Be the first to tag this record!
|
| _version_ | 1867614411930206208 |
|---|---|
| access_status_str | Open Access |
| author | Hayes, Max Nieuwoudt |
| author2 | Bassett, Bruce |
| author_browse | Bassett, Bruce Hayes, Max Nieuwoudt |
| author_facet | Bassett, Bruce Hayes, Max Nieuwoudt |
| author_sort | Hayes, Max Nieuwoudt |
| collection | Thesis |
| description | Since reinforcement learning algorithms have to fully solve a task in order to evaluate a set of hyperparameter values, conventional hyperparameter tuning methods can be highly sample inefficient and computationally expensive. Many widely used reinforcement learning architectures originate from scientific papers which include optimal hyperparameter values in the publications themselves, but do not indicate how the hyperparameter values were found. To address the issues related to hyperparameter tuning, three different experiments were investigated. In the first two experiments, Bayesian Optimisation and random search are compared. In the third and final experiment, the hyperparameter values found in second experiment are used to solve a more difficult reinforcement learning task, effectively performing hyperparameter transfer learning (later referred to as meta-transfer learning). The results from experiment 1 showed that there are certain scenarios in which Bayesian Optimisation outperforms random search for hyperparameter tuning, while the results of experiment 2 show that as more hyperparameters are simultaneously tuned, Bayesian Optimisation consistently finds better hyperparameter values than random search. However, BO took more than twice the amount of time to find these hyperparameter values than random search. Results from the third and final experiment indicate that hyperparameter values learned while tuning hyperparameters for a relatively easy to solve reinforcement learning task (Task A), can be used to solve a more complex task (Task B). With the available computing power for this thesis, hyperparameter optimisation was possible on the tasks in experiment 1 and experiment 2. This was not possible on the task in experiment 3, due to limited computing resources and the increased complexity of the reinforcement learning task in experiment 3, making the transfer of hyperparameters from one task (Task A) to the more difficult task (Task B) highly beneficial for solving the more computationally expensive task. The purpose of this work is to explore the effectiveness of Bayesian Optimisation as a hyperparameter tuning algorithm on the reinforcement learning algorithm NEAT's hyperparemters. An additional goal of this work is the experimental use of hyperparameter value transfer between reinforcement learning tasks, referred to in this work as Meta-Transfer Learning. This is introduced and discussed in greater detail in the Introduction chapter. All code used for this work is available in the repository: • https://github.com/maaxnaax/MSc_code |
| format | Thesis |
| id | oai:open.uct.ac.za:11427/35595 |
| institution | University of Cape Town (South Africa) |
| language | eng |
| last_indexed | 2026-06-10T12:51:37.608Z |
| license_str | Not specified — see source repository |
| provenance_str_mv | Harvested via OAI-PMH from UCTD — University of Cape Town Open Access Repository |
| publishDate | 2022 |
| publishDateRange | 2022 |
| publishDateSort | 2022 |
| publisher | Department of Statistical Sciences |
| publisherStr | Department of Statistical Sciences |
| record_format | dspace |
| source_str | UCTD — University of Cape Town Open Access Repository |
| spelling | oai:open.uct.ac.za:11427/35595 Optimising the Optimiser: Meta NeuroEvolution for Artificial Intelligence Problems Hayes, Max Nieuwoudt Bassett, Bruce Clark, Allan Advance Analytics Since reinforcement learning algorithms have to fully solve a task in order to evaluate a set of hyperparameter values, conventional hyperparameter tuning methods can be highly sample inefficient and computationally expensive. Many widely used reinforcement learning architectures originate from scientific papers which include optimal hyperparameter values in the publications themselves, but do not indicate how the hyperparameter values were found. To address the issues related to hyperparameter tuning, three different experiments were investigated. In the first two experiments, Bayesian Optimisation and random search are compared. In the third and final experiment, the hyperparameter values found in second experiment are used to solve a more difficult reinforcement learning task, effectively performing hyperparameter transfer learning (later referred to as meta-transfer learning). The results from experiment 1 showed that there are certain scenarios in which Bayesian Optimisation outperforms random search for hyperparameter tuning, while the results of experiment 2 show that as more hyperparameters are simultaneously tuned, Bayesian Optimisation consistently finds better hyperparameter values than random search. However, BO took more than twice the amount of time to find these hyperparameter values than random search. Results from the third and final experiment indicate that hyperparameter values learned while tuning hyperparameters for a relatively easy to solve reinforcement learning task (Task A), can be used to solve a more complex task (Task B). With the available computing power for this thesis, hyperparameter optimisation was possible on the tasks in experiment 1 and experiment 2. This was not possible on the task in experiment 3, due to limited computing resources and the increased complexity of the reinforcement learning task in experiment 3, making the transfer of hyperparameters from one task (Task A) to the more difficult task (Task B) highly beneficial for solving the more computationally expensive task. The purpose of this work is to explore the effectiveness of Bayesian Optimisation as a hyperparameter tuning algorithm on the reinforcement learning algorithm NEAT's hyperparemters. An additional goal of this work is the experimental use of hyperparameter value transfer between reinforcement learning tasks, referred to in this work as Meta-Transfer Learning. This is introduced and discussed in greater detail in the Introduction chapter. All code used for this work is available in the repository: • https://github.com/maaxnaax/MSc_code 2022-01-27T07:03:23Z 2022-01-27T07:03:23Z 2021 2022-01-26T13:42:32Z Master Thesis Masters MSc http://hdl.handle.net/11427/35595 eng application/pdf Department of Statistical Sciences Faculty of Science |
| spellingShingle | Advance Analytics Hayes, Max Nieuwoudt Optimising the Optimiser: Meta NeuroEvolution for Artificial Intelligence Problems |
| thesis_degree_str | Master's |
| title | Optimising the Optimiser: Meta NeuroEvolution for Artificial Intelligence Problems |
| title_full | Optimising the Optimiser: Meta NeuroEvolution for Artificial Intelligence Problems |
| title_fullStr | Optimising the Optimiser: Meta NeuroEvolution for Artificial Intelligence Problems |
| title_full_unstemmed | Optimising the Optimiser: Meta NeuroEvolution for Artificial Intelligence Problems |
| title_short | Optimising the Optimiser: Meta NeuroEvolution for Artificial Intelligence Problems |
| title_sort | optimising the optimiser meta neuroevolution for artificial intelligence problems |
| topic | Advance Analytics |
| url | http://hdl.handle.net/11427/35595 |
| work_keys_str_mv | AT hayesmaxnieuwoudt optimisingtheoptimisermetaneuroevolutionforartificialintelligenceproblems |