Full Text Available
Note: Clicking the button above will open the full text document at the original institutional repository in a new window.
Investigating the evolution of modularity in neural networks
Thesis (MEng)--Stellenbosch University, 2020.
Saved in:
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Language: | English |
| Published: |
Stellenbosch : Stellenbosch University
2020
|
| Subjects: | |
| Tags: |
No Tags, Be the first to tag this record!
|
| _version_ | 1867613830510542848 |
|---|---|
| access_status_str | Open Access |
| author | Werle van der Merwe, Andreas |
| author2 | Van den Heever, David Jacobus |
| author_browse | Van den Heever, David Jacobus Werle van der Merwe, Andreas |
| author_facet | Van den Heever, David Jacobus Werle van der Merwe, Andreas |
| author_sort | Werle van der Merwe, Andreas |
| collection | Thesis |
| dc_rights_str_mv | Stellenbosch University |
| description | Thesis (MEng)--Stellenbosch University, 2020. |
| format | Thesis |
| id | oai:scholar.sun.ac.za:10019.1/108161 |
| institution | Stellenbosch University (South Africa) |
| language | English |
| last_indexed | 2026-06-10T12:42:21.587Z |
| license_str | Other — see source repository |
| provenance_str_mv | Harvested via OAI-PMH from SUNScholar — Stellenbosch University Repository |
| publishDate | 2020 |
| publishDateRange | 2020 |
| publishDateSort | 2020 |
| publisher | Stellenbosch : Stellenbosch University |
| publisherStr | Stellenbosch : Stellenbosch University |
| record_format | dspace |
| source_str | SUNScholar — Stellenbosch University Repository |
| spelling | oai:scholar.sun.ac.za:10019.1/108161 Investigating the evolution of modularity in neural networks Werle van der Merwe, Andreas Van den Heever, David Jacobus Du Plessis, Stefan Stellenbosch University. Faculty of Engineering. Dept. of Mechanical and Mechatronic Engineering. Evolutionary computation Artificial neural networks Human-computer interaction Neural modularity HyperNEAT UCTD Modular programming Neural networks (Computer Science) Thesis (MEng)--Stellenbosch University, 2020. ENGLISH ABSTRACT: Neural networks are not inherently interpretable as a direct consequence of their operating principle and the high dimensional opacity of their internal computations. The neural network interpretability problem is detrimental to reliability, meaningful human-AI interaction and the ethics of deployment. The problem can be approached from the perspective of neural modularity which frames a modular network as one that contains any number of disjoint subnetworks and identifies an interpretable modular network as one that groups its internal representations within such subnetworks in an explanative, task specific way. This study aims to investigate how neural modularity evolves and how it can benefit interpretability. HyperNEAT under connectivity constraints is the chosen neuroevolutionary method, and the following key points of research are investigated with respect to the evolution of neural modularity: general substrates, a variety of connection cost and novel input competition constraints, HyperNEAT modifications based on CPPN disjoints, and the interaction between lifetime and evolutionary learning with neuron nomination given the inclusion of a training phase. The results indicate that the connectivity constraints successfully promote the evolution of neural modularity across a variety of tasks on a general substrate and show that the novel input competition constraints are competitive with the established connection costs as a means of driving the evolution of neural modularity. The HyperNEAT modifications based on CPPN disjoints did not benefit the evolution of neural modularity. Investigating the interaction between lifetime and evolutionary learning with neuron nomination links greater concurrency between the processes that determine a network’s form and function with higher levels of evolved neural modularity for the connection cost constraints. The interpretability assessment shows that while the evolved networks’ interpretable qualities are task dependent, two connectivity constraints deliver statistically different functional module overlap distributions. This study highlights new possibilities for future research and contributes to the knowledge basis on evolving neural modularity by showing that input competition constraints are competitive with connection cost constraints, by examining how the interaction between lifetime and evolutionary learning influences the evolution of neural modularity as well as looking at the interpretable qualities of the evolved networks. AFRIKAANSE OPSOMMING: Neurale netwerke is nie inherent interpreteerbaar nie as 'n gevolg van hulle werkbeginsel en die hoë dimensionele ondeursigtigheid van hulle interne berekeninge. Die neurale netwerk interpreteerbaarheidsprobleem is nadelig tot betroubaarheid, betekenisvolle mens-AI-interaksie en die etiek van ontplooiing. Die probleem kan benader word vanuit die perspektief van neurale modulariteit. 'n Modulêre netwerk bevat 'n aantal afwykende subnetwerke. 'n Interpreteerbare modulêre netwerk is 'n neurale netwerk met interne voorstellings wat binne sulke afwykende subnetwerke op 'n verklarende en taakspesifieke manier groepeer is. Hierdie studie ondersoek hoe neurale modulariteit ontwikkel en hoe dit interpreteerbaarheid kan bevoordeel. HyperNEAT met konnektiwiteitsbeperkings is die gekose neuro-evolusionêre metode en die volgende navorsing sleutelpunte word ondersoek met betrekking tot die evolusie van neurale modulariteit: (1) algemene raamwerke, (2) 'n verskeidenheid van verbindingskoste en nuwe insetkompetisiebeperkings, (3) HyperNEAT-modifikasies gebaseer op CPPNafwykings, en (4) die interaksie tussen lewenslange en evolusionêre leer met neuronenominasie gegewe die insluiting van 'n opleidingsfase. Die resultate dui daarop dat die konnektiwiteitsbeperkings die evolusie van neurale modulariteit oor 'n verskeidenheid take op 'n algemene raamwerk suksesvol bevorder en toon dat die nuwe insetkompetisiebeperkings mededingend is met die vasgestelde verbindingskoste as 'n manier om die evolusie van neurale modulariteit te dryf. Die HyperNEAT-modifikasies gebaseer op CPPN-afwykings het nie die ontwikkeling van neurale modulariteit bevoordeel nie. Die ondersoek van interaksie tussen leeftyd en evolusionêre leer met neuronenominasie dui aan dat hoër neurale modulariteits vlakke met verbindingskoste veroorsaak word 'n groter mate van samewerking tussen die prosesse wat die vorm en funksie van 'n netwerk bepaal. Die interpretasie evaluering toon aan dat alhoewel die ontwikkelde netwerke se interpreteerbare eienskappe taakafhanklik is, die funksie-oorvleuel van twee konnektiwiteitsbeperkings oorvleuel is statisties verskillend. Hierdie studie belig nuwe moontlikhede vir toekomstige navorsing en dra by tot die kennisbasis oor die ontwikkeling van neurale modulariteit deur aan te toon dat insetkompetisiebeperkings mededingend is met verbindingskoste beperkings, deur te ondersoek hoe die interaksie tussen leeftyd en evolusionêre leer die ontwikkeling van neurale modulariteit beïnvloed, sowel as die ondersoek van die interpreteerbare eienskappe van die ontwikkelde netwerke. 2020-02-26T11:01:40Z 2020-04-28T12:22:49Z 2020-02-26T11:01:40Z 2020-04-28T12:22:49Z 2020-03 Thesis http://hdl.handle.net/10019.1/108161 en Stellenbosch University Masters xiv, 142 leaves : illustrations (some color) application/pdf Stellenbosch : Stellenbosch University |
| spellingShingle | Evolutionary computation Artificial neural networks Human-computer interaction Neural modularity HyperNEAT UCTD Modular programming Neural networks (Computer Science) Werle van der Merwe, Andreas Investigating the evolution of modularity in neural networks |
| title | Investigating the evolution of modularity in neural networks |
| title_full | Investigating the evolution of modularity in neural networks |
| title_fullStr | Investigating the evolution of modularity in neural networks |
| title_full_unstemmed | Investigating the evolution of modularity in neural networks |
| title_short | Investigating the evolution of modularity in neural networks |
| title_sort | investigating the evolution of modularity in neural networks |
| topic | Evolutionary computation Artificial neural networks Human-computer interaction Neural modularity HyperNEAT UCTD Modular programming Neural networks (Computer Science) |
| url | http://hdl.handle.net/10019.1/108161 |
| work_keys_str_mv | AT werlevandermerweandreas investigatingtheevolutionofmodularityinneuralnetworks |