Full Text Available
Note: Clicking the button above will open the full text document at the original institutional repository in a new window.
LABCAT: locally adaptive Bayesian optimization using principal-component-aligned trust regions
Visser, E. 2025. LABCAT: Locally adaptive Bayesian optimization using principal-component-aligned trust regions. Unpublished doctoral dissertation. Stellenbosch: Stellenbosch University [online]. Available: https://scholar.sun.ac.za/items/18500d0c-a92b-49f8-80bd-37b67d81f7f2
Saved in:
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Language: | English |
| Published: |
Stellenbosch : Stellenbosch University
2025
|
| Subjects: | |
| Tags: |
No Tags, Be the first to tag this record!
|
| _version_ | 1867613768279654400 |
|---|---|
| access_status_str | Open Access |
| author | Visser, Emile |
| author2 | Schoeman, J. C. |
| author_browse | Schoeman, J. C. Visser, Emile |
| author_facet | Schoeman, J. C. Visser, Emile |
| author_sort | Visser, Emile |
| collection | Thesis |
| dc_rights_str_mv | Stellenbosch University |
| description | Visser, E. 2025. LABCAT: Locally adaptive Bayesian optimization using principal-component-aligned trust regions. Unpublished doctoral dissertation. Stellenbosch: Stellenbosch University [online]. Available: https://scholar.sun.ac.za/items/18500d0c-a92b-49f8-80bd-37b67d81f7f2 |
| format | Thesis |
| id | oai:scholar.sun.ac.za:10019.1/132326 |
| institution | Stellenbosch University (South Africa) |
| language | English |
| last_indexed | 2026-06-10T12:41:23.238Z |
| license_str | Other — see source repository |
| provenance_str_mv | Harvested via OAI-PMH from SUNScholar — Stellenbosch University Repository |
| publishDate | 2025 |
| publishDateRange | 2025 |
| publishDateSort | 2025 |
| publisher | Stellenbosch : Stellenbosch University |
| publisherStr | Stellenbosch : Stellenbosch University |
| record_format | dspace |
| source_str | SUNScholar — Stellenbosch University Repository |
| spelling | oai:scholar.sun.ac.za:10019.1/132326 LABCAT: locally adaptive Bayesian optimization using principal-component-aligned trust regions Visser, Emile Schoeman, J. C. Van Daalen, Corne E. Stellenbosch University. Faculty of Engineering. Dept. of Electrical and Electronic Engineering. Bayesian statistical decision theory Mathematical optimization -- Statistical methods Principal components analysis Gaussian process UCTD Visser, E. 2025. LABCAT: Locally adaptive Bayesian optimization using principal-component-aligned trust regions. Unpublished doctoral dissertation. Stellenbosch: Stellenbosch University [online]. Available: https://scholar.sun.ac.za/items/18500d0c-a92b-49f8-80bd-37b67d81f7f2 Thesis (PhD)--Stellenbosch University, 2025. ENGLISH ABSTRACT: Bayesian optimization (BO) is a popular and well-studied technique for the optimization of black-box objective functions, using an iteratively updated Gaussian process (GP) surrogate model to approximate the objective function and inform the selection of the next point to evaluate from the objective function. BO is especially renowned for a high degree of sample efficiency, which allows it to find good solutions with relatively few objective function evaluations. However, this method has several notable shortcomings which hinder its use across a broad range of optimization problems and objective functions. Specifically, BO can experience significant computational slowdown as the number of algorithm iterations increases, which poses challenges for real-time applications. Additionally, BO may struggle with non-stationary and ill-conditioned objective functions due to the reliance on a kernel-based GP surrogate model that often requires manual kernel engineering to model these objective functions adequately. Finally, a lack of theoretical guarantees and practical, numerical limitations mean that BO often exhibits poor convergence characteristics. Several algorithms have been proposed that incorporate local strategies into the BO framework to mitigate these limitations, such as trust regions or domain partitioning; however, none address all of them satisfactorily. To address these shortcomings, this dissertation presents the locally adaptive Bayesian optimization using principal-component-aligned trust regions (LABCAT) algorithm, which follows the example of other trust-region-based BO algorithms. These algorithms incorporate an iteratively resized region, known as a trust region, that is used to constrain the choice of the next sample point and, by extension, the region of the objective function being approximated by the surrogate model. Using a trust region relaxes the global focus of BO to a sequence of local optimization problems that are, ideally, easier to solve. The proposed LABCAT algorithm extends the trust-region-based BO framework through the addition of a novel rotation which aligns the trust region with the weighted principal components of the observed data and an adaptive rescaling strategy based on the lengthscales of the local GP surrogate model with automatic relevance determination. These two extensions allow better adaptation of the trust region to ill-conditioning or non-stationarity and, when combined with approximative hyperparameter estimation and observation discarding schemes, allow for improved computational and convergence performance characteristics. Through extensive numerical experiments using a set of synthetic test functions and the well-known COCO benchmarking software, the LABCAT algorithm is shown to outperform several state-of-the-art BO and other black-box optimization algorithms. AFRIKAANSE OPSOMMING: Bayesiese optimering (BO) is ’n gewilde en goed-bestudeerde tegniek vir die optimering van swartboksdoelfunksies deur gebruik te maak van ’n iteratief opgedateerde Gaussiese proses (GP) surrogaatmodel om die doelfunksie te benader en die keuse van die volgende punt in die evaluering van die doelfunksie te beïnvloed. BO is veral bekend vir ’n hoë mate van evaluasiedoeltreffendheid, wat leen tot die vind van goeie oplossings met relatief min doelfunksie evaluerings. Hierdie metode het wel verskeie terkortkominge wat die gebruik daarvan belemmer vir ’n wye groep van optimeringsprobleme en -doelfunksies. Eerstens kan BO aansienlike berekeningsvertraging ervaar soos die aantal iterasies van die algoritme vermeerder, wat uitdagings bied vir intydse toepassings. Bonop kan BO sukkel met doelfunksies wat nie stasionêr is nie of wat sleg gekondisioneer is as gevolg van die afhanklikheid van ’n kernfunksiegebaseerde GP wat dikwels handgedrewe kernfunksieverstelling benodig om hierdie tipes doelfunksies tot ’n voldoende mate te modelleer. Uiteindelik veroorsaak ’n gebrek aan teoretiese waarborge en praktiese, numeriese beperkings dat BO dikwels swak konvergensie-eienskappe vertoon. Alhoewel verskeie algoritmes voorgestel is wat plaaslike strategieë kombineer met die BO raamwerk om hierdie terkortkominge aan te spreek, soos vertroue streke of domeinondervedeling, spreek geen een van hulle volledig die terkortkominge aan nie. Om die terkortkominge aan te spreek stel hierdie proefskrif die “locally adaptive Bayesian optimization using principalcomponent-aligned trust regions (LABCAT)” algoritme voor, wat die voorbeeld volg van ander vetroustreeksgebaseerde algoritmes. Hierdie algoritmes inkorporeer ’n iteratief aangepasde area, wat bekend staan as ’n vertroue streek, om die keuse van die volgende evaluasiepunt te beperk en, deur uitbreiding, die area van die doelfunksie wat benader word deur die surrogaatmodel te beperk. Die gebruik van ’n vertroue streek verslap die globale fokus van standard BO na ’n reeks van lokale optimeringsprobleme wat ideaal makliker is om op te los. Die voorgestelde LABCAT algoritme brei die vetroustreeksgebaseerde BO raamwerk uit deur middel van ’n nuwe rotasie wat die vertroue streek belyn met die geweegde hoofkomponente van die waargenome data en ’n aanpasbare herskaleeringstrategie gebaseer op die lengteskaal van die lokale GP surrogaatmodel met automatiese relevansiebepaling. Hierdie twee uitbeidings veroorsak beter aanpassing van die vertroue streek net die nie-stasionariteit of swak konditionering en, gekombineer met benaderde hiperparameterskatting- en waarnemingsverwyderingskema, laat verbeterde berekenings- en konvergensie-eienskappe toe. Deur gebruik te maak van omvattende numeriese experimente met ’n stel sintetiese doelfunksies en die bekende COCO maatstafsagteware, vertoon die LABCAT algoritme beter resultate as verskeie moderne Bayesiese- en ander swartboksoptimeringsalgoritmes. Doctoral 2025-06-03T14:19:58Z 2025-06-03T14:19:58Z 2025-03 Thesis https://scholar.sun.ac.za/handle/10019.1/132326 en Stellenbosch University xiv, 116 pages : illustrations application/pdf Stellenbosch : Stellenbosch University |
| spellingShingle | Bayesian statistical decision theory Mathematical optimization -- Statistical methods Principal components analysis Gaussian process UCTD Visser, Emile LABCAT: locally adaptive Bayesian optimization using principal-component-aligned trust regions |
| title | LABCAT: locally adaptive Bayesian optimization using principal-component-aligned trust regions |
| title_full | LABCAT: locally adaptive Bayesian optimization using principal-component-aligned trust regions |
| title_fullStr | LABCAT: locally adaptive Bayesian optimization using principal-component-aligned trust regions |
| title_full_unstemmed | LABCAT: locally adaptive Bayesian optimization using principal-component-aligned trust regions |
| title_short | LABCAT: locally adaptive Bayesian optimization using principal-component-aligned trust regions |
| title_sort | labcat locally adaptive bayesian optimization using principal component aligned trust regions |
| topic | Bayesian statistical decision theory Mathematical optimization -- Statistical methods Principal components analysis Gaussian process UCTD |
| url | https://scholar.sun.ac.za/handle/10019.1/132326 |
| work_keys_str_mv | AT visseremile labcatlocallyadaptivebayesianoptimizationusingprincipalcomponentalignedtrustregions |