Full Text Available
Note: Clicking the button above will open the full text document at the original institutional repository in a new window.
Calibrating models to data: a comparison of methods
Thesis (MSc)--Stellenbosch University, 2020.
Saved in:
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Language: | en_ZA |
| Published: |
Stellenbosch : Stellenbosch University
2020
|
| Subjects: | |
| Tags: |
No Tags, Be the first to tag this record!
|
| _version_ | 1867613889741455360 |
|---|---|
| access_status_str | Open Access |
| author | Suboi, Zenabu |
| author2 | Hazelbag, Marijn |
| author_browse | Hazelbag, Marijn Suboi, Zenabu |
| author_facet | Hazelbag, Marijn Suboi, Zenabu |
| author_sort | Suboi, Zenabu |
| collection | Thesis |
| dc_rights_str_mv | Stellenbosch University |
| description | Thesis (MSc)--Stellenbosch University, 2020. |
| format | Thesis |
| id | oai:scholar.sun.ac.za:10019.1/109473 |
| institution | Stellenbosch University (South Africa) |
| language | en_ZA |
| last_indexed | 2026-06-10T12:43:19.203Z |
| license_str | Other — see source repository |
| provenance_str_mv | Harvested via OAI-PMH from SUNScholar — Stellenbosch University Repository |
| publishDate | 2020 |
| publishDateRange | 2020 |
| publishDateSort | 2020 |
| publisher | Stellenbosch : Stellenbosch University |
| publisherStr | Stellenbosch : Stellenbosch University |
| record_format | dspace |
| source_str | SUNScholar — Stellenbosch University Repository |
| spelling | oai:scholar.sun.ac.za:10019.1/109473 Calibrating models to data: a comparison of methods Suboi, Zenabu Hazelbag, Marijn Delva, Wim Stellenbosch University. Faculty of Science. Dept. of Mathematical Sciences. Computer simulation -- Models Calibration -- Computer simulation Bayesian calibration -- Data processing Algorithms -- Data processing Bayesian statistical decision theory -- Mathematical models Calibration -- Mathematical models Calibration -- Data processing UCTD Thesis (MSc)--Stellenbosch University, 2020. ENGLISH ABSTRACT: Complex models are often fitted to data using simulation-based calibration, a com- putationally challenging process. Several calibration methods to improve computa- tional efficiency have been developed with no consensus on which methods perform best. We did a simulation study comparing the performance of 5 methods that differed in their Goodness-of-Fit (GOF) metrics and parameter search strategies. Posterior densities for two parameters of a simple Sucseptible-Infected-Recovered model were obtained for each calibration method under two scenarios. Scenario 1 (S1) allowed 60000 model runs and provided two target statistics, whereas scenario 2 (S2) allowed 75000 model runs and provided three target statistics. For both scenarios, we obtained reference posteriors against which we compare all other methods by running rejection ABC for 5.000.000 parameter combinations, retaining the 0.1% best. We as- sessed performance by applying a 2D-grid to all posterior densities and quantifying the percentage overlap with the reference posterior. In the basic calibration methods, Bayesian calibration (Bc) Sampling Importance Re- sampling (S1: 34.8%, S2: 39.8%) outperformed Rejection Approximate Bayesian Com- putation (ABC) (S1: 2.3%, S2: 1.8%). In the adaptive sampling methods, Bc Incremen- tal Mixture Importance Sampling (S1: 72.7%, S2: 85.5%) outperformed Abc Sequential Monte-Carlo (S1: 53.9%, S2: 72.9%) and Sequential ABC (S1: 21.6%, S2: 62.7%). Basic methods led to sub-optimal calibration results. Methods using the surrogate Likelihood as a GOF outperformed methods using a distance measure. Adaptive sam- pling methods were over five-thousand times more efficient compared to their basic counterparts and resulted in more accurate posterior distributions. When sufficient tar- get statistics were available, the adaptive sampling methods performed similarly. The results of this study suggest that the choice of adaptive sampling method can therefore be informed based on differences in requirements and researcher preference. AFRIKAANSE OPSOMMING: Simulasie gebaseerde kalibrasie, ‘n rekenaar-uitdagende proses, word dikwels ge- bruik om ingewikkelde modelle na data te pas. Verskeie metodes is al ontwikkel al- hoewel daar geen ooreenstemming is oor watter metodes die beste vaar nie. Die pas- gehalte van 5 metodes, verskillend in hul pasgehaltemaatstaf (GOF) statistieke en pa- rametersoek strategieë, word hier vergelyk met behulp van ‘n simulasie studie. Ons gebruik ‘n eenvoudige stogastiese Susceptible-Infected-Recovered model om posterior verdelings van die model se twee parameters te verkry, vir elk van die passingsmetodes, onder twee scenarios. Scenario 1 (S1) het 60000 modellopies toegelaat en twee teikensta- tistieke verskaf, terwyl scenario 2 (S2) 75000 modellopies toegelaat het en drie teikensta- tistieke verskaf het. Ons het verwysings posteriors verkry vir albei scenarios waarteen ons alle ander metodes kon vergelyk deur toepassing van verwerping Approximate- Bayesian-Computation (ABC) vir 5 000 000 parameter kombinasies. Die 0.1% parameter kombinasies met die beste passing, was behou. Ons het die prestasie beoordeel deur ’n 2D-rooster op alle posterior verdelings toe te pas en die persentasie oorvleueling met die verwysing posterior te kwantifiseer. In terme van die basiese kalibrasiemetodes het die Bayesiaanse kalibrasie metode (Bc) Sampling-Importance-Resampling (S1:34.8%, S2: 39.8%) beter gevaar as verwerping ABC (S1: 2.3%, S2: 1.8%). Uit die aanpassende steek- proefnemingmetodes, het Bc Incremental Mixture Importance Sampling (S1: 72.7%, S2: 85.5%) beter gevaar as beide ABC Sequential Monte-Carlo (S1: 53.9%, S2: 72.9%) en Sequential ABC (S1: 21.6%, S2:62.7%). Basiese metodes het gelei na sub-optimale kali- brasie resultate. Metodes wat gebruik maak van plaasvervanger aanneemlikheid as ‘n GOF verrig beter as metodes wat ‘n afstandsmaat gebruik. Aanpasbare steekproefne- mingmetodes was meer as vyf duisend keer meer doeltreffend in vergelyking met hul basiese eweknieë en het gelei tot nuttige posterior verdelings. In die geval waar ge- noeg teikenstatistieke beskikbaar was, het die aanpassende steekproefnemingmetodes soortgelyk gevaar. Die resultate van hierdie studie dui daarop dat die gebruik van aan- pasbare steekproefmetodes en/of ’n superrekenaar nodig mag wees om ingewikkelde modelle te kalibreer. 2020-11-25T09:21:57Z 2021-02-01T07:56:36Z 2020-11-25T09:21:57Z 2021-02-01T07:56:36Z 2020-12 Thesis http://hdl.handle.net/10019.1/109473 en_ZA Stellenbosch University xi, 44 pages : illustrations (some color) application/pdf Stellenbosch : Stellenbosch University |
| spellingShingle | Computer simulation -- Models Calibration -- Computer simulation Bayesian calibration -- Data processing Algorithms -- Data processing Bayesian statistical decision theory -- Mathematical models Calibration -- Mathematical models Calibration -- Data processing UCTD Suboi, Zenabu Calibrating models to data: a comparison of methods |
| title | Calibrating models to data: a comparison of methods |
| title_full | Calibrating models to data: a comparison of methods |
| title_fullStr | Calibrating models to data: a comparison of methods |
| title_full_unstemmed | Calibrating models to data: a comparison of methods |
| title_short | Calibrating models to data: a comparison of methods |
| title_sort | calibrating models to data a comparison of methods |
| topic | Computer simulation -- Models Calibration -- Computer simulation Bayesian calibration -- Data processing Algorithms -- Data processing Bayesian statistical decision theory -- Mathematical models Calibration -- Mathematical models Calibration -- Data processing UCTD |
| url | http://hdl.handle.net/10019.1/109473 |
| work_keys_str_mv | AT suboizenabu calibratingmodelstodataacomparisonofmethods |