Full Text Available
Note: Clicking the button above will open the full text document at the original institutional repository in a new window.
Entropy-based attention for inference in probabilistic graphical models
Thesis (MEng)--Stellenbosch University, 2025.
Saved in:
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Language: | English |
| Published: |
Stellenbosch : Stellenbosch University
2025
|
| Subjects: | |
| Tags: |
No Tags, Be the first to tag this record!
|
| _version_ | 1867614025690382336 |
|---|---|
| access_status_str | Open Access |
| author | Bergesen, Alexander |
| author2 | Van Daalen, C. E. (Corné) |
| author_browse | Bergesen, Alexander Van Daalen, C. E. (Corné) |
| author_facet | Van Daalen, C. E. (Corné) Bergesen, Alexander |
| author_sort | Bergesen, Alexander |
| collection | Thesis |
| dc_rights_str_mv | Stellenbosch University |
| description | Thesis (MEng)--Stellenbosch University, 2025. |
| format | Thesis |
| id | oai:scholar.sun.ac.za:10019.1/134519 |
| institution | Stellenbosch University (South Africa) |
| language | English |
| last_indexed | 2026-06-10T12:45:28.762Z |
| license_str | Other — see source repository |
| provenance_str_mv | Harvested via OAI-PMH from SUNScholar — Stellenbosch University Repository |
| publishDate | 2025 |
| publishDateRange | 2025 |
| publishDateSort | 2025 |
| publisher | Stellenbosch : Stellenbosch University |
| publisherStr | Stellenbosch : Stellenbosch University |
| record_format | dspace |
| source_str | SUNScholar — Stellenbosch University Repository |
| spelling | oai:scholar.sun.ac.za:10019.1/134519 Entropy-based attention for inference in probabilistic graphical models Bergesen, Alexander Van Daalen, C. E. (Corné) Stellenbosch University. Faculty of Engineering. Dept. of Electrical & Electronic Engineering. Probabilistic graphical models Pattern recognition systems Entropy Thesis (MEng)--Stellenbosch University, 2025. Bergesen, A. 2025. Entropy-Based Attention for Inference in Probabilistic Graphical Models. Unpublished masters thesis. Stellenbosch: Stellenbosch University [online]. Available: https://scholar.sun.ac.za/items/b9a84719-2d2b-4b96-9dc3-20826af6ef0b ENGLISH ABSTRACT: Probabilistic graphical models (PGMs) are powerful tools for modelling complex systems with inherent uncertainty, followed by probabilistic reasoning or inference. Traditional inference methods, such as loopy belief propagation (LBP) and loopy belief update (LBU), perform approximate inference in PGMs with fixed, fully-connected loopy graph structures. However, these approaches face significant challenges in large, densely connected graphs, including poor scalability, high computational and memory demands, and convergence issues, which restrict the practical use of PGMs in large-scale applications. To overcome these limitations, this thesis introduces a novel entropy-based attention mechanism for inference in PGMs, integrating information-theoretic principles with dynamic weighting inspired by attention mechanisms from deep learning. The proposed attentive inference approach enables dynamic construction and adaptive control of cluster graphs throughout the inference process, representing a significant departure from the static graph structures and message passing schedules of traditional methods. By prioritising the most uncertain and informative regions of the cluster graph, the entropy-based attention mechanism focuses computational resources where they are most needed, resulting in substantial improvements in computational efficiency, scalability, and robustness, while maintaining high approximation accuracy. Empirical validation using simulated motion segmentation datasets demonstrates that attentive inference achieves dramatic reductions in computational time and memory usage compared to traditional methods, along with reliable convergence. The results show execution time improvements of up to 150 times and scalability up to PGMs 50 times larger than the maximum size manageable by traditional methods, with only a minor decrease in accuracy relative to the exact posterior beliefs. This work establishes a principled framework for integrating information-theoretic metrics and guided inference strategies into probabilistic inference, laying the groundwork for future research into adaptive inference mechanisms in PGMs and advancing probabilistic modelling and reasoning across a wide range of engineering and scientific applications. AFRIKAANSE OPSOMMING: Geen opsomming beskikbaar. Masters 2025-12-12T06:10:58Z 2025-12-12T06:10:58Z 2025-12 Thesis https://scholar.sun.ac.za/handle/10019.1/134519 en Stellenbosch University xi, 88 pages : illustration application/pdf Stellenbosch : Stellenbosch University |
| spellingShingle | Probabilistic graphical models Pattern recognition systems Entropy Bergesen, Alexander Entropy-based attention for inference in probabilistic graphical models |
| title | Entropy-based attention for inference in probabilistic graphical models |
| title_full | Entropy-based attention for inference in probabilistic graphical models |
| title_fullStr | Entropy-based attention for inference in probabilistic graphical models |
| title_full_unstemmed | Entropy-based attention for inference in probabilistic graphical models |
| title_short | Entropy-based attention for inference in probabilistic graphical models |
| title_sort | entropy based attention for inference in probabilistic graphical models |
| topic | Probabilistic graphical models Pattern recognition systems Entropy |
| url | https://scholar.sun.ac.za/handle/10019.1/134519 |
| work_keys_str_mv | AT bergesenalexander entropybasedattentionforinferenceinprobabilisticgraphicalmodels |