Full Text Available
Note: Clicking the button above will open the full text document at the original institutional repository in a new window.
Semantic and visual robotic navigation in human-centered environments
Indermun, S. 2025. Semantic and visual robotic navigation in human-centered environments. Unpublished doctoral dissertation. Stellenbosch: Stellenbosch University [online]. Available: https://scholar.sun.ac.za/items/4a3c4077-ecbc-44de-aa64-ac12e4f81b61
Saved in:
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Published: |
Stellenbosch : Stellenbosch University
2025
|
| Subjects: | |
| Tags: |
No Tags, Be the first to tag this record!
|
| _version_ | 1867613863303708672 |
|---|---|
| access_status_str | Open Access |
| author | Indermun, Shival |
| author2 | Schreve, Kristiaan |
| author_browse | Indermun, Shival Schreve, Kristiaan |
| author_facet | Schreve, Kristiaan Indermun, Shival |
| author_sort | Indermun, Shival |
| collection | Thesis |
| dc_rights_str_mv | Stellenbosch University |
| description | Indermun, S. 2025. Semantic and visual robotic navigation in human-centered environments. Unpublished doctoral dissertation. Stellenbosch: Stellenbosch University [online]. Available: https://scholar.sun.ac.za/items/4a3c4077-ecbc-44de-aa64-ac12e4f81b61 |
| format | Thesis |
| id | oai:scholar.sun.ac.za:10019.1/132477 |
| institution | Stellenbosch University (South Africa) |
| last_indexed | 2026-06-10T12:42:53.367Z |
| license_str | Other — see source repository |
| provenance_str_mv | Harvested via OAI-PMH from SUNScholar — Stellenbosch University Repository |
| publishDate | 2025 |
| publishDateRange | 2025 |
| publishDateSort | 2025 |
| publisher | Stellenbosch : Stellenbosch University |
| publisherStr | Stellenbosch : Stellenbosch University |
| record_format | dspace |
| source_str | SUNScholar — Stellenbosch University Repository |
| spelling | oai:scholar.sun.ac.za:10019.1/132477 Semantic and visual robotic navigation in human-centered environments Indermun, Shival Schreve, Kristiaan Ratsch, Matthias Stellenbosch University. Faculty of Engineering. Dept. of Mechanical and Mechatronic Engineering. Human-robot interaction Mobile robots -- Automatic control Wireless localization UCTD Indermun, S. 2025. Semantic and visual robotic navigation in human-centered environments. Unpublished doctoral dissertation. Stellenbosch: Stellenbosch University [online]. Available: https://scholar.sun.ac.za/items/4a3c4077-ecbc-44de-aa64-ac12e4f81b61 Thesis (PhD)--Stellenbosch University, 2025. ENGLISH ABSTRACT: As hardware and machine learning tools advance, the application of Autonomous Mobile Robots (AMRs) in public areas with significant human presence is concurrently expanding. This work addresses the challenges of dynamic environments in visual robotic mapping and navigation. However, a key assumption is made, suggesting that the dynamics in the environment are primarily due to human interaction. Many modern visual SLAM algorithms rely on tracking features within scenes to determine camera trajectory. Despite the robustness of methods like ORBSLAM3, their effectiveness is limited in dynamic scenarios due to the assumption of a static environment, resulting in poor tracking and data association, compromised pose estimation accuracy, and system failure. This research proposes a novel approach, Human-Object Interaction Detection (HOID), utilizing spatial reasoning to mitigate the influence of dynamic entities. By evaluating the intersecting area between the bounding boxes of a person and an object, the method identifies dynamic objects. Tested by extending the ORBSLAM3 RGB-D SLAM algorithm, the method filters out ORB features associated with dynamic objects. Evaluations on dynamic sequences from the TUM RGB-D dataset demonstrated significant performance enhancements over ORBSLAM3. Furthermore, the results remained competitive with other state-of-the-art methods, highlighting the simplicity and effectiveness of the approach. Addressing tracking and pose estimation, this method enhances robust visual SLAM. However, public spaces with human activity also impact human safety and comfort, crucial in AMR deployment. While modern approaches often rely on complex tools such as deep reinforcement learning and behavioural methods, it is essential to focus on foundational navigation tools like costmaps for significant improvements and adaptability in risk-aware navigation. To address safety concerns, a method was developed to create a risk-aware 2D costmap using RGB-D sensors and convolutional neural networks (CNN). This innovative approach segments objects into safety classes, generates semantic occupancy grids, and integrates them into a comprehensive costmap. Evaluated using both simulated and real-world datasets, the method demonstrated significant improvements in safety margins and enhanced semantic richness of costmaps. Additionally, integrating the HOID approach with the semantic, risk-aware occupancy grid created a human-object interaction layer. This novel costmap layer dynamically adjusts safety margins based on human-object interactions, improving efficiency and safety in environments where robots operate alongside humans and dynamic objects. Similarly, evaluations demonstrated significant improvements in detecting and handling dynamic entities, effectively generating risk-specific safety zones. The effectiveness of the HOID approach was demonstrated through path planning simulations. The results showed that while the original costmap provided efficient navigation with direct paths, it compromised safety by navigating in close proximity to obstacles. In contrast, the HOID method proved to be a robust approach, offering dynamically adjustable inflation zones based on risk classifications and interactions. This made it particularly suitable for dynamic, human-centered environments, effectively balancing safety, adaptability, and efficiency. AFRIKAANSE OPSOMMING: Soos hardeware en masjienleerinstrumente vorder, word die toepassing van outonome mobiele robotte (Autonomous Mobile Robots - AMRs) in openbare gebiede met ’n aansienlike menslike teenwoordigheid gelyktydig uitgebrei. Hierdie werk spreek die uitdagings van dinamiese omgewings in visuele robotkartering en navigasie aan. ’n Sleutelaanname word egter gemaak, naamlik dat die dinamika in die omgewing hoofsaaklik te wyte is aan menslike interaksie. Baie moderne visuele SLAM-algoritmes maak staat op opsporingskenmerke binne tonele om die kamerabaan te bepaal. Ten spyte van die robuustheid van metodes soos ORBSLAM3, is hul doeltreffendheid beperk in dinamiese scenario’s as gevolg van die aanname van ’n statiese omgewing, wat lei tot swak opsporing en dataassosiasie, foutiewe posisie en oriëntasie akkuraatheid en stelselmislukking. Hierdie navorsing stel ’n nuwe benadering, Mens-Voorwerp Interaksie Herkenning (Human-Object Interaction Detection - HOID) voor, wat ruimtelike redenering gebruik om die invloed van dinamiese entiteite te versag. Deur die oorvleueling van die grensrame van ’n persoon en ’n voorwerp te evalueer, identifiseer die metode dinamiese voorwerpe. Getoets deur die ORBSLAM3 RGBD SLAM-algoritme uit te brei, filter die metode ORB-kenmerke wat met dinamiese voorwerpe geassosieer word, uit. Evaluasies op dinamiese reekse van die TUM RGB-D-datastel het beduidende prestasieverbeterings oor ORBSLAM3 getoon. Verder het die resultate mededingend gebly met ander uitgelese metodes, wat die eenvoud en doeltreffendheid van die benadering beklemtoon. Met die aanspreek van opsporing en posisie- en oriëntasieskatting, verbeter hierdie metode robuuste visuele SLAM. Openbare ruimtes met menslike aktiwiteit beïnvloed egter ook menslike veiligheid en gemak, wat deurslaggewend is in AMR-ontplooiing. Terwyl moderne benaderings dikwels staatmaak op komplekse gereedskap soos diepversterkingsleer en gedragsmetodes, is dit noodsaaklik om op grondliggende navigasiehulpmiddels soos kostekaarte te focus vir aansienlike verbeterings en aanpasbaarheid in risikobewuste navigasie. Om veiligheidskwessies aan te spreek, is ’n metode ontwikkel om ’n risikobewuste 2D-kostekaart te skep met behulp van RGB-D-sensors en konvolusionêre neurale netwerke (CNN). Hierdie innoverende benadering segmenteer voorwerpe in veiligheidsklasse, genereer semantiese besettingsroosters, en integreer dit in ’n omvattende kostekaart. Geëvalueer deur gebruik te maak van beide gesimuleerde en werklike datastelle, het die metode beduidende verbeterings in veiligheidsmarges en verbeterde semantiese rykdom van kostekaarte getoon. Daarbenewens het die integrasie van die HOID-benadering met die semantiese, risikobewuste besettingsrooster ’n mens-voorwerp-interaksielaag geskep. Hierdie nuwe kostekaartlaag pas veiligheidsmarges dinamies aan gebaseer op mens-voorwerp-interaksies, wat doeltreffendheid en veiligheid verbeter in omgewings waar robotte saam met mense en dinamiese voorwerpe werk. Evaluasies het eweneens beduidende verbeterings getoon in die opsporing en hantering van dinamiese entiteite, wat effektief risiko-spesifieke veiligheidssones genereer. Die doeltreffendheid van die HOID-benadering is gedemonstreer deur padbeplanningsimulasies. Die resultate het getoon dat hoewel die oorspronklike kostekaart doeltreffende navigasie met direkte paaie verskaf het, dit veiligheid in die gedrang gebring het deur te naby hindernisse te navigeer. Daarteenoor blyk die HOID-metode ’n robuuste benadering te wees, wat dinamies verstelbare inflasiesones bied gebaseer op risikoklassifikasies en interaksies. Dit is dus veral geskik vir dinamiese, mensgesentreerde omgewings, wat veiligheid, aanpasbaarheid en doeltreffendheid effektief balanseer. Doctoral 2025-06-09T12:18:55Z 2025-06-09T12:18:55Z 2025-03 Thesis https://scholar.sun.ac.za/handle/10019.1/132477 Stellenbosch University xviii, 146 pages : illustrations application/pdf Stellenbosch : Stellenbosch University |
| spellingShingle | Human-robot interaction Mobile robots -- Automatic control Wireless localization UCTD Indermun, Shival Semantic and visual robotic navigation in human-centered environments |
| title | Semantic and visual robotic navigation in human-centered environments |
| title_full | Semantic and visual robotic navigation in human-centered environments |
| title_fullStr | Semantic and visual robotic navigation in human-centered environments |
| title_full_unstemmed | Semantic and visual robotic navigation in human-centered environments |
| title_short | Semantic and visual robotic navigation in human-centered environments |
| title_sort | semantic and visual robotic navigation in human centered environments |
| topic | Human-robot interaction Mobile robots -- Automatic control Wireless localization UCTD |
| url | https://scholar.sun.ac.za/handle/10019.1/132477 |
| work_keys_str_mv | AT indermunshival semanticandvisualroboticnavigationinhumancenteredenvironments |