Full Text Available
Note: Clicking the button above will open the full text document at the original institutional repository in a new window.
Visualisation and manipulation of 3D patient-specific bone geometry using augmented reality
Computer-mediated reality technologies have the potential to improve the imageguided surgery (IGS) workflow; specifically, pre-surgical planning, intra-operative guidance, post-surgical assessment, and rehabilitation. Augmented reality (AR), a form of computer-mediated reality, uses an electronic di...
Saved in:
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Language: | English |
| Published: |
Division of Biomedical Engineering
2020
|
| Subjects: | |
| Tags: |
No Tags, Be the first to tag this record!
|
| _version_ | 1867613201745575936 |
|---|---|
| access_status_str | Open Access |
| author | Coertze, Johannes A |
| author2 | Mutsvangwa, Tinashe |
| author_browse | Coertze, Johannes A Mutsvangwa, Tinashe |
| author_facet | Mutsvangwa, Tinashe Coertze, Johannes A |
| author_sort | Coertze, Johannes A |
| collection | Thesis |
| description | Computer-mediated reality technologies have the potential to improve the imageguided surgery (IGS) workflow; specifically, pre-surgical planning, intra-operative guidance, post-surgical assessment, and rehabilitation. Augmented reality (AR), a form of computer-mediated reality, uses an electronic display or projection module to add a hologram in the user's field of view (FOV). For intra-operative guidance, AR could aid in reducing the cognitive overload experienced by clinicians due to integrating multi-modal imaging data from several sources while performing the intervention on the patient. Three AR HMD systems have been developed to explore the capabilities of the Microsoft HoloLens as an AR HMD to be used in developing an AR HMD medical system. The three AR HMD systems required different software and hardware system architectures, however, each of the AR HMD system's software applications has been developed in Unity combined with the Mixed Reality Toolkit (MRTK). Each of the AR HMD systems implemented different registration techniques to localize the virtual object in the real-world coordinate system. The registration techniques were user calibration alignment to identified anatomical landmarks, fiducial marker tracking, and markerless tracking. For user calibration with anatomical landmarks, the MRTK was manipulated to allow alignment of the virtual object. For fiducial registration, the Vuforia Software Development Kit (SDK) was added to assess the alignment and spatial anchoring of the virtual object as specified. Finally, the Leap Motion Controller (LMC) and Leap's Orion SDK was used for exploring markerless tracking. The AR HMD systems developed enabled performance assessments, and alignment errors were identified during trials of the three systems. Most notably the location drift of the 3D virtual object in the spatial space due to the clinician moving around the registered location. This project entailed preliminary development towards the AR HMD medical system to create an in-vivo view of 3D patient-specific bone geometries as a hologram in the clinician's FOV. |
| format | Thesis |
| id | oai:open.uct.ac.za:11427/32211 |
| institution | University of Cape Town (South Africa) |
| language | eng |
| last_indexed | 2026-06-10T12:32:21.936Z |
| license_str | Not specified — see source repository |
| provenance_str_mv | Harvested via OAI-PMH from UCTD — University of Cape Town Open Access Repository |
| publishDate | 2020 |
| publishDateRange | 2020 |
| publishDateSort | 2020 |
| publisher | Division of Biomedical Engineering |
| publisherStr | Division of Biomedical Engineering |
| record_format | dspace |
| source_str | UCTD — University of Cape Town Open Access Repository |
| spelling | oai:open.uct.ac.za:11427/32211 Visualisation and manipulation of 3D patient-specific bone geometry using augmented reality Coertze, Johannes A Mutsvangwa, Tinashe Douglas, Tania Bracio, Boris R Biomedical Engineering Computer-mediated reality technologies have the potential to improve the imageguided surgery (IGS) workflow; specifically, pre-surgical planning, intra-operative guidance, post-surgical assessment, and rehabilitation. Augmented reality (AR), a form of computer-mediated reality, uses an electronic display or projection module to add a hologram in the user's field of view (FOV). For intra-operative guidance, AR could aid in reducing the cognitive overload experienced by clinicians due to integrating multi-modal imaging data from several sources while performing the intervention on the patient. Three AR HMD systems have been developed to explore the capabilities of the Microsoft HoloLens as an AR HMD to be used in developing an AR HMD medical system. The three AR HMD systems required different software and hardware system architectures, however, each of the AR HMD system's software applications has been developed in Unity combined with the Mixed Reality Toolkit (MRTK). Each of the AR HMD systems implemented different registration techniques to localize the virtual object in the real-world coordinate system. The registration techniques were user calibration alignment to identified anatomical landmarks, fiducial marker tracking, and markerless tracking. For user calibration with anatomical landmarks, the MRTK was manipulated to allow alignment of the virtual object. For fiducial registration, the Vuforia Software Development Kit (SDK) was added to assess the alignment and spatial anchoring of the virtual object as specified. Finally, the Leap Motion Controller (LMC) and Leap's Orion SDK was used for exploring markerless tracking. The AR HMD systems developed enabled performance assessments, and alignment errors were identified during trials of the three systems. Most notably the location drift of the 3D virtual object in the spatial space due to the clinician moving around the registered location. This project entailed preliminary development towards the AR HMD medical system to create an in-vivo view of 3D patient-specific bone geometries as a hologram in the clinician's FOV. 2020-09-10T08:36:44Z 2020-09-10T08:36:44Z 2020 2020-09-10T08:36:22Z Master Thesis Masters MSc http://hdl.handle.net/11427/32211 eng application/pdf Division of Biomedical Engineering Faculty of Health Sciences |
| spellingShingle | Biomedical Engineering Coertze, Johannes A Visualisation and manipulation of 3D patient-specific bone geometry using augmented reality |
| thesis_degree_str | Master's |
| title | Visualisation and manipulation of 3D patient-specific bone geometry using augmented reality |
| title_full | Visualisation and manipulation of 3D patient-specific bone geometry using augmented reality |
| title_fullStr | Visualisation and manipulation of 3D patient-specific bone geometry using augmented reality |
| title_full_unstemmed | Visualisation and manipulation of 3D patient-specific bone geometry using augmented reality |
| title_short | Visualisation and manipulation of 3D patient-specific bone geometry using augmented reality |
| title_sort | visualisation and manipulation of 3d patient specific bone geometry using augmented reality |
| topic | Biomedical Engineering |
| url | http://hdl.handle.net/11427/32211 |
| work_keys_str_mv | AT coertzejohannesa visualisationandmanipulationof3dpatientspecificbonegeometryusingaugmentedreality |