Full Text Available
Note: Clicking the button above will open the full text document at the original institutional repository in a new window.
Developing a tool for eliciting users moral theories for automated moral agents
In recent work, Rautenbach and Keet have developed a model of a system, which they name Genet, that allows the user to choose which moral theory their automated moral agent will follow. What remains unclear, however, is how the users will make this choice, given that most of them will not have the v...
Saved in:
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Language: | English |
| Published: |
Department of Computer Science
2025
|
| Subjects: | |
| Tags: |
No Tags, Be the first to tag this record!
|
| _version_ | 1867613185119354881 |
|---|---|
| access_status_str | Open Access |
| author | Seakgwa, Kyle |
| author2 | Keet, Catharina |
| author_browse | Keet, Catharina Seakgwa, Kyle |
| author_facet | Keet, Catharina Seakgwa, Kyle |
| author_sort | Seakgwa, Kyle |
| collection | Thesis |
| description | In recent work, Rautenbach and Keet have developed a model of a system, which they name Genet, that allows the user to choose which moral theory their automated moral agent will follow. What remains unclear, however, is how the users will make this choice, given that most of them will not have the vocabulary to classify themselves in the moral philosophical terms used by Genet. This issue is what this thesis is meant to address. This was done by building three high fidelity prototypes and then conducting online user evaluations of them. Each of these prototypes implemented an algorithm that was designed based on the elicitation approach of one of three fields: cognitive science, human computer interaction and knowledge engineering. Each of these aimed to computationally determine a user's preferred moral theory, by availing itself of a human-in-the-loop component enabled by discipline specific elicitation stimuli and rules to classify the user. These prototypes were then evaluated from a usability perspective using the System Usability Scale (SUS), and from an accuracy perspective, to determine which is most validly able to elicit users' moral preferences in the form required by Genet. This latter evaluation was done using validation measures based on existing approaches to validation in moral psychology. It was observed that all the prototypes performed equally well in terms of usability, with each having an acceptable SUS score. However, each of the prototypes also performed equally inaccurately in terms of the validity of the moral theory categorizations made. While this evaluation was carried out with only a small sample size (n=20) and thus has limited generalizability, as the first study to compare and computationally implement different moral theory elicitation approaches, the present study contributes to evidence for (or at least fails to falsify) problems with the project of making the design of Automated Moral Agents dependent on elicitation of a user's one preferred moral theory. A positive claim that the data collected here does support is that, at least for some potential users, even computational elicitation tools that use empirically validated measures of moral theory preferences (like those from cognitive science) do not allow one to predict the moral judgements they will make. |
| format | Thesis |
| id | oai:open.uct.ac.za:11427/41289 |
| institution | University of Cape Town (South Africa) |
| language | eng |
| last_indexed | 2026-06-10T12:32:07.214Z |
| license_str | Not specified — see source repository |
| provenance_str_mv | Harvested via OAI-PMH from UCTD — University of Cape Town Open Access Repository |
| publishDate | 2025 |
| publishDateRange | 2025 |
| publishDateSort | 2025 |
| publisher | Department of Computer Science |
| publisherStr | Department of Computer Science |
| record_format | dspace |
| source_str | UCTD — University of Cape Town Open Access Repository |
| spelling | oai:open.uct.ac.za:11427/41289 Developing a tool for eliciting users moral theories for automated moral agents Seakgwa, Kyle Keet, Catharina Information Technology In recent work, Rautenbach and Keet have developed a model of a system, which they name Genet, that allows the user to choose which moral theory their automated moral agent will follow. What remains unclear, however, is how the users will make this choice, given that most of them will not have the vocabulary to classify themselves in the moral philosophical terms used by Genet. This issue is what this thesis is meant to address. This was done by building three high fidelity prototypes and then conducting online user evaluations of them. Each of these prototypes implemented an algorithm that was designed based on the elicitation approach of one of three fields: cognitive science, human computer interaction and knowledge engineering. Each of these aimed to computationally determine a user's preferred moral theory, by availing itself of a human-in-the-loop component enabled by discipline specific elicitation stimuli and rules to classify the user. These prototypes were then evaluated from a usability perspective using the System Usability Scale (SUS), and from an accuracy perspective, to determine which is most validly able to elicit users' moral preferences in the form required by Genet. This latter evaluation was done using validation measures based on existing approaches to validation in moral psychology. It was observed that all the prototypes performed equally well in terms of usability, with each having an acceptable SUS score. However, each of the prototypes also performed equally inaccurately in terms of the validity of the moral theory categorizations made. While this evaluation was carried out with only a small sample size (n=20) and thus has limited generalizability, as the first study to compare and computationally implement different moral theory elicitation approaches, the present study contributes to evidence for (or at least fails to falsify) problems with the project of making the design of Automated Moral Agents dependent on elicitation of a user's one preferred moral theory. A positive claim that the data collected here does support is that, at least for some potential users, even computational elicitation tools that use empirically validated measures of moral theory preferences (like those from cognitive science) do not allow one to predict the moral judgements they will make. 2025-03-27T13:20:53Z 2025-03-27T13:20:53Z 2024 2025-03-27T13:16:47Z Thesis / Dissertation Masters MPhil http://hdl.handle.net/11427/41289 eng application/pdf Department of Computer Science Faculty of Science University of Cape town |
| spellingShingle | Information Technology Seakgwa, Kyle Developing a tool for eliciting users moral theories for automated moral agents |
| thesis_degree_str | Master's |
| title | Developing a tool for eliciting users moral theories for automated moral agents |
| title_full | Developing a tool for eliciting users moral theories for automated moral agents |
| title_fullStr | Developing a tool for eliciting users moral theories for automated moral agents |
| title_full_unstemmed | Developing a tool for eliciting users moral theories for automated moral agents |
| title_short | Developing a tool for eliciting users moral theories for automated moral agents |
| title_sort | developing a tool for eliciting users moral theories for automated moral agents |
| topic | Information Technology |
| url | http://hdl.handle.net/11427/41289 |
| work_keys_str_mv | AT seakgwakyle developingatoolforelicitingusersmoraltheoriesforautomatedmoralagents |