Full Text Available
Note: Clicking the button above will open the full text document at the original institutional repository in a new window.
Leveraging Retrieval-Augmented Generation, Prompt Engineering, and Vision Language Models for Surface Defect Classification and Root Cause Analysis in Manufacturing
Thesis (MEng)--Stellenbosch University, 2026.
Saved in:
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Language: | English |
| Published: |
Stellenbosch : Stellenbosch University
2026
|
| Tags: |
No Tags, Be the first to tag this record!
|
| _version_ | 1867613983815499776 |
|---|---|
| access_status_str | Open Access |
| author | Mocke, Johannes Jacobus |
| author2 | Louw, Louis |
| author_browse | Louw, Louis Mocke, Johannes Jacobus |
| author_facet | Louw, Louis Mocke, Johannes Jacobus |
| author_sort | Mocke, Johannes Jacobus |
| collection | Thesis |
| dc_rights_str_mv | Stellenbosch University |
| description | Thesis (MEng)--Stellenbosch University, 2026. |
| format | Thesis |
| id | oai:scholar.sun.ac.za:10019.1/136200 |
| institution | Stellenbosch University (South Africa) |
| language | English |
| last_indexed | 2026-06-10T12:44:49.127Z |
| license_str | Other — see source repository |
| provenance_str_mv | Harvested via OAI-PMH from SUNScholar — Stellenbosch University Repository |
| publishDate | 2026 |
| publishDateRange | 2026 |
| publishDateSort | 2026 |
| publisher | Stellenbosch : Stellenbosch University |
| publisherStr | Stellenbosch : Stellenbosch University |
| record_format | dspace |
| source_str | SUNScholar — Stellenbosch University Repository |
| spelling | oai:scholar.sun.ac.za:10019.1/136200 Leveraging Retrieval-Augmented Generation, Prompt Engineering, and Vision Language Models for Surface Defect Classification and Root Cause Analysis in Manufacturing Mocke, Johannes Jacobus Louw, Louis Lucke, D. Stellenbosch University. Faculty of Engineering. Dept. of Industrial Engineering. Thesis (MEng)--Stellenbosch University, 2026. Mocke, J. J. 2026. Leveraging Retrieval-Augmented Generation, Prompt Engineering, and Vision Language Models for Surface Defect Classification and Root Cause Analysis in Manufacturing. Unpublished masters thesis. Stellenbosch: Stellenbosch University [online]. Available: https://scholar.sun.ac.za/items/a1e4dce0-c0aa-4c69-b21e-3e30890ee295 This study investigates the integration of Vision Language Model (VLM)s, Retrieval-Augmented Generation (RAG), and structured prompt-engineering strategies to enhance surface defect classification and Root Cause Analysis (RCA) in manufacturing. The research examines whether generative and multimodal Artificial Intelligence (AI) systems can deliver accurate, explainable, and cost efficient quality control for manufactured components. The first study assessed GPT-4o’s capability for surface defect detection and classification using two datasets: the MVTec AD screw subset and a custom #3DBenchy dataset developed by the author. Results showed that image-only prompting consistently outperformed multimodal (image + text) inputs, achieving up to 94.4% defect identification and 82.6% classification accuracy on the #3DBenchy dataset, and 90.5% defect identification with 76% classification accuracy on the MVTec AD screw subset. Small-sample fine-tuning (5–30 images per class) further improved classification accuracy across both datasets. The second study extended the framework to RCA using RAG and structured prompting, incorporating domain-specific manufacturing knowledge to improve diagnostic reasoning. Three GPT-5 model variants (gpt-5-nano, gpt-5-mini, and gpt-5) were evaluated across reasoning effort levels and prompting strategies (ReAct and ReAct + 5 Whys). Larger models achieved higher diagnostic accuracy, whereas smaller variants—particularly gpt-5-nano at low reasoning effort—offered optimal cost-efficiency. Inconsistent RAG performance underscored the need for curated retrieval contexts. The final study introduced a gated ReAct-style RCA pipeline applying sequential reasoning across the Man, Method, and Machine categories of the 6M framework. Incorporating multimodal data such as G-code parameters, operator logs, temperature time-series, and bed-mesh images enabled near-perfect parameter-level diagnosis. Token usage scaled predictably with reasoning depth, validating the trade-off between interpretability and computational efficiency. Overall, the findings confirm that VLMs, when combined with structured prompting and domain-specific retrieval, can serve as reliable tools for automated defect diagnosis and causal reasoning—laying the groundwork for scalable, explainable, and data-driven quality control systems in advanced manufacturing. Masters 2026-04-24T13:45:36Z 2026-04-24T13:45:36Z 2026-03 Thesis https://scholar.sun.ac.za/handle/10019.1/136200 en Stellenbosch University 264 pages application/pdf Stellenbosch : Stellenbosch University |
| spellingShingle | Mocke, Johannes Jacobus Leveraging Retrieval-Augmented Generation, Prompt Engineering, and Vision Language Models for Surface Defect Classification and Root Cause Analysis in Manufacturing |
| title | Leveraging Retrieval-Augmented Generation, Prompt Engineering, and Vision Language Models for Surface Defect Classification and Root Cause Analysis in Manufacturing |
| title_full | Leveraging Retrieval-Augmented Generation, Prompt Engineering, and Vision Language Models for Surface Defect Classification and Root Cause Analysis in Manufacturing |
| title_fullStr | Leveraging Retrieval-Augmented Generation, Prompt Engineering, and Vision Language Models for Surface Defect Classification and Root Cause Analysis in Manufacturing |
| title_full_unstemmed | Leveraging Retrieval-Augmented Generation, Prompt Engineering, and Vision Language Models for Surface Defect Classification and Root Cause Analysis in Manufacturing |
| title_short | Leveraging Retrieval-Augmented Generation, Prompt Engineering, and Vision Language Models for Surface Defect Classification and Root Cause Analysis in Manufacturing |
| title_sort | leveraging retrieval augmented generation prompt engineering and vision language models for surface defect classification and root cause analysis in manufacturing |
| url | https://scholar.sun.ac.za/handle/10019.1/136200 |
| work_keys_str_mv | AT mockejohannesjacobus leveragingretrievalaugmentedgenerationpromptengineeringandvisionlanguagemodelsforsurfacedefectclassificationandrootcauseanalysisinmanufacturing |