Full Text Available
Note: Clicking the button above will open the full text document at the original institutional repository in a new window.
A Comparative Analysis of Pragmatic Understanding of the Arabic Language by Major “Chat-prompted” LLMs (Large Language Models) (Gemini 2.5 Pro, ChatGPT 4o, and DeepThink-R1)
This study aims to assess the latest developments in the capabilities of three of the most recent “chat-prompted” LLMs (Large Language Models), Gemini 2.5 Pro, ChatGPT 4o, and DeepThink-R1, as of July 2025 in understanding the pragmatics of the Arabic language and detecting speech acts via testing o...
Saved in:
| Main Author: | |
|---|---|
| Format: | Thesis |
| Published: |
AUC Knowledge Fountain
2026
|
| Subjects: | |
| Tags: |
No Tags, Be the first to tag this record!
|
| _version_ | 1867613431759110144 |
|---|---|
| access_status_str | Open Access |
| author | Ahmed, Doha M. |
| author_browse | Ahmed, Doha M. |
| author_facet | Ahmed, Doha M. |
| author_sort | Ahmed, Doha M. |
| collection | Thesis |
| description | This study aims to assess the latest developments in the capabilities of three of the most recent “chat-prompted” LLMs (Large Language Models), Gemini 2.5 Pro, ChatGPT 4o, and DeepThink-R1, as of July 2025 in understanding the pragmatics of the Arabic language and detecting speech acts via testing on segments derived from audiovisual media material, for example, the Egyptian Arabic talk show “Maa’koum Mona El-Shazly” “ ﻲﻟذﺎﺸﻟا ﻰﻨﻣ ﻢﻜﻌﻣ .” Unlike social media interactions and simple conversations, other media formats, as in talk shows, integrate more pragmatic context(s), and are expected to present a challenging dataset for LLMs to test their abilities. Investigating “pragmatic understanding” is limited in the field(s) of linguistics and Natural Language Processing (NLP), as focus is often on semantic understanding of LLMs only, as in sentiment analysis. The study aims to fill this gap by offering a new dimension of analysis via testing on “chat-prompted” LLMs. The study relied on a mixed-method approach for data analysis of LLMs’ outputs. A total of 23 excerpts selected from an audiovisual media episode were used to run the pragmatic analysis test on the three models (Gemini 2.5, ChatGPT 4, and DeepThink-R1) via the “chat-prompt” interface. Findings revealed that the first model was more adept in interpreting the pragmatic features in the utterances compared to the second and third models, both qualitatively and quantitatively, followed by the third model. The second model seems to be less performant in recognizing indirect speech acts, compared to the other two models. The study concluded with an example for an in-class AFL application that focuses on pragmatics (Arabic Pragmatic Analyzer) that integrates the first model for an enhanced learning and instruction experience, providing guidance for pragmatic instruction in AFL classes using LLMs and AI. |
| format | Thesis |
| id | oai:fount.aucegypt.edu:etds-3718 |
| institution | American University in Cairo (Egypt) |
| last_indexed | 2026-06-10T12:35:59.828Z |
| license_str | Not specified — see source repository |
| provenance_str_mv | Harvested via OAI-PMH from AUC Knowledge Fountain — bepress |
| publishDate | 2026 |
| publishDateRange | 2026 |
| publishDateSort | 2026 |
| publisher | AUC Knowledge Fountain |
| publisherStr | AUC Knowledge Fountain |
| record_format | dspace |
| source_str | AUC Knowledge Fountain — bepress |
| spelling | oai:fount.aucegypt.edu:etds-3718 A Comparative Analysis of Pragmatic Understanding of the Arabic Language by Major “Chat-prompted” LLMs (Large Language Models) (Gemini 2.5 Pro, ChatGPT 4o, and DeepThink-R1) Ahmed, Doha M. This study aims to assess the latest developments in the capabilities of three of the most recent “chat-prompted” LLMs (Large Language Models), Gemini 2.5 Pro, ChatGPT 4o, and DeepThink-R1, as of July 2025 in understanding the pragmatics of the Arabic language and detecting speech acts via testing on segments derived from audiovisual media material, for example, the Egyptian Arabic talk show “Maa’koum Mona El-Shazly” “ ﻲﻟذﺎﺸﻟا ﻰﻨﻣ ﻢﻜﻌﻣ .” Unlike social media interactions and simple conversations, other media formats, as in talk shows, integrate more pragmatic context(s), and are expected to present a challenging dataset for LLMs to test their abilities. Investigating “pragmatic understanding” is limited in the field(s) of linguistics and Natural Language Processing (NLP), as focus is often on semantic understanding of LLMs only, as in sentiment analysis. The study aims to fill this gap by offering a new dimension of analysis via testing on “chat-prompted” LLMs. The study relied on a mixed-method approach for data analysis of LLMs’ outputs. A total of 23 excerpts selected from an audiovisual media episode were used to run the pragmatic analysis test on the three models (Gemini 2.5, ChatGPT 4, and DeepThink-R1) via the “chat-prompt” interface. Findings revealed that the first model was more adept in interpreting the pragmatic features in the utterances compared to the second and third models, both qualitatively and quantitatively, followed by the third model. The second model seems to be less performant in recognizing indirect speech acts, compared to the other two models. The study concluded with an example for an in-class AFL application that focuses on pragmatics (Arabic Pragmatic Analyzer) that integrates the first model for an enhanced learning and instruction experience, providing guidance for pragmatic instruction in AFL classes using LLMs and AI. 2026-02-15T08:00:00Z thesis application/pdf https://fount.aucegypt.edu/etds/2661 https://fount.aucegypt.edu/context/etds/article/3718/viewcontent/Completed_Thesis___Doha_Ahmed__with_Forms_.pdf Theses and Dissertations AUC Knowledge Fountain Pragmatic understanding Large Language Models (LLMs) speech acts audiovisual media material. Arabic Language and Literature Educational Technology Higher Education Higher Education and Teaching Television |
| spellingShingle | Pragmatic understanding Large Language Models (LLMs) speech acts audiovisual media material. Arabic Language and Literature Educational Technology Higher Education Higher Education and Teaching Television Ahmed, Doha M. A Comparative Analysis of Pragmatic Understanding of the Arabic Language by Major “Chat-prompted” LLMs (Large Language Models) (Gemini 2.5 Pro, ChatGPT 4o, and DeepThink-R1) |
| title | A Comparative Analysis of Pragmatic Understanding of the Arabic Language by Major “Chat-prompted” LLMs (Large Language Models) (Gemini 2.5 Pro, ChatGPT 4o, and DeepThink-R1) |
| title_full | A Comparative Analysis of Pragmatic Understanding of the Arabic Language by Major “Chat-prompted” LLMs (Large Language Models) (Gemini 2.5 Pro, ChatGPT 4o, and DeepThink-R1) |
| title_fullStr | A Comparative Analysis of Pragmatic Understanding of the Arabic Language by Major “Chat-prompted” LLMs (Large Language Models) (Gemini 2.5 Pro, ChatGPT 4o, and DeepThink-R1) |
| title_full_unstemmed | A Comparative Analysis of Pragmatic Understanding of the Arabic Language by Major “Chat-prompted” LLMs (Large Language Models) (Gemini 2.5 Pro, ChatGPT 4o, and DeepThink-R1) |
| title_short | A Comparative Analysis of Pragmatic Understanding of the Arabic Language by Major “Chat-prompted” LLMs (Large Language Models) (Gemini 2.5 Pro, ChatGPT 4o, and DeepThink-R1) |
| title_sort | comparative analysis of pragmatic understanding of the arabic language by major chat prompted llms large language models gemini 2 5 pro chatgpt 4o and deepthink r1 |
| topic | Pragmatic understanding Large Language Models (LLMs) speech acts audiovisual media material. Arabic Language and Literature Educational Technology Higher Education Higher Education and Teaching Television |
| url | https://fount.aucegypt.edu/etds/2661 https://fount.aucegypt.edu/context/etds/article/3718/viewcontent/Completed_Thesis___Doha_Ahmed__with_Forms_.pdf |
| work_keys_str_mv | AT ahmeddoham acomparativeanalysisofpragmaticunderstandingofthearabiclanguagebymajorchatpromptedllmslargelanguagemodelsgemini25prochatgpt4oanddeepthinkr1 AT ahmeddoham comparativeanalysisofpragmaticunderstandingofthearabiclanguagebymajorchatpromptedllmslargelanguagemodelsgemini25prochatgpt4oanddeepthinkr1 |