Full Text Available
Note: Clicking the button above will open the full text document at the original institutional repository in a new window.
A Comparative Analysis of Pragmatic Understanding of the Arabic Language by Major “Chat-prompted” LLMs (Large Language Models) (Gemini 2.5 Pro, ChatGPT 4o, and DeepThink-R1)
This study aims to assess the latest developments in the capabilities of three of the most recent “chat-prompted” LLMs (Large Language Models), Gemini 2.5 Pro, ChatGPT 4o, and DeepThink-R1, as of July 2025 in understanding the pragmatics of the Arabic language and detecting speech acts via testing o...
Saved in:
| Main Author: | |
|---|---|
| Format: | Thesis |
| Published: |
AUC Knowledge Fountain
2026
|
| Subjects: | |
| Tags: |
No Tags, Be the first to tag this record!
|
| Summary: | This study aims to assess the latest developments in the capabilities of three of the most recent “chat-prompted” LLMs (Large Language Models), Gemini 2.5 Pro, ChatGPT 4o, and DeepThink-R1, as of July 2025 in understanding the pragmatics of the Arabic language and detecting speech acts via testing on segments derived from audiovisual media material, for example, the Egyptian Arabic talk show “Maa’koum Mona El-Shazly” “ ﻲﻟذﺎﺸﻟا ﻰﻨﻣ ﻢﻜﻌﻣ .” Unlike social media interactions and simple conversations, other media formats, as in talk shows, integrate more pragmatic context(s), and are expected to present a challenging dataset for LLMs to test their abilities. Investigating “pragmatic understanding” is limited in the field(s) of linguistics and Natural Language Processing (NLP), as focus is often on semantic understanding of LLMs only, as in sentiment analysis. The study aims to fill this gap by offering a new dimension of analysis via testing on “chat-prompted” LLMs. The study relied on a mixed-method approach for data analysis of LLMs’ outputs. A total of 23 excerpts selected from an audiovisual media episode were used to run the pragmatic analysis test on the three models (Gemini 2.5, ChatGPT 4, and DeepThink-R1) via the “chat-prompt” interface. Findings revealed that the first model was more adept in interpreting the pragmatic features in the utterances compared to the second and third models, both qualitatively and quantitatively, followed by the third model. The second model seems to be less performant in recognizing indirect speech acts, compared to the other two models. The study concluded with an example for an in-class AFL application that focuses on pragmatics (Arabic Pragmatic Analyzer) that integrates the first model for an enhanced learning and instruction experience, providing guidance for pragmatic instruction in AFL classes using LLMs and AI. |
|---|