Full Text Available
Note: Clicking the button above will open the full text document at the original institutional repository in a new window.
Neural network-based language modelling for code-switching in South African languages.
Thesis (PhD)--Stellenbosch University, 2024.
Saved in:
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Language: | en_ZA en_ZA |
| Published: |
Stellenbosch : Stellenbosch University
2024
|
| Subjects: | |
| Tags: |
No Tags, Be the first to tag this record!
|
| _version_ | 1867613796386734080 |
|---|---|
| access_status_str | Open Access |
| author | Jansen van Vuren, Joshua |
| author2 | Niesler, Thomas |
| author_browse | Jansen van Vuren, Joshua Niesler, Thomas |
| author_facet | Niesler, Thomas Jansen van Vuren, Joshua |
| author_sort | Jansen van Vuren, Joshua |
| collection | Thesis |
| dc_rights_str_mv | Stellenbosch University |
| description | Thesis (PhD)--Stellenbosch University, 2024. |
| format | Thesis |
| id | oai:scholar.sun.ac.za:10019.1/130517 |
| institution | Stellenbosch University (South Africa) |
| language | en_ZA en_ZA |
| last_indexed | 2026-06-10T12:41:50.126Z |
| license_str | Other — see source repository |
| provenance_str_mv | Harvested via OAI-PMH from SUNScholar — Stellenbosch University Repository |
| publishDate | 2024 |
| publishDateRange | 2024 |
| publishDateSort | 2024 |
| publisher | Stellenbosch : Stellenbosch University |
| publisherStr | Stellenbosch : Stellenbosch University |
| record_format | dspace |
| source_str | SUNScholar — Stellenbosch University Repository |
| spelling | oai:scholar.sun.ac.za:10019.1/130517 Neural network-based language modelling for code-switching in South African languages. Jansen van Vuren, Joshua Niesler, Thomas Stellenbosch University. Faculty of Engineering. Dept. of Electrical and Electronic Engineering. Code switching (Linguistics) Automatic speech recognition Programming languages (Electronic computers) Neural networks (Computer science) UCTD Thesis (PhD)--Stellenbosch University, 2024. ENGLISH ABSTRACT: Code-switching, defined as utilising two or more languages within or between sentences, is widespread in multilingual communities. Yet research considering language modelling and automatic speech recognition for code-switching remains limited and challenging. One reason for this is the lack of datasets containing language switching. Since code-switching occurs predominantly in spontaneous speech and rarely in formal communication, the collection of such speech for transcription is complex. Additionally, the necessity for highly skilled multilingual transcribers makes the compilation of code-switched corpora expensive and challenging. For several years, the digital signal processing research group at Stellenbosch University has been compiling a corpus of code-switched speech in South African languages. This corpus serves as the primary data source for this dissertation and consists of four bilingual sub-corpora, each comprising English and a respective Bantu language. Of these, the four Southern Bantu languages, isiZulu, isiXhosa, Sesotho, and Setswana, are all highly under-resourced, and due to the difficulty of transcribing speech in multiple languages, this corpus is substantially smaller than available monolingual speech datasets. Although previous work has established robust acoustic model baselines, developing well performing language models for this corpus, despite several investigations, remains a challenge. The focus of this dissertation is, therefore, the development of improved language models for the highly under-resourced scenario of code-switched speech in South African languages. Our research achieves advancements both in terms of language modelling performance and subsequent automatic speech recognition. We consider the direct mitigation of data sparsity for code-switching and develop new architectures better suited to modelling language switches. The first avenue we explore is the optimisation of LSTM-based language models for the generation of synthetic corpora of code-switched text. We present a novel technique inspired by prompting in large language models for text synthesis, which improves the perplexity measured directly over codeswitches when the synthetic text is used to augment n-gram language models. Despite the considerable improvements in perplexity, these techniques, however, only marginally improve speech recognition. In a subsequent investigation, we optimise LSTM-based language models for the purpose of rescoring N-best lists. We compare pre-training using monolingual corpora in the five considered languages with utilising other, much larger pre-trained language models. We also show the first results of applying bidirectional large language models to N-best list rescoring in code-switched settings. Overall, the pre-trained large language models outperform our meticulously trained LSTM models. All experiments, however, show considerable improvements in speech recognition. In the final investigation, we propose a new neural language model architecture specifically designed to model code-switches. This architecture incorporates an explicit language prediction component designed to model language transitions. This proposed architecture outperforms a classical LSTM language model in terms of perplexity and speech recognition and matches the speech recognition accuracy achieved by a much larger BERT language model. AFRIKAANSE OPSOMMING: Kodewisseling, gedefinieer as die gebruik van twee of meer tale binne of tussen sinne, is wydverspreid in meertalige gemeenskappe. Tog bly navorsing oor taalmodellering en outomatiese herkenning van kode-wisselende spraak beperk en uitdagend. Een rede hiervoor is die gebrek aan datastelle wat taalwisseling bevat. Aangesien kodewisseling hoofsaaklik in spontane spraak voorkom, en selde in formele kommunikasie, is die versameling van sulke spraak vir transkripsie kompleks. Daarbenewens maak die noodsaaklikheid vir hoogs opgeleide meertalige transkribeerders die samestelling van kode-wisselende korpora duur en uitdagend. Die digitale seinverwerking navorsingsgroep by die Universiteit van Stellenbosch is reeds ’n aantal jare betrokke by die insameling van ’n korpus met kode-wisselende spraak in Suid-Afrikaanse tale. Hierdie korpus dien as die primêre databron vir hierdie proefskrif en bestaan uit vier tweetalige subkorpusse, elk bestaande uit Engels en ’n spesifieke Bantoetaal. Van hierdie tale - die vier Suidelike Bantoetale, isiZulu, isiXhosa, Sesotho en Setswana - is almal aansienlik onderbeproef, en as gevolg van die moeilikheid om spraak in meervoudige tale te transkribeer, is hierdie korpus ook aansienlik kleiner as beskikbare eentalige spraakdatastelle. Hoewel vorige werk sterk akoestiese modelle gevestig het, en ten spyte van verskeie ondersoeke, bly die ontwikkeling van goed presterende taalmodelle vir hierdie korpus ’n uitdaging. Die fokus van hierdie proefskrif is dus die ontwikkeling van verbeterde taalmodelle vir die hoogs onderontwikkelde scenario van kode-wisselende spraak in Suid-Afrikaanse tale. Ons navorsing behaal vooruitgang in terme van taalmodellering sowel as die daaropvolgende outomatiese spraakherkenning. Ons oorweeg die direkte vermindering van data-skaarsheid vir kodewisseling en ontwikkel nuwe strukture wat beter geskik is vir die modellering van die wisseling van taal. Die eerste metode wat ons ondersoek is die optimalisering van LSTMgebaseerde taalmodelle vir die generasie van sintetiese korpusse van kodewisselende teks. Ons stel ’n nuwe tegniek voor wat geïnspireer is deur die toepassing van aansporing in groot taalmodelle vir tekssintese, so is ons in staat om die perpleksiteit direk oor kodewissels te verbeter wanneer die sintetiese teks gebruik word om n-gram taalmodelle aan te vul. Ten spyte van die aansienlike verbeterings in perpleksiteit, verbeter spraakherkenning net marginaal. In ’n daaropvolgende ondersoek optimaliseer ons LSTM-gebaseerde taalmodelle met die doel om N-beste lyste te herrangskik. Ons vergelyk voorafrigting (pre-training) deur die gebruik van eentalige korpusse - in die vyf gespesifiseerde tale - met die gebruik van ander, veel groter voorafgerigte (pretrained) taalmodelle. Ons wys ook die eerste resultate van die toepassing van groot bidireksionele (bidirectional) taalmodelle op N-beste lys herrangskiking in kode-wisselende spraak. Oor die algemeen presteer die voorafgerigte groot taalmodelle beter as ons sorgvuldige afgerigte LSTM-modelle. Alle eksperimente toon egter aansienlike verbeteringe in spraakherkenning. In die finale ondersoek stel ons ’n nuwe neurale taalmodel argitektuur voor wat spesifiek ontwerp is om kodewisseling te modelleer. Hierdie argitektuur bevat ’n eksplisiete taalvoorspellingskomponent wat ontwerp is om taaloorgange te modelleer. Hierdie voorgestelde argitektuur oortref ’n klassieke LSTMtaalmodel in terme van perpleksiteit sowel as spraakherkenning en bereik die spraakherkenningsakkuraatheid wat deur ’n veel groter BERT-taalmodel behaal word. Doctorate 2024-02-29T08:03:07Z 2024-04-26T20:29:32Z 2024-02-29T08:03:07Z 2024-04-26T20:29:32Z 2024-03 Thesis https://scholar.sun.ac.za/handle/10019.1/130517 en_ZA en_ZA Stellenbosch University xxvi, 172 pages : illustrations application/pdf Stellenbosch : Stellenbosch University |
| spellingShingle | Code switching (Linguistics) Automatic speech recognition Programming languages (Electronic computers) Neural networks (Computer science) UCTD Jansen van Vuren, Joshua Neural network-based language modelling for code-switching in South African languages. |
| title | Neural network-based language modelling for code-switching in South African languages. |
| title_full | Neural network-based language modelling for code-switching in South African languages. |
| title_fullStr | Neural network-based language modelling for code-switching in South African languages. |
| title_full_unstemmed | Neural network-based language modelling for code-switching in South African languages. |
| title_short | Neural network-based language modelling for code-switching in South African languages. |
| title_sort | neural network based language modelling for code switching in south african languages |
| topic | Code switching (Linguistics) Automatic speech recognition Programming languages (Electronic computers) Neural networks (Computer science) UCTD |
| url | https://scholar.sun.ac.za/handle/10019.1/130517 |
| work_keys_str_mv | AT jansenvanvurenjoshua neuralnetworkbasedlanguagemodellingforcodeswitchinginsouthafricanlanguages |