Text Generation is a subfield of Natural Language Processing (NLP) which attempts to teach machines to automatically generate texts that are indistinguishable from texts written by humans.

Transformers & Large Language Models is concerned with models based on the transformer neural network architecture that introduced the attention mechanism and has become foundational in various machine learning applications. In addition, this concept includes Large Language Models, a specific application of transformers that are pre-trained on large amounts of textual data and excel at various natural language processing tasks.

Speech & Audio in NLP is a subfield of Multimodality that deals with the processing of spoken language and audio data in conjunction with textual data. It involves developing algorithms and models that can recognize and transcribe speech and audio, understand and interpret their meaning, and generate natural language responses.

Language Models are a subfield of Semantic Text Processing that deal with the probabilistic modeling and generation of natural language, based on a learned probability function of sequences of words. Language models generate probabilities by training on corpora of natural language texts and can be fine-tuned for a variety of different downstream tasks.

Semantic Text Processing is a subfield of Natural Language Processing (NLP) that attempts to derive meaning from natural language and helps machines interpret textual data semantically. It involves developing algorithms and models that can understand and represent the semantics of words and sentences, and the relationships between them.

Automatic Speech Recognition (ASR), or Speech-to-Text (STT), is a technology that converts spoken language into written text. It is a field that focuses on the development of systems and algorithms capable of transcribing spoken language accurately. ASR systems are designed to understand and interpret human speech, allowing computers to process and analyze spoken words.

Multimodality is a subfield of Natural Language Processing (NLP) that refers to the capability of a system or method to process input of different types or “modalities”, such as natural language text, speech, audio, images, video, and programming languages in NLP applications. It involves developing algorithms and models that can process and analyze information from multiple modalities, and integrate them to form a unified representation of the input.

Natural Language Processing (NLP) is an interdisciplinary subfield of linguistics, computer science, and artificial intelligence concerned with the interactions between computers and human language, in particular how to program computers to process and analyze large amounts of natural language data. The goal is a computer capable of 'understanding' the contents of documents, including the contextual nuances of the language within them. The technology can then accurately extract information and insights contained in the documents as well as categorize and organize the documents themselves.

<h1>ASR SSN CSE@ LT-EDI-2023:Pretrained Transformer based Automatic Speech Recognition system for Elderly People</h1> <h2>Abstract</h2> <p>This paper discusses about the result submitted in Shared Task on Speech Recognition for Vulnerable Individuals in TamilLT-EDI2023(B et al., 2023). The task is to develop an automatic speech recognition system for Tamil language. The dataset provided in the task is collected from the elderly people who converse in Tamil language. The proposed ASR system is designed with pre-trained model anuragshas/wav2vec2-xlsr-53-tamil. The pretrained model used in our system is fine-tuned with Tamil common voice dataset. The test data released from the task is given to the proposed system, now the transcriptions are generated for the test samples and the generated transcriptions is submitted to the task. The result submitted is evaluated by task, the evaluation metric used is Word Error Rate (WER). Our Proposed system attained a WER of 39.8091%.</p> <h2>1 Introduction</h2> <p>This shared task tackles a difficult area in Tamil automatic speech recognition system for vulnerable elderly and transgender individuals. To take care of their daily necessities, elderly people go to important places including banks, hospitals, and administrative offices. Many elderly folks are not aware of how to use the tools provided to help people. Similar to how transgender persons lack access to basic schooling due to societal discrimination, speech is the only channel that can help them meet their demands. The data on spontaneous speech is collected from elderly and transgender people who are unable to take benefit of these amenities (Fukuda et al., 2019; H¨ am¨ al¨ ainen et al., 2015). 2 hours of speech data will be made available for testing, while the speech corpus containing 5.5 hours of transcribed speech will be made available for the training set. Recently, the majority of people have begun using various electronic devices to access the internet. In this situation, the elderly people have also started using smart phones to access the internet (Vacher et al., 2015). Some elderly people attempt to acquire information from the internet using their audio message because they are not well-versed in technology. An acoustic model must be created to handle these types of audio messages from elderly individuals; the model will identify their speech and extract the output of the speech data. As a result, a text file will be the output. The speech’s output will be used to determine the WER value. The WER number demonstrates how accurately the model predicted the outcome . No other corpus for elderly people is larger than the Japanese Newspaper Article Sentences (JNAS), Japanese Newspaper Article Sentences Read Speech Corpus of the Aged (S-JNAS), and Corpus of Spontaneous Japanese (CSJ) corpora (Fukuda et al., 2020). It has been determined that Automatic Speech Recognition using some standard models has not achieved a good performance (Nakajima and Aono, 2020). It is challenging to identify conversational speech in public settings since each person may have their own accent and pronunciation. Additionally, the methodology for identifying standard speech cannot be applied to the conversational speech corpus because it raises WER. A transformer model technique is utilised to treat this type of older people’s conversational speech. The paper is organised as follows: Section 2 discusses the examination of related literature, Section 3 describes the data set, Section 4 discusses the methodology, Section 5 describes the implementation, Section 6 describes the observations, and Section 7 discusses the discussion. Section 8 of the essay concludes with a section on future research.</p> <h2>2 Related Work</h2> <p>There have been numerous studies on recognising the speech of elderly persons using the adaption acoustic model for CSJ corpus (Fukuda et al., 2020), which yields the lowest WER values. The performance of continuous word identification and phoneme recognition is measured from the two distinct age groups, and the corpus is collected in Bengali (Das et al., 2011). Prosodic and spectral properties are retrieved for senior people speech.The exploration of additional features (Lin and Yu, 2015), such as the speech’s volume level, sampling frequency, fundamental frequency, and sentence segmentation, is also possible. Other measurements were locating the pause in the sentence and calculating how long it lasted (Nakajima and Aono, 2020). Low number of utterances is a sign of inadequate performance. If the recorded voice quality is poor (Iribe et al., 2015), the WER value rises. By integrating the transformers for a broad context (Masumura et al., 2021), the E2E ASR transformer can perform encoding and decoding in a hierarchical manner. The WER is decreased by 25.4% via transfer learning when using the hybridbased LSTM transformer (Zeng et al., 2021). Additionally, the LSTM decoder lowers WER by 13%. Self-attention and a multi-head attention layer (Lee et al., 2021) can be used to carry out the encoding and decoding of transformer models. The transformer model is utilised for CTC/Attention based End-To-End ASR, and it produces a WER of 23.66% (Miao et al., 2020). Transformers for streaming ASR are the foundation of the end-to-end ASR system, where an output must be produced quickly after each spoken word. Time-restricted self-attention is employed for the encoder, and prompted attention is used for the encoder-decoder attention mechanism. The innovative fusion attention technique results in a WER reduction of 16.7% on the Wall Street Journal test compared to the nonfusion standard transformer and 12.1% compared to other authors’ transformer-based benchmarks. Findings of the automatic speech recognition for vulnerable individuals are given in (S and B, 2022) (B et al., 2022), have used transformer models used for transformer based ASR for Vulnerable Individuals in Tamil.</p> <h2>3 Dataset Description</h2> <p>Tamil conversational speech data is collected from the elderly people. The speech corpus contains a total of 6 hours and 42 minutes of speech data (Bharathi et al., 2022). The recorded speech of elderly people contains how the elderly people communicate in primary locations like market, bank, shop, public transport and hospitals. It includes both male and female utterances and also this speech data is collected from the transgender people. Table 1. contains the detailed description about the collected data. 30 67 The below Figure 1. shows the sample prediction for the given corpus.</p> <h2>4 Proposed Work</h2> <p>In our proposed system, the pretrained transformer model anuragshas/wav2vec2-xlsr-53-tamil 1 is used. The pretrained model ”anuragshas/wav2vec2-xlsr53-tamil” is based on the Wav2Vec2 architecture and specifically trained for the Tamil language. Wav2Vec2 2 is a state-of-the-art speech recognition model developed by Facebook AI. It utilizes a selfsupervised learning approach, where it learns from large amounts of unlabeled speech data to build representations that capture meaningful information about the audio. The model is based on the transformer architecture, which has proven to be highly effective for various natural language processing tasks. Transformers enable the model to efficiently capture long-range dependencies in the audio input. The model is pretrained on a large corpus of multilingual and monolingual data containing Tamil speech. During pretraining, it learns to predict masked or distorted portions of the input audio, which helps it understand the underlying structure and features of the speech data. After pretraining, the model undergoes fine-tuning using labeled data for specific downstream tasks. Fine-tuning allows the model to adapt to a particular speech recognition task, such as transcription or keyword spotting, in this case, for Tamil language. Although the model is specifically trained for Tamil, it benefits from the multilingual nature of its pretraining data. It can understand and process speech from various languages, making it useful for multilingual applications or tasks involving code-switching. The model has been trained on a vast vocabulary, enabling it to handle a wide range of words and phrases. This makes it suitable for tasks that involve transcribing or recognizing speech with diverse vocabulary. The model’s training data and fine-tuning procedure focus on capturing the unique characteristics of the Tamil language, including its phonetics, phonology, and syntax. This enhances its ability to accurately recognize and transcribe Tamil speech.</p> <h2>5 Implementation</h2> <p>To create an efficient acoustic model based on a pre-trained transformer model. There are many publicly accessible transformer-based pre-trained models. Here, the ”anuragshas/wav2vec2-xlsr-53tamil” pretrained model for handling Tamil speech corpus is used. This pretrained model is fine-tuned from ”facebook/wav2vec2-large-xlsr-53” 3 by common voice dataset in Tamil. The model can be used directly and only accepts input if the voice data is sampled at 16 KHz. It is independent of any language model. The model for creating the wav2vec uses the XSLR (Cross-Lingual Speech Representation), which additionally tests cross-lingual speech data. The quantization of latents, which is common to all languages, can be learned by XLSR. The voice utterance is loaded into the library, saved in a variable, and tokenized using the tokenizer. This process converts the audio to text, and the results are transcripts of the audio file that is loaded into the library. The transcripts are kept in a separate folder after the voice recognition process is complete. Between the transcripts produced by the model and the actual transcripts of the audio written by humans, the WER (Word Error Rate) is determined. The degree of voice recognition can be calculated using the WER value.</p> <h2>6 Evaluation of Results</h2> <p>The evaluation metric used by the task to test the results submitted by us is based on the WER computed between the ASR hypotheses submitted by the participants and the ground truth of human speech transcription. WER ( Word Error Rate) = ( S + D + I) / N where, S = No. of substitutions D = No. of deletions I = No. of insertions N = No. of words in the reference transcription</p> <h2>7 Observation</h2> <p>The name of the speech data and its WER value are included in the result. Similar to this, the same procedure is used for all audio files. The number of subgroups into which each audio file is divided is also listed in the table. The test set audio files’ average WER value, which comprises utterances from men, women, and transgender people, is determined in Table 2. 19 6 Audio-42 24 43.11616667 7 Audio-43 30 37.94115667 8 28 37.29702143 9 26 44.35576154 10 47 39.35465106</p> <h2>8 Discussion</h2> <p>From the Table 2, the experimental result says that the average WER for the testing dataset. The number of test speech utterances are 239. Similarly, Table 3, says the result of total 239 audio subset files from 10 audio files which is given for testing and the WER measured is 39.80%. We ranked second position in shared task competition.</p> <h2>9 Conclusion</h2> <p>Conversational speech data is used to enhance the speech recognition system’s ability to identify older persons. Using a trained model, an automatic speech recognition system is created. Older adults and transgender people with Tamil as their mother tongue are the subjects of a dataset collection. The dataset’s utterance was captured during a conversation in a primary site in Tamil. As the system’s pre-trained model was improved using a common speech dataset, in the future the model might be trained using our own dataset and utilised for testing, which could improve performance.</p> <h2>References</h2> <p>Bharathi B, Bharathi Raja Chakravarthi, Subalalitha Chinnaudayar Navaneethakrishnan, Sripirya N, Rajeswari Natarajan, Suhasini S, and Swetha Valli. 2023. Overview of the second shared task on speech recognition for vulnerable individuals in tamil. In Proceedings of the Third Workshop on Speech and Language Technologies for Dravidian Languages , Varna, Bulgaria. Recent Advances in Natural Language Processing. Bharathi B, Dhanya Srinivasan, Josephine Varsha, Thenmozhi Durairaj, and Senthil Kumar B. 2022. SSNCSE NLP@LT-EDI-ACL2022:hope speech detection for equality, diversity and inclusion using sentence transformers. In Proceedings of the Second Workshop on Language Technology for Equality, Diversity and Inclusion , pages 218–222, Dublin, Ireland. Association for Computational Linguistics. B Bharathi, Bharathi Raja Chakravarthi, Subalalitha Chinnaudayar Navaneethakrishnan, N Sripriya, easy-to-understand speech synthesis for japanese elderly adults. In 2020 23rd Conference of the Oriental COCOSDA International Committee for the Coordination and Standardisation of Speech Databases and Assessment Techniques (O-COCOSDA) , pages 145–150. IEEE. Suhasini S and Bharathi B. 2022. SUH ASR@LT-EDIACL2022: Transformer based approach for speech recognition for vulnerable individuals in Tamil. In Proceedings of the Second Workshop on Language Technology for Equality, Diversity and Inclusion , pages 177–182, Dublin, Ireland. Association for Computational Linguistics. Michel Vacher, Fr´ ed´ eric Aman, Solange Rossato, and Franc¸ois Portet. 2015. Development of automatic speech recognition techniques for elderly home support: Applications and challenges. In International Conference on Human Aspects of IT for the Aged Population , pages 341–353. Springer. Zhiping Zeng, Haihua Xu, Yerbolat Khassanov, Eng Siong Chng, Chongjia Ni, Bin Ma, et al. 2021. Leveraging text data using hybrid transformer-lstm based end-to-end asr in transfer learning. In 2021 12th International Symposium on Chinese Spoken Language Processing (ISCSLP) , pages 1–5. IEEE.</p>

Submission of the paper for the result submitted in Shared Task on Speech Recognition for Vulnerable Individuals in Tamil- LT-EDI-2023. The task is to develop an automatic speech recognition system for Tamil language. The dataset provided in the task is collected from the elderly people who converse in Tamil language. The proposed ASR system is designed with pre-trained model. The pre-trained model used in our system is fine-tuned with Tamil common voice dataset. The test data released from the task is given to the proposed system, now the transcriptions are generated for the test samples and the generated transcriptions is submitted to the task. The result submitted is evaluated by task, the evaluation metric used is Word Error Rate (WER). Our Proposed system attained a WER of 39.8091%.

Publication:

ASR_SSN_CSE@LTEDI- 2023: Pretrained Transformer based Automatic Speech Recognition system for Elderly People

Related Fields of Study

Citations

References