Decompounding is an essential preprocessing step in text-processing tasks such as machine translation, speech recognition, and information retrieval (IR). Here, the IR issues are explored from five viewpoints. (A) Does word decompounding impact the Indian language IR? If yes, to what extent? (B) Can corpus-based decompounding models be used in the Indian language IR? If yes, how? (C) Can machine learning and deep learning-based decompounding models be applied in the Indian language IR? If yes, how? (D) Among the different decompounding models (corpus-based, hybrid machine learning-based, and deep learning-based), which provides the best effectiveness in the IR domain? (E) Among the different IR models, which provides the best effectiveness from the IR perspective? This study proposes different corpus-based, hybrid machine learning-based, and deep learning-based decompounding models in Indian languages (Marathi, Hindi, and Sanskrit). Moreover, we evaluate the effectiveness of each activity from an IR perspective only. It is observed that the different decompounding models improve IR effectiveness. The deep learning-based decompounding models outperform the corpus-based and hybrid machine learning-based models in Indian language IR. Among the different deep learning-based models, the Bi-LSTM-A model performs best and improves mean average precision (MAP) by 28.02% in Marathi. Similarly, the Bi-RNN-A model improves MAP by 18.18% and 6.1% in Hindi and Sanskrit, respectively. Among the retrieval models, the In_expC2 model outperforms others in Marathi and Hindi, and the BB2 model outperforms others in Sanskrit.

This paper presents XeraWord, an innovative tool for automatically generating nominal phrases. XeraWord can be used for different tasks, ranging from teaching languages to the creation of examples in lexicography, or even for the development of resources for natural language processing. In this area, Xera was the first experiment, allowing the automatic generation of nominal phrases in three languages: German, French and Spanish. This tool was extended to support other languages, namely, Portuguese and Galician.

We start by presenting the theory behind the development of Xera and its new version, XeraWord, namely, the applied base methodology, and the natural language processing resources used to support it. Then, TraduWord, a tool specifically developed to construct resources for new languages, is presented. This tool allows the semi-automatic translation of the data required for the nominal phrase generation. For this, we discuss its advantages and disadvantages, analysing the quality of the translated resources, as well as the amount of manual work required to validate and correct these resources.

Ambiguity is considered an indispensable attribute of all natural languages. The process of associating the precise interpretation to an ambiguous word taking into consideration the context in which it occurs is known as word sense disambiguation (WSD). Supervised approaches to WSD are showing better performance in contrast to their counterparts. These approaches, however, require sense annotated corpus to carry out the disambiguation process. This paper presents the first-ever standard WSD dataset for the Kashmiri language. The raw corpus used to develop the sense annotated dataset is collected from different resources and contains about 1 M tokens. The sense-annotated corpus is then created using this raw corpus for 124 commonly used ambiguous Kashmiri words. Kashmiri WordNet, an important lexical resource for the Kashmiri language, is used for obtaining the senses used in the annotation process. The developed sense-tagged corpus is multifarious in nature and has 19,854 sentences. Based on this annotated corpus, the Lexical Sample WSD task for Kashmiri is carried out using different machine-learning algorithms (J48, IBk, Naive Bayes, Dl4jMlpClassifier, SVM). To train these models for the WSD task, bag-of-words (BoW) and word embeddings obtained using the Word2Vec model are used. We used different standard measures, viz. accuracy, precision, recall, and F1-measure, to calculate the performance of these algorithms. Different machine learning algorithms reported different values for these measures on using different features. In the case of BoW model, SVM reported better results than other algorithms used, whereas Dl4jMlpClassifier performed better with word embeddings.

Folk culture represents the social, ethnic, and traditional livelihood of people belonging to a certain tribe or community and is important in keeping their culture and tradition alive. The Mizo people are a Tibeto-Burmese ethnic group, native to the Indian state of Mizoram and neighboring regions of Northeast India. Mizo folk culture is an amalgamation of festivity, celebration, liveliness, kinship, brotherhood, and merriment, and above all, preserves the ethnicity of this tribal community that is fundamentally entrenched. Unfortunately, the Mizos are fast giving up their old customs and adopting the new mode of life that is greatly influenced by the western culture. This makes it all the more crucial to preserve the intangible cultural heritage of this ethnic tribe whose folk cultures are vanishing day by day. To the best of our knowledge, this work is the first attempt at preservation and classification of Mizo folk songs. The first part of this paper presents a literature survey on preservation, analysis, and classification of folk songs. The second part presents the methodology for preliminary classification of Mizo folk songs. Three categories of Mizo folk songs—Hunting chants (Hlado), Children’s songs (Pawnto hla), and Elderly songs (Pi pu zai)—are used in this study. A total of 29 acoustic features are used. A long short-term memory network using custom attention layer has been proposed for classification, whose results are compared with four supervised models (Support Vector Machines, K-Nearest Neighbor, Naive Baye’s, and Ensemble). Experimental results from the proposed model are promising, with an implication of scope for future research in acoustic analysis and classification of Mizo folk songs using recent unsupervised methods.

Sarcasm detection research in the Bengali language so far can be considered to be narrow due to the unavailability of resources. In this paper, we introduce a large-scale self-annotated Bengali corpus for sarcasm detection research problem in the Bengali language named ‘Ben-Sarc’ containing 25,636 comments, manually collected from different public Facebook pages and evaluated by external evaluators. Then we present a complete strategy to utilize different models of traditional machine learning, deep learning, and transfer learning to detect sarcasm from text using the Ben-Sarc corpus. Finally, we demonstrate a comparison between the performance of traditional machine learning, deep learning, and transfer learning models on our Ben-Sarc corpus. Transfer learning using Indic-Transformers Bengali Bidirectional Encoder Representations from Transformers as a pre-trained source model has achieved the highest accuracy of 75.05%. The second-highest accuracy is obtained by the long short-term memory model with 72.48% and Multinomial Naive Bayes is acquired the third highest with 72.36% accuracy for deep learning and machine learning, respectively. The Ben-Sarc corpus is made publicly available in the hope of advancing the Bengali Natural Language Processing Community. The Ben-Sarc is available at https://github.com/sanzanalora/Ben-Sarc.

Most existing discourse annotation guidelines have focused on the English language. As a result, there is a significant lack of research and resources concerning computational discourse-level language understanding and generation for other languages. To fill this relevant gap, we introduce the first discourse annotation guideline using the rhetorical structure theory (RST) for low-resource languages. Specifically, this guideline provides accurate examples of discourse coherence relations in three romance languages: Italian, Portuguese, and Spanish. We further discuss theoretical definitions of RST and compare different artificial intelligence discourse frameworks, hence offering a reliable and accessible survey to new researchers and annotators.