Survey on Publicly Available Sinhala Natural Language Processing Tools and Research

arXiv:1906.02358 · cs.CL · Submitted 2019-06-05 · Read on arXiv

cs.CL

Submitted: 2019-06-05

Updated: 2026-09-06

Code: https://github.com/lknlp/lknlp.github.io

Project page: https://openslr.org/52

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Machine Learning — 53× in this paper
• Natural Language Processing — 35× in this paper
• Benchmark — 31× in this paper · explained in 2 episodes
• Accuracy — 27× in this paper · explained in 1 episode
• Low-Resource Languages — 19× in this paper · explained in 4 episodes
• Alignment — 18× in this paper · explained in 1 episode
• Transformer — 18× in this paper · explained in 2 episodes
• Association for Computational Linguistics — 16× in this paper
• Embeddings — 16× in this paper · explained in 2 episodes
• Fine-Tuning — 15× in this paper · explained in 5 episodes
• Statistical Machine Translation — 13× in this paper
• Part of Speech — 12× in this paper
• Analyzer — 10× in this paper · explained in 1 episode
• International Conference on Advanced Research in Computing — 10× in this paper
• Optical Character Recognition — 10× in this paper
• Clustering — 9× in this paper · explained in 2 episodes
• International Conference on Information Technology Research — 9× in this paper
• International Conference on Advancements in Computing — 8× in this paper
• Ontology — 8× in this paper · explained in 2 episodes
• Smart Computing and Systems Engineering — 8× in this paper
• Active Learning — 7× in this paper · explained in 6 episodes
• International Conference on Industrial and Information Systems — 7× in this paper
• Adaptation — 6× in this paper · explained in 1 episode
• Data Augmentation — 6× in this paper · explained in 7 episodes
• Feature Extraction — 6× in this paper · explained in 2 episodes
• Hidden Markov Model — 6× in this paper
• National Information Technology Conference — 5× in this paper
• Semantic Similarity — 5× in this paper · explained in 2 episodes
• Sri Lanka Association for Artificial Intelligence — 5× in this paper
• Tokenization — 5× in this paper · explained in 2 episodes
• Transfer Learning — 5× in this paper · explained in 10 episodes
• Artificial Neural Network — 4× in this paper
• From Innovation to Impact — 4× in this paper
• Information Extraction — 4× in this paper
• Natural Language Understanding — 4× in this paper
• Universal Dependencies — 4× in this paper
• Common Crawl — 3× in this paper
• Hybrid Approach — 3× in this paper · explained in 4 episodes
• Multi-Agent System — 3× in this paper · explained in 5 episodes
• Printed Character Recognition — 3× in this paper
• RAG — 3× in this paper · explained in 3 episodes
• Support Vector Machines — 3× in this paper
• Automatic Speech Recognition (ASR) — 2× in this paper · explained in 2 episodes
• Benchmarking — 2× in this paper · explained in 3 episodes
• Conditional Random Fields — 2× in this paper
• Contrastive Learning — 2× in this paper · explained in 5 episodes
• Differential Object Marking — 2× in this paper
• Distillation — 2× in this paper · explained in 2 episodes
• Ensemble Approach — 2× in this paper · explained in 3 episodes
• Explainability — 2× in this paper · explained in 3 episodes
• Feature Selection — 2× in this paper · explained in 4 episodes
• Few-Shot Learning — 2× in this paper · explained in 4 episodes
• Instruction Tuning — 2× in this paper · explained in 2 episodes
• International Conference on Asian Language Processing — 2× in this paper
• International Informatics and Software Engineering Conference — 2× in this paper
• Interpretability — 2× in this paper · explained in 15 episodes
• Knowledge Distillation — 2× in this paper · explained in 11 episodes
• National Science Foundation — 2× in this paper
• Parameter-Efficient Fine-Tuning — 2× in this paper · explained in 2 episodes
• Reinforcement Learning — 2× in this paper · explained in 4 episodes
• Representation Learning — 2× in this paper · explained in 2 episodes
• Scale Invariant Feature Transform — 2× in this paper
• Semantic Alignment — 2× in this paper · explained in 3 episodes
• Sparsity — 2× in this paper · explained in 4 episodes
• Task Adapters — 2× in this paper
• 1D CNN — 1× in this paper · explained in 2 episodes
• Ablation — 1× in this paper · explained in 1 episode
• Alignment Methods — 1× in this paper · explained in 1 episode
• Alternation — 1× in this paper · explained in 1 episode
• Building Educational Applications — 1× in this paper
• Computational Biology — 1× in this paper · explained in 2 episodes
• Context-Free Grammar — 1× in this paper
• Cooperation — 1× in this paper · explained in 2 episodes
• Core — 1× in this paper · explained in 2 episodes
• Cross-lingual Transfer — 1× in this paper · explained in 2 episodes
• Diarization Error Rate — 1× in this paper
• English Massive Multitask Language Understanding — 1× in this paper
• F1 Score — 1× in this paper · explained in 3 episodes
• Generalizability — 1× in this paper · explained in 6 episodes
• Generalization — 1× in this paper · explained in 13 episodes
• Homogeneity — 1× in this paper · explained in 2 episodes
• Hybrid System — 1× in this paper · explained in 2 episodes
• Intent Mapping — 1× in this paper
• Internet Archive — 1× in this paper
• Massively Multilingual Speech — 1× in this paper
• Maximum Entropy — 1× in this paper
• Named Entity Recognition (NER) — 1× in this paper · explained in 2 episodes
• Parameter-Efficient Fine-Tuning (PEFT) — 1× in this paper · explained in 4 episodes
• Perplexity — 1× in this paper · explained in 3 episodes
• Pointwise Mutual Information (PMI) — 1× in this paper · explained in 3 episodes
• Pruning — 1× in this paper · explained in 4 episodes
• Research Review — 1× in this paper · explained in 6 episodes
• Sinhala Automatic Speech Recognition — 1× in this paper
• Sinhalaspeaking Children with Autism Spectrum Disorder — 1× in this paper
• Starting with Named Entity Recognition — 1× in this paper
• Subspace Gaussian Mixture Model — 1× in this paper
• System Identification — 1× in this paper · explained in 3 episodes
• Transformer Models — 1× in this paper · explained in 2 episodes
• Vector Representations — 1× in this paper · explained in 2 episodes

Sources

• Geographically-Informed Language Identification
• The FLoRes Evaluation Datasets for Low-Resource Machine Translation: Nepali-English and Sinhala-English
• Sinhala Language Corpora and Stopwords from a Decade of Sri Lankan Facebook
• Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets
• Exploiting Parallel Corpora to Improve Multilingual Embedding based Document and Sentence Alignment
• Google Crowdsourced Speech Corpora and Related Open-Source Resources for Low-Resource Languages and Dialects: An Overview
• NSINA: A News Corpus for Sinhala
• Glot500: Scaling Multilingual Corpora and Language Models to 500 Languages
• Adapting the Tesseract Open-Source OCR Engine for Tamil and Sinhala Legacy Fonts and Creating a Parallel Corpus for Tamil-Sinhala-English
• FastText.zip: Compressing text classification models
• Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translation
• No Language Left Behind: Scaling Human-Centered Machine Translation
• Stylomech: Unveiling Authorship via Computational Stylometry in English and Romanized Sinhala
• SOLD: Sinhala Offensive Language Dataset
• Predicting the Type and Target of Offensive Posts in Social Media
• XTREME-UP: A User-Centric Scarce-Data Benchmark for Under-Represented Languages
• An Open Dataset and Model for Language Identification
• Sinhala-English Word Embedding Alignment: Introducing Datasets and Benchmark for a Low Resource Language
• CulturaX: A Cleaned, Enormous, and Multilingual Dataset for Large Language Models in 167 Languages
• MADLAD-400: A Multilingual And Document-Level Large Audited Dataset

Related papers

• Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
• Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
• Subliminal Steering: Stronger Encoding of Hidden Signals
• MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
• The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
• Untangling the Mechanisms of Misleading Context in Medical Question Answering