Sentiment analysis of the burmese language using the distributed representation of n-gram-based words

Abstract

Due to the availability of people's opinions and customer reviews, the need to analyze those texts have been more important. Sentiment analysis, or opinion mining, estimates their polarity, whether they are positive or negative, using machine learning techniques. Many methods have been proposed but they assume the basic preprocessing of text data including word segmentation and word sentiment values. However, such preprocessing is not easily available for low resource languages such as Burmese, Khmer and Lao due to the unavailability of annotated big corpora and basic natural language processing tools. The objective of this research is to solve these difficulties of low resource language processing. The goal is to propose an effective and efficient method to enable sentiment analysis without considering language specific characteristics. The scope of the research is the languages without word boundaries in written text, specifically Burmese. The methodology consists of two proposals, a character-based variable-length n-gram word model and a word grouping method with word similarities calculated with distributive word representation models. The proposed method is compared with Conditional Random Field (CRF) baseline approach, which is also proposed newly in this thesis, and achieved a similar result as the CRF-based word segmentation with a small size of supervised data. The proposed method is also validated with a larger size of data using Amazon product reviews. Thus, the proposed methods in this thesis provide an effective and efficient way for low resource language processing without focusing on language specific characteristics.