MSc Dissertation

Under the supervision of Lianlian Qi at Emotech, I completed my master's dissertation project at UCL from March 2020 to September 2020.

Driven by an expansion in the use of online reviews, an established body of research into sentiment analysis of review targets (e.g. restaurants) has motivated the study of finer-grained target aspects (e.g. restaurant service, ambiance). With the advent of transformer-based neural network architectures such as BERT, resulting gains in performance have required significantly high usage of computational resources. A recent transformer model called Albert has shown competitive performance on benchmark language tasks with a much smaller memory footprint than BERT-based competitors. The goal of this project was to investigate the performance of Albert on ABSA tasks such as aspect extraction (AE), aspect sentiment classification (ASC), and end-to-end ABSA (E2E-ABSA).

Architecture

First, I adapted an adversarial training framework for ABSA tasks, substituting the smallest Albert model for further pre-trained BERT models.

Datasets

In addition, for E2E-ABSA I adapted existing English (Lapt14/Rest14/Unified/MAMS) and Mandarin datasets to use a consistent and unified tagging scheme.

Results

Using optimized hyperparameters, variants of the novel Albat architecture achieved state of the art performance on all E2E-ABSA metrics, two AE metrics, and one ASC metric. Furthermore, on four AE metrics and four ASC metrics the optimized Albat model variants demonstrated competitive performance regarding the state of the art.

Resource Usage

Using Albert instead of BERT has appreciably reduced the number of trainable parameters and consequently model size.

Outcomes

At the conclusion of the project I submitted my dissertation and presented the main findings at an Emotech knowledge-sharing plenary session. The project code is open-sourced with the repository hosted at GitHub.