Day 2: Double Embedding and CNN-based Sequence Labelling for Aspect Extraction
Introduction and Purpose
In this paper proposed a simple model to solve the aspect extraction problem using a simple CNN model and two types of pre-trained embeddings: general-purpose embeddings and domain-specific embeddings. The proposed model called DE-CNN. Here, they try to solve two problems: 1. How to achieve competitive performance without manually crafting features? 2. How to make a simple model?
To address the first consideration, they propose double embedding mechanism. First embedding is general-purpose embedding such as GloVe. The second embedding is domain-specific embeddings. In this case they use the SemEval 2014 dataset which contain two domains: restaurant and laptop domain. For the restaurant domain, review corpus is from Yelp Review Dataset. Meanwhile, the laptop review corpus is from Amazon Review Dataset.
Propose Model
Dataset : The first dataset is from laptop domain on SemEval 2014 task 4. The second datasets is from the restaurant domain on SemEval 2016 task 5.
The proposed model is based on Figure 1.
From Figure 1. The proposed model consists of 2 embedding layers, 4 CNN layers, a fully connected layer shared across all positions of words, and a softmax layer over the labelling space Y={B,I,O} for each position inputs. The red colour on Figure 1 describe the zero vectors. Purple triangels are CNN filters. For the embedding process, we give brief example. If the training/testing data is in the laptop domain, then embeddings from the electronics domain (Amazon) are consider to be out of domain embedding. That is only laptop reviews are considered to be in-domain.
Then they concatenate two embeddings and feed the result into a stack of 4 CNN layers. A CNN layer has many 1D-convolution filters and each filter has a fixed kernel size.
Result
DE-CNN can give better performance than complex method. It got 81.59% and 74.37% F1 score for laptop domain and restaurant domain respectively. For more detail about this research, I give the paper link in reference part.
