Hendy Saputra
Back to projects

Research

Indonesian Sentiment Analysis

Which Transformer model reads Indonesian student feedback best? I fine-tuned four of them and compared the results.

Result
IndoBERT came out ahead at 92.9% accuracy.
Stack
Python · PyTorch · Transformers · IndoBERT · NLP
Links
Source Code Read the paper
Model architecture: BERT and RoBERTa classify from the [CLS] embedding, GPT-2 from its final hidden states, each through a linear layer and softmax

Model architecture: BERT and RoBERTa classify from the [CLS] embedding, GPT-2 from its final hidden states, each through a linear layer and softmax

Overview

As a research assistant at the University of Riau, I worked on sentiment analysis of student evaluation feedback written in Indonesian.

I fine-tuned four Transformer models (IndoBERT, mBERT, RoBERTa, and GPT-2) on the same data and benchmarked them against each other. IndoBERT, the one pre-trained on Indonesian text, came out ahead at 92.9% accuracy.

The work was published in the Indonesian Journal of Electrical Engineering and Computer Science in 2024.

Highlights

  1. 01Four models fine-tuned on the same dataset: IndoBERT, mBERT, RoBERTa, and GPT-2.
  2. 02Like-for-like benchmark, so the comparison reflects the models and not the setup.
  3. 03Best result: IndoBERT at 92.9% accuracy.
  4. 04Published in IJEECS, 2024.

Want something like this built?

Work with me