In this post I explore one of the most innovative models that are part of the long range Transformers; Big Bird by researchers at Google. This post provides a quick description of what the main innovations of the model are and gives a tutorial on how to apply it to a text classification problem.

In this post I explore how to modify the Longformer architecture to perform multilabel classification and evaluate how much improvement does the Longformer offers in comparison with other Transformer architectures that are limited to 512 tokens like RoBERTa.

One of the hottest topics of research in deep learning is graph neural networks. The last few years saw the numbers of publications regarding graph neural networks grow in some of the major conferences.

In this post I explore the Hugging Face implementation of the Longformer a model that can process sequences of texts longer than 512 characters.

In this post I explore the Trainer class of the Transformers library to conduct text classification using Robustly Optimized BERT Pretraining Approach, or more commonly known as RoBERTa.