A pen is all you need : online handwriting recognition using transformers

In 2020, STABILO released the novel OnHW-chars dataset, which contains time series data recordings of characters written with a sensor-enhanced pen. This dataset was accompanied by an extensive exploratory work, presenting baseline accuracies for numerous types of Machine Learning and Deep Learning based classifiers. In 2021, STABILO released a new dataset, containing recordings of written mathematical equations. In this paper, we explore the possibility of applying the recently popularized Transformer architecture to this new dataset. We present a simple but effective adaptation of the Transformer model, where we use two convolutional layers for embedding the input sequence data. With this model, we achieve 92.69\% accuracy per predicted individual token. This accuracy successfully highlights the effectiveness of the Transformer architecture in sequence to sequence problems, and encourages further experiments with this model on the OnHW datasets.