Resumen de Introduction to transformers: a break through in NLP

Sabrina Queipo, Antonio García Cabot , Eva García López , David de Fitero Domínguez

• Transformers are amongst the most powerful classes of NLP models invented to date. From machine translation to speech recognition systems, models such as BERT and GPT-2 are achieving state-of-the-art performance both in terms of evaluation score (BLEU score) and training time. What’s novel about transformers? Like many other scientific breakthroughs, transformers are the synthesis of several ideas, including transfer learning, attention, and scaling up neural networks. This paper describes the key mechanisms behind their success. In addition, variants of the transformer architecture are described, as well as an overview to the Hugging Face ecosystem. This paper aims to introduce the theory of transformers, essential for students to start deploying and training their own models for practical applications