Ir al contenido

Documat


Introduction to transformers: a break through in NLP

    1. [1] Massachusetts Institute of Technology

      Massachusetts Institute of Technology

      City of Cambridge, Estados Unidos

    2. [2] Universidad de Alcalá

      Universidad de Alcalá

      Alcalá de Henares, España

  • Localización: ATICA2022: Aplicación de Tecnologías de la Información y Comunicaciones Avanzadas y Accesibilidad / coord. por Luis Bengochea Martínez, Paola Ingavélez Guerra, José Ramón Hilera González Árbol académico, 2023, ISBN 9788419745538, págs. 142-147
  • Idioma: inglés
  • Texto completo no disponible (Saber más ...)
  • Resumen
    • Transformers are amongst the most powerful classes of NLP models invented to date. From machine translation to speech recognition systems, models such as BERT and GPT-2 are achieving state-of-the-art performance both in terms of evaluation score (BLEU score) and training time. What’s novel about transformers? Like many other scientific breakthroughs, transformers are the synthesis of several ideas, including transfer learning, attention, and scaling up neural networks. This paper describes the key mechanisms behind their success. In addition, variants of the transformer architecture are described, as well as an overview to the Hugging Face ecosystem. This paper aims to introduce the theory of transformers, essential for students to start deploying and training their own models for practical applications


Fundación Dialnet

Mi Documat

Opciones de artículo

Opciones de compartir

Opciones de entorno