Ir al contenido

Documat

The tiny poet: artificial poetry generation with a constrained GPT-2 model

  • Autores: Sergiu Stoia, Luis Alfonso Ureña López Árbol académico, Arturo Montejo Ráez Árbol académico
  • Localización: Procesamiento del lenguaje natural, ISSN 1135-5948, Nº. 74, 2025, págs. 321-333
  • Idioma: inglés
  • Títulos paralelos:
    • The tiny poet: generación de poesía artificial con un modelo GPT-2 restringido
  • Enlaces
  • Resumen
    • español

      Este trabajo presenta un modelo del lenguaje con restricciones basado en GPT-2 entrenado para la generación de poesía en español. Nuestra propuesta aplica restricciones a las secuencias generadas para satisfacer rima y métrica, mediante un proceso de backtracking en el proceso de generación de texto. Para su evaluación se ha llevado a cabo un test de Turing sobre una muestra de población lega, y una evaluación de diversos factores sobre una escala de Likert por expertos. A pesar de la simplificidad relativa de GPT-2 frente a modelos actuales, los resultados obtenidos ponen en valor los sistemas de generación basados en restricciones frente a modelos con mayor número de parámetros y más costosos de entrenar.

    • English

      This paper presents a GPT-2 based constrained language model trained for poetry generation in Spanish. Our proposal applies constraints to the generated sequences to satisfy rhyme and meter, by means of a backtracking process in the text generation process. For its evaluation, a Turing test has been carried out on a sample of lay population, and an evaluation of several factors on a Likert scale by experts. Despite the relative simplicity of the GPT-2 model compared to current ones, the results obtained highlight the value of constraint-based generation systems as opposed to models with a larger number of parameters and which are far more expensive to train.

  • Referencias bibliográficas
    • Bang, Y., S. Cahyawijaya, N. Lee, W. Dai, D. Su, B. Wilie, H. Lovenia, Z. Ji, T. Yu, W. Chung, Q. V. Do, Y. Xu, and P. Fung. 2023. A Multitask,...
    • Bia, A. and A. Pedreño. 2001. The Miguel de cervantes digital library: the Hispanic voice on the web. Literary and linguistic computing, 16(2):161–177.
    • Brown, T., B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al. 2020a. Language...
    • Brown, T. B., B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss,...
    • Cohen, A. D., A. Roberts, A. Molina, A. Butryna, A. Jin, A. Kulshreshtha, B. Hutchinson, B. Zevenbergen, B. H. Aguera-Arcas, C. ching Chang,...
    • Díaz-Agudo, B., P. Gervás, and P. Gonzalez-Calero. 2002. Poetry generation in colibri, 09.
    • Fandiño, A. G., J. A. Estapé, M. Pàmies, J. L. Palao, J. S. Ocampo, C. P. Carrino, C. A. Oller, C. R. Penagos, A. G. Agirre, and M. Villegas....
    • Federico, M., M. Cettolo, F. Brugnara, and G. Antoniol. 1995. Language modelling for efficient beam-search. Computer Speech and Language,...
    • Garbacea, C. and Q. Mei. 2022. Why is constrained neural language generation particularly challenging?
    • Garneau, N. and L. Lamontagne. 2023. Guided beam search to improve generalization in low-resource data-to-text generation. In Proceedings...
    • Gervás, P. 2001. An expert system for the composition of formal spanish poetry.
    • Gonçalo Oliveira, H. 2017. A survey on intelligent poetry generation: Languages, features, techniques, reutilisation and evaluation, September.
    • Gonçalo Oliveira, H. 2012. Poetryme: a versatile platform for poetry generation, 08.
    • Hu, J. E., H. Khayrallah, R. Culkin, P. Xia, T. Chen, M. Post, and B. Van Durme. 2019. Improved lexically constrained decoding for translation...
    • Hämäläinen, M., K. Alnajjar, and T. Poibeau. 2022. Modern French poetry generation with roberta and gpt-2.
    • King, D., Z. Shen, N. Subramani, D. S. Weld, I. Beltagy, and D. Downey. 2022. Don’t say what you don’t know: Improving the consistency of...
    • Lau, J. H., T. Cohn, T. Baldwin, J. Brooke, and A. Hammond. 2018. Deep-speare: A joint neural model of poetic language, meter and rhyme, 07.
    • Lo, K.-L., R. Ariss, and P. Kurz. 2022. Gpoet-2: A gpt-2 based poem generator.
    • Manurung, R. 2003. An evolutionary algorithm approach to poetry generation, 01.
    • Mcgregor, S., M. Purver, and G. Wiggins. 2016. Process based evaluation of computer generated poetry, 01.
    • Open AI. 2023. GPT-4 Technical Report.
    • Popescu-Belis, A., `A. Atrio, V. Minder, A. Xanthos, G. Luthier, S. Mattei, and A. Rodriguez. 2022. Constrained language models for interactive...
    • Porter, B. and E. Machery. 2024. Ai-generated poetry is indistinguishable from human-written poetry and is rated more favorably. Scientific...
    • Post, M. and D. Vilar. 2018. Fast lexically constrained decoding with dynamic beam allocation for neural machine translation. In Proceedings...
    • Radford, A. and K. Narasimhan. 2018. Improving language understanding by generative pre-training.
    • Radford, A., J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog,...
    • Raffel, C., N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu. 2020. Exploring the limits of transfer learning...
    • Roush, A., S. Basu, A. Moorthy, and D. Dubovoy. 2022. Most language models can be poets too: An AI writing assistant and constrained text...
    • Scholak, T., N. Schucher, and D. Bahdanau. 2021. PICARD: Parsing incrementally for constrained auto-regressive decoding from language models....
    • Shen, C., L. Cheng, L. Bing, Y. You, and L. Si. 2022. Sentbs: Sentence-level beam search for controllable summarization. arXiv preprint arXiv:2210.14502.
    • Shibata, Y., T. Kida, S. Fukamachi, M. Takeda, A. Shinohara, T. Shinohara, and S. Arikawa. 1999. Byte pair encoding: A text compression scheme...
    • Shihadeh, J. and M. Ackerman. 2020. Emily: An emily dickinson machine. In International Conference on Innovative Computing and Cloud Computing.
    • Taori, R., I. Gulrajani, T. Zhang, Y. Dubois, X. Li, C. Guestrin, P. Liang, and T. B. Hashimoto. 2023. Stanford alpaca: An instruction-following...
    • Touvron, H., T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozi`ere, N. Goyal, E. Hambro, F. Azhar, A. Rodriguez,...
    • Vaswani, A., N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin. 2017. Attention is all you need.
    • Wang, J., X. Zhang, Y. Zhou, C. Suh, and C. Rudin. 2021. There Once Was a Really Bad Poet, It Was Automated but You Didn’t Know It. Transactions...
    • Zhang, S., Z. Chen, Y. Shen, M. Ding, J. B. Tenenbaum, and C. Gan. 2023. Planning with large language models for code generation.
    • Zhou, C., Q. Li, C. Li, J. Yu, Y. Liu, G. Wang, K. Zhang, C. Ji, Q. Yan, L. He, et al. 2023. A comprehensive survey on pretrained foundation...
Fundación Dialnet

Mi Documat

Opciones de artículo

Opciones de compartir

Opciones de entorno