Ir al contenido

Documat

Generative Artificial Intelligence: Fundamentals

  • Corchado, Juan M. [1] Árbol académico ; López F., Sebastian [1] ; Núñez V., Juan M. [1] ; Garcia S., Raul [1] ; Chamoso, Pablo [1] Árbol académico
    1. [1] Universidad de Salamanca

      Universidad de Salamanca

      Salamanca, España

  • Localización: ADCAIJ: Advances in Distributed Computing and Artificial Intelligence Journal, ISSN-e 2255-2863, Vol. 12, Nº. 1, 2023
  • Idioma: inglés
  • DOI: 10.14201/adcaij.31704
  • Enlaces
  • Resumen

    • Generative language models have witnessed substantial traction, notably with the introduction of refined models aimed at more coherent user-AI interactions—principally conversational models. The epitome of this public attention has arguably been the refinement of the GPT-3 model into ChatGPT and its subsequent integration with auxiliary capabilities such as search features in Microsoft Bing. Despite voluminous prior research devoted to its developmental trajectory, the model’s performance, and applicability to a myriad of quotidian tasks remained nebulous and task specific. In terms of technological implementation, the advent of models such as LLMv2 and ChatGPT-4 has elevated the discourse beyond mere textual coherence to nuanced contextual understanding and real-world task completion. Concurrently, emerging architectures that focus on interpreting latent spaces have offered more granular control over text generation, thereby amplifying the model’s applicability across various verticals. Within the purview of cyber defense, especially in the Swiss operational ecosystem, these models pose both unprecedented opportunities and challenges. Their capabilities in data analytics, intrusion detection, and even misinformation combatting is laudable; yet the ethical and security implications concerning data privacy, surveillance, and potential misuse warrant judicious scrutiny.

  • Referencias bibliográficas
    • Abdullah, M., Madain, A., & Jararweh, Y., 2022, November. ChatGPT: Fundamentals, applications and social impacts. In 2022 Ninth International...
    • Abraham, A., Corchado, E., & Corchado, J. M., 2009. Hybrid learning machines. Neurocomputing: An International Journal, 72(13-15), 2729–2730....
    • Adams, L. C., Busch, F., Truhn, D., Makowski, M. R., Aerts, H. J., & Bressem, K. K., 2023. What Does DALL-E 2 Know About Radiology? Journal...
    • Alizadehsani, Z., Ghaemi, H., Shahraki, A., Gonzalez-Briones, A., & Corchado, J. M., 2023. DCServCG: A data-centric service code generation...
    • Ba, J. L., Kiros, J. R., & Hinton, G. E., 2016. Layer normalization. arXiv preprint arXiv:1607.06450.
    • Bahdanau, D., Cho, K., & Bengio, Y., 2014. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473.
    • Baidoo-Anu, D., & Owusu Ansah, L., 2023, January 25. Education in the era of generative artificial intelligence (AI): Understanding the...
    • Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021). On the Dangers of Stochastic Parrots: Can Language Models Be Too...
    • Bengio, Y., 2009. Learning deep architectures for AI. Foundations and trends® in Machine Learning, 2(1), 1–127. 10.1561/2200000006
    • Brown, T. B., et al., 2020. Language Models are Few-Shot Learners. arXiv preprint arXiv:2005.14165.
    • Bubeck, S., Chandrasekaran, V., Eldan, R., Gehrke, J., Horvitz, E., Kamar, E., … & Zhang, Y., 2023. Sparks of artificial general intelligence:...
    • Caliskan, A., Bryson, J. J., & Narayanan, A., 2017. Semantics derived automatically from language corpora contain human-like biases. Science,...
    • Castellanos-Gómez, A., 2023. Good Practices for Scientific Article Writing with ChatGPT and Other Artificial Intelligence Language Models....
    • Chamoso, P., González-Briones, A., Rivas, A., De La Prieta, F., & Corchado, J. M., 2019. Social computing in currency exchange. Knowledge...
    • Chan, W. H., Mohamad, M. S., Deris, S., Zaki, N., Kasim, S., Omatu, S., Corchado J. M. & Al Ashwal, H., 2016. Identification of informative...
    • Chang, Y., Wang, X., Wang, J., Wu, Y., Zhu, K., Chen, H., … & Xie, X., 2023. A survey on evaluation of large language models. arXiv preprint...
    • Chaplin, J. R., Heller, V., Farley, F. J. M., Hearn, G. E., & Rainey, R. C. T., 2012. Laboratory testing the Anaconda. Philosophical Transactions...
    • Chen, D., et al., 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv preprint arXiv:1810.04805.
    • Corchado J. M., 2023. El Despertar de la Inteligencia Artificial Global. Real Academia de Medicina - Salamanca.
    • Corchado, J. M., & Aiken, J., 2002. Hybrid artificial intelligence methods in oceanographic forecast models. IEEE Transactions on Systems,...
    • Corchado, J. M., Chamoso, P., Hernández, G., Gutierrez, A. S. R., Camacho, A. R., González-Briones, A., … & Omatu, S., 2021. Deepint....
    • Corchado, J. M., Díaz, F., Borrajo, L., & Fernández, F., 2000. Redes neuronales artificiales. Un enfoque práctico. Servicio de Publicacións...
    • Denny, P., Kumar, V., & Giacaman, N., 2022. Conversing with Copilot: Exploring Prompt Engineering for Solving CS1 Problems Using Natural...
    • Devlin, J., Chang, M. W., Lee, K., & Toutanova, K., 2018. Bert: Pre-training of deep bidirectional transformers for language understanding....
    • Díaz, F., Fernández–Riverola, F., & Corchado, J. M., 2006. gene-CBR: A Case-Based Reasonig Tool for Cancer Diagnosis Using Microarray...
    • Doshi-Velez, F., Kortz, M., Budish, R., Bavitz, C., Gershman, S., O’Brien, D., … & Waldo, J., 2017. Accountability of AI Under the Law:...
    • Eloundou, T., Manning, S., Mishkin, P., & Rock, D., 2023. Gpts are gpts: An early look at the labor market impact potential of large language...
    • Finnie-Ansley, J., Denny, P., Becker, B. A., Luxton-Reilly, A., & Prather, J., 2022. The Robots Are Coming: Exploring the Implications...
    • Gala, Y., Fernández, Á., Díaz, J., & Dorronsoro, J. R., 2016. Hybrid machine learning forecasting of solar radiation values. Neurocomputing,...
    • Galvez Vallejo, J. L., Snowdon, C., Stocks, R., Kazemian, F., Yan Yu, F. C., Seidl, C., … & Barca, G. M., 2023. Toward an extreme-scale...
    • Gao, C. A., Howard, F. M., Markov, N. S., Dyer, E., Ramesh, S., Luo, Y., Pearson, A. T., 2023. Comparing scientific abstracts generated by...
    • Gerón, A., 2022. Hands-on machine learning with Scikit-Learn, Keras, and TensorFlow. O’Reilly Media, Inc.
    • González-Briones, A., De La Prieta, F., Mohamad, M. S., Omatu, S., & Corchado, J. M., 2018. Multi-agent systems applications in energy...
    • Goodfellow, I., Bengio, Y., & Courville, A., 2016. Deep learning. MIT press.
    • Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., … & Bengio, Y. (2014). Generative adversarial nets....
    • Guan, W., Smetannikov, I., & Tianxing, M., 2020, October. Survey on automatic text summarization and transformer models applicability....
    • Haarnoja, T., Zhou, A., Abbeel, P., & Levine, S., 2018. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with...
    • Hao, K., 2020. We read the paper that forced Timnit Gebru out of Google. Here’s what it says. MIT Technology Review.
    • Hassani, H., Silva, E. S., 2023. The Role of ChatGPT in Data Science: How AI-Assisted Conversational Interfaces are Revolutionizing the Field....
    • He, K., Zhang, X., Ren, S., & Sun, J., 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer...
    • Hendler, J., & Mulvehill, A. M., 2016. Social machines: the coming collision of artificial intelligence, social networking, and humanity....
    • Hernández, G., Rodríguez, S., González, A., Corchado, J. M., & Prieto, J., 2021. Video analysis system using deep learning algorithms....
    • Hernandez-Nieves, E., Hernández, G., Gil-Gonzalez, A. B., Rodríguez-González, S., & Corchado, J. M., 2021. CEBRA: A CasE-Based Reasoning...
    • Ho, J., Song, D., & Elizalde, B., 2020. Denoising Diffusion Probabilistic Models. ArXiv Preprint ArXiv.
    • Hochreiter, S., & Schmidhuber, J., 1997. Long short-term memory. Neural computation, 9(8), 1735–1780. 10.1162/neco.1997.9.8.1735
    • Homolak, J. 2023. Opportunities and risks of ChatGPT in medicine, science, and academic publishing: a modern Promethean dilemma. 10.3325/cmj.2023.64.1
    • Hovy, D., & Spruit, S. L., 2016. The Social Impact of Natural Language Processing. ACL 2016. 10.18653/v1/P16-2096
    • Hu, L., 2022. Generative AI and Future. Retrieved January 23, 2023, from URL https://pub.towardsai.net/generative-ai-and-future-c3b1695876f2
    • Huang, R., Zhou, Z., Zhang, Y., & Zhao, Z., 2023. Make-an-audio: Text-to-audio generation with prompt-enhanced diffusion models. ArXiv...
    • Itoh, S., & Okada, K., 2023. The Power of Large Language Models: A ChatGPT-driven Textual Analysis of Fundamental Data. Available at SSRN...
    • Janbi, N., Mehmood, R., Katib, I., Albeshri, A., Corchado, J. M., & Yigitcanlar, T., 2022. Imtidad: A Reference Architecture and a Case...
    • Jiang, H., Sun, D., Jampani, V., Yang, M. H., Learned-Miller, E., & Kautz, J., 2018. Super SloMo: High Quality Estimation of Multiple...
    • Jiao, F., Ding, B., Luo, T., & Mo, Z., 2023. Panda LLM: Training Data and Evaluation for Open-Sourced Chinese Instruction-Following Large...
    • Jovanovic, M., & Campbell, M., 2022. Generative artificial intelligence: Trends and prospects. Computer, 55(10), 107–112. 10.1109/MC.2022.3192720
    • Jurafsky, D., & Martin, J. H., 2019. Speech and Language Processing. Pearson.
    • Kingma, D. P., & Welling, M., 2013. Auto-Encoding Variational Bayes. arXiv preprint arXiv:1312.6114.
    • Kothadiya, D., Bhatt, C., Sapariya, K., Patel, K., Gil-González, A. B., & Corchado, J. M., 2022. Deepsign: Sign language detection and...
    • Krizhevsky, A., Sutskever, I., & Hinton, G. E., 2012. Imagenet classification with deep convolutional neural networks. Advances in neural...
    • Kudo, T., & Richardson, J., 2018. Sentencepiece: A simple and language independent subword tokenizer and detokenizer for neural text processing....
    • Li, Y., Li, Z., Zhang, K., Dan, R., Jiang, S., & Zhang, Y., 2023. ChatDoctor: A Medical Chat Model Fine-Tuned on a Large Language Model...
    • Liu, H., Sferrazza, C., & Abbeel, P., 2023. Languages are rewards: Hindsight finetuning using human feedback. arXiv preprint arXiv:2302.02676.
    • Lund, B. D., Wang, T., Mannuru, N. R., Nie, B., Shimray, S., & Wang, Z., 2023. ChatGPT and a new academic reality: Artificial Intelligence-written...
    • M. Salvagno, Taccone, F., Gerli, A., 2023. Can artificial intelligence help for scientific writing? 10.1186/s13054-023-04380-2
    • Madotto, A., Lin, Z., Winata, G. I., & Fung, P., 2021. Few-shot bot: Prompt-based learning for dialogue systems. arXiv preprint arXiv:2110.08118.
    • Manning, C. D., et al., 2008. Introduction to Information Retrieval. Cambridge University Press.
    • Maroto-Gómez, M., Castro-González, Á., Castillo, J. C., Malfaz, M., & Salichs, M. Á., 2023. An adaptive decision-making system supported...
    • McCarthy, J., Minsky, M. L., Rochester, N., & Shannon, C. E., 2006. A proposal for the dartmouth summer research project on artificial...
    • McCulloch, W. S., & Pitts, W., 1943. A logical calculus of the ideas immanent in nervous activity. The bulletin of mathematical biophysics,...
    • McGregor, S., Memon, N., & Levy, K., 2020. Cybersecurity and Human Rights. In Proceedings of the 2020 Conference on Fairness, Accountability,...
    • Miao, X., Wang, Y., Jiang, Y., Shi, C., Nie, X., Zhang, H., & Cui, B., 2022. Galvatron: Efficient transformer training over multiple gpus...
    • Moor, J., 2006. The Dartmouth College artificial intelligence conference: The next fifty years. Ai Magazine, 27(4), 87–87.
    • Nadkarni, P. M., Ohno-Machado, L., & Chapman, W. W., 2011. Natural language processing: an introduction. Journal of the American Medical...
    • Nakano, R., Hilton, J., Balaji, S., Wu, J., Ouyang, L., Kim, C., … & Schulman, J., 2021. Webgpt: Browser-assisted question-answering with...
    • Nguyen, T. M., & Wu, Q. J., 2013. A fuzzy logic model based Markov random field for medical image segmentation. Evolving systems, 4, 171-181.
    • Nguyen, T. T., Wilson, C., & Dalins, J., 2023. Fine-Tuning Llama 2 Large Language Models for Detecting Online Sexual Predatory Chats and...
    • Nichol, A., et al., 2021. GLIDE: A Generative Language Model for Text-Driven Applications. Journal of Artificial Intelligence Research, 49(2),...
    • OpenAI, 2021. ChatGPT API Documentation. OpenAI. Recuperado el 22 de agosto de 2023, de https://platform.openai.com/docs/guides/chat
    • OpenAI, 2023, July 20. Custom instructions for ChatGPT. OpenAI Blog. https://openai.com/blog/custom-instructions-for-chatgpt
    • OpenAI, 2023. GPT-4 Technical Report. ArXiv, abs/2303.08774.
    • Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., … & Lowe, R., 2022. Training language models to follow instructions...
    • Parikh, V., Shah, J., Bhatt, C., Corchado, J. M., & Le, D. N., 2022, July. Deep Learning Based Automated Chest X-ray Abnormalities Detection....
    • Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., … & Chintala, S., 2019. Pytorch: An imperative style, high-performance...
    • Pavlyshenko, B. M., 2023. Financial News Analytics Using Fine-Tuned Llama 2 GPT Model. arXiv preprint arXiv:2308.13032.
    • Pérez-Pons, M. E., Alonso, R. S., García, O., Marreiros, G., & Corchado, J. M., 2021. Deep q-learning and preference based multi-agent...
    • Pérez-Pons, M. E., Parra-Dominguez, J., Hernández, G., Bichindaritz, I., & Corchado, J. M., 2023. OCI-CBR: A hybrid model for decision...
    • Radford, A., Narasimhan, K., Salimans, T., & Sutskever, I., 2018. Improving language understanding by generative pre-training.
    • Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., & Sutskever, I., 2021. Language Models are Few-Shot Learners. OpenAI Blog, 9(2),...
    • Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., & Sutskever, I., 2019. Language Models are Unsupervised Multitask Learners. OpenAI...
    • Reiter, E., & Dale, R., 2020. Building Natural Language Generation Systems. Cambridge University Press.
    • Rombach, M., et al., 2022. Generative Latent Diffusion Models for High Definition Video Generation. IEEE Transactions on Multimedia, 24(1),...
    • Rosenblatt, F., 1958. The perceptron: a probabilistic model for information storage and organization in the brain. Psychological review, 65(6),...
    • Rumelhart, D. E., Hinton, G. E., & Williams, R. J., 1986. Learning representations by back-propagating errors. nature, 323(6088), 533–536....
    • Russell, S. J., & Norvig, P., 2014. Artificial intelligence: a modern approach. Harlow.
    • Sarsa, S., Denny, P., Hellas, A., & Leinonen, J., 2022. Automatic Generation of Programming Exercises and Code Explanations Using Large...
    • Sennrich, R., & Zhang, B., 2019. Revisiting low-resource neural machine translation: A case study. arXiv preprint arXiv:1905.11901. 10.18653/v1/P19-1021
    • Serban, I. V., et al., 2017. A Deep Reinforcement Learning Chatbot. arXiv preprint arXiv:1709.02349.
    • Shazeer, N., 2020. Glu variants improve transformer. arXiv preprint arXiv:2002.05202.
    • Sherstinsky, A., 2020. Fundamentals of recurrent neural network (RNN) and long short-term memory (LSTM) network. Physica D: Nonlinear Phenomena,...
    • Siddique, S.; Chow, J. C. L., 2021. Machine Learning in Healthcare Communication. Encyclopedia, 1, 220–239. 10.3390/encyclopedia1010021
    • Simonyan, K., & Zisserman, A., 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556.
    • Su, Z., Chow, J. K., Tan, P. S., Wu, J., Ho, Y. K., & Wang, Y. H., 2021. Deep convolutional neural network–based pixel-wise landslide...
    • Tang, D., Rong, W., Qin, S., Yang, J., & Xiong, Z., 2020. A n-gated recurrent unit with review for answer selection. Neurocomputing, 371,...
    • Tate, E., 2023. High definition video generation: A comprehensive review. Journal of Multimedia Processing.
    • Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., … & Scialom, T., 2023. Llama 2: Open foundation and fine-tuned...
    • Tulyakov, S., Liu, M. Y., Yang, X., & Kautz, J., 2018. Mocogan: Decomposing motion and content for video generation. In Proceedings of...
    • Turing A. M., 1950. Computing machinery and intelligence. In: Mind 59. 236, pp. 433–460. 10.1093/mind/LIX.236.433
    • Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., … & Polosukhin, I., 2017. Attention is all you need. Advances...
    • Von Oswald, J., Niklasson, E., Randazzo, E., Sacramento, J., Mordvintsev, A., Zhmoginov, A., & Vladymyrov, M., 2023, July. Transformers...
    • Vondrick, C., Pirsiavash, H., & Torralba, A., 2016. Generating videos with scene dynamics. In Advances in neural information processing...
    • Wallace, E., et al., 2019. Universal Adversarial Triggers for Attacking and Analyzing NLP. EMNLP 2019. 10.18653/v1/D19-1221
    • Wei, R., & Mahmood, A., 2020. Recent advances in variational autoencoders with representation learning for biomedical informatics: A survey....
    • Williams, J., 2023. Efficacy of AI-generated text detectors. Journal of Educational Technology.
    • Xie, J., Xu, L., & Chen, E., 2018. Image Denoising and Inpainting with Deep Neural Networks. In Advances in Neural Information Processing...
    • Xu, L., Sanders, L., Li, K., Chow, J. C. L., 2021. Chatbot for Health Care and Oncology Applications Using Artificial Intelligence and Machine...
    • Yalalov, D. 2023. La evolución de los chatbots: De la era T9 y GPT-1 a ChatGPT. Mpost. URL: https://mpost.io/es/la-evoluci%C3%B3n-de-los-chatbots-de-la-era-t9-y-gpt-1-a-chatgpt/
    • Zadeh, L. A., 2008. Is there a need for fuzzy logic? Information sciences, 178(13), 2751–2779. 10.1016/j.ins.2008.02.012
    • Zaremba, W., & Brockman, G., 2021, August 10. OpenAI Codex. OpenAI. https://openai.com/blog/openai-codex/
    • Zhao, J., Wang, T., Yatskar, M., Cotterell, R., Ordonez, V., & Chang, K. W., 2018. Gender bias in coreference resolution. In Proceedings...
    • Zhao, Z., et al. 2023. Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Models. Proceedings of the 40th International...
    • Zoran, D., Kabra, R., Lerchner, A., & Rezende, D. J., 2021. Parts: Unsupervised segmentation with slots, attention and independence maximization....
Fundación Dialnet

Mi Documat

Opciones de artículo

Opciones de compartir

Opciones de entorno