Resumen de Trustworthy conversational agents for children
Marina Escobar Planas
This thesis explores the development of trustworthy conversational agents (CAs) that may interact with children, aligning with the European Commission's human-centred artificial intelligence (AI) initiative. CAs are becoming increasingly integral to children's lives, used in various sectors such as education, entertainment, and healthcare. However, children, as a unique user group, present particular needs and challenges, necessitating the development of guidelines and rigorous evaluations to ensure that these systems are trustworthy.
The research begins by examining the fundamentals of CAs, focusing on their interaction with children. A literature review highlighted both the benefits and risks associated with these technologies, emphasising the need for multidisciplinary approaches to mitigate potential ethical concerns.
The concept of trustworthiness is introduced, as well as a significant research gap in CA ethical development.
To address this gap, the thesis integrates knowledge from multiple domains, combining theoretical exploration with empirical studies. This approach leads to the development of a set of guidelines that align with the needs of children and CAs, highlighting the importance of transparency, age-appropriate behaviour, AI awareness, stakeholder involvement, and risk management.
These guidelines were then applied in the design and development of a collaborative storytelling CA. The system's trustworthiness was assessed using the ALTAI framework, demonstrating an improvement in trustworthiness. This evaluation is further supported by an experimental study, examining how children perceive and interact with these systems, with a specific focus on risks such as misinterpreting the CA's non-human nature and children's data sharing behaviour. Key findings underscored the positive impact of transparency, and the need for a more effective approach to clarify the CA's agency.
The contributions of this thesis provide both theoretical insights and practical guidance for developing trustworthy CAs for children, ensuring they are ethically aligned with societal values and children's rights. Future research is encouraged to expand these findings to broader cultural contexts and develop more nuanced evaluation tools for assessing trustworthiness for children.
