Publication: Aprendizaje por refuerzo en espacios de estados continuos
Loading...
Identifiers
Publication date
2002
Defense date
2003-04-01
Authors
Advisors
Tutors
Journal Title
Journal ISSN
Volume Title
Publisher
Impact
Export
Abstract
El aprendizaje por refuerzo es un modelo de aprendizaje que permite implementar comportamientos inteligentes de forma automática. La mayor parte de la teoría del aprendizaje por refuerzo tiene su fundamento en la programación dinámica, y por tanto, en lo que se denominan funciones de valor. Sin embargo, la implementación tradicional de estas funciones en forma tabular no es práctica cuando el espacio de estados es muy grande, o incluso infinito. Cuando se produce esta situación, se deben aplicar métodos de generalización que permitan extrapolar la experiencia adquirida para un conjunto limitado de estados, a la totalidad del espacio. Existen dos aproximaciones básicas para resolver este problema. Por un lado, están aquellas técnicas que se basan en obtener una discretización adecuada del espacio de estados. Por otro lado, están los métodos basados en implementar las funciones de valor con algún método supervisado de aproximación de funciones, como, por ejemplo, una red de neuronas. En esta tesis doctoral se pretende desarrollar métodos de aprendizaje por refuerzo que sean aplicables en dominios con espacios de estados continuos, partiendo de las dos aproximaciones planteadas anteriormente, fundiendo las ventajas de una y otra en un método eficaz y eficiente que permita que el aprendizaje sea un proceso totalmente automático.
Reinforcement Learning is a technique that aliows to implement intelli gent behaviours automatically without the need of introducing knowledge or modeis about the domain. Most of the reinforcement learning theory is based on dynamic programming, and hence, on value functions. These func tions provide information about how good it is, in order to solve a defined task, to be in a given situation in the dornain, typically narned state, or even how good it is to execute a defined action if the system is in a given state. These functions, typically implernented using look-up tables, are used to represent the action policy that must guide the behaviour of the system. However, the traditional implementation of these functions as look-up tables is not practical when the state space is very large, or even infinite. When one of these situations appears, generalization methods must be applied in order to extrapolate the acquired experience for a limited set of states, to the whole space, so optirnal behaviours can be achieved, even when the whole domain has not been explored. Two main approaches can be found in the literature. Qn the one hand, there are methods based on learning an adequate state space discretization, so the continuous state space is mapped to a finite and reduced one. Qn the other hand, methods based oil irnplementing the value functions with sorne supervised learning technique for function approximation, for instance, a neural network, can be found. This dissertation tries to develop reinfor cernent learning methods that can be applied in domains with a continuous state space. The start point is given by the two approaches aboye, and it tries to j oin the advantages of one and another in an efficient and effective method that aliows the learning process be a fully automatic process where the designer has to introduce the less possible arnount of information about the task to solve.
Reinforcement Learning is a technique that aliows to implement intelli gent behaviours automatically without the need of introducing knowledge or modeis about the domain. Most of the reinforcement learning theory is based on dynamic programming, and hence, on value functions. These func tions provide information about how good it is, in order to solve a defined task, to be in a given situation in the dornain, typically narned state, or even how good it is to execute a defined action if the system is in a given state. These functions, typically implernented using look-up tables, are used to represent the action policy that must guide the behaviour of the system. However, the traditional implementation of these functions as look-up tables is not practical when the state space is very large, or even infinite. When one of these situations appears, generalization methods must be applied in order to extrapolate the acquired experience for a limited set of states, to the whole space, so optirnal behaviours can be achieved, even when the whole domain has not been explored. Two main approaches can be found in the literature. Qn the one hand, there are methods based on learning an adequate state space discretization, so the continuous state space is mapped to a finite and reduced one. Qn the other hand, methods based oil irnplementing the value functions with sorne supervised learning technique for function approximation, for instance, a neural network, can be found. This dissertation tries to develop reinfor cernent learning methods that can be applied in domains with a continuous state space. The start point is given by the two approaches aboye, and it tries to j oin the advantages of one and another in an efficient and effective method that aliows the learning process be a fully automatic process where the designer has to introduce the less possible arnount of information about the task to solve.
Description
Keywords
Inteligencia artificial, Aprendizaje