An Analysis of Reinforcement Learning with Function Approximation
En palabras de los autores
We address the problem of computing the optimal Q-function in Markov decision problems with infinite state-space. We analyze the convergence properties of several variations of Q-learning when combined with function approximation, extending the analysis of TD-learning in Tsitsiklis and Van Roy (1996) to stochastic control settings. We identify conditions under which such approximate methods converge with probability 1. We conclude with a brief discussion on the general applicability of our results and compare them with several related works.
Apareció: martes, 10 de noviembre. arXiv. Preprint con versión publicada.
Versión publicada: Francisco S. Melo, Sean P. Meyn, M. Isabel Ribeiro. "An analysis of reinforcement learning with function approximation." Proc. 25th Int. Conf. Machine Learning, pp. 664-671, 2008
Comentario de los autores: This version of the paper corrects the one that was presented at the 25th International Conference on Machine Learning. We corrected condition (7) of Theorem 1 and its proof, as well as the importance-sampling update (11). We also made the