pipette
ENEnglish

An Analysis of Reinforcement Learning with Function Approximation

Francisco S. Melo, Sean P. Meyn, M. Isabel Ribeiro

Preprint con versión publicada

En palabras de los autores

We address the problem of computing the optimal Q-function in Markov decision problems with infinite state-space. We analyze the convergence properties of several variations of Q-learning when combined with function approximation, extending the analysis of TD-learning in Tsitsiklis and Van Roy (1996) to stochastic control settings. We identify conditions under which such approximate methods converge with probability 1. We conclude with a brief discussion on the general applicability of our results and compare them with several related works.

Resultado principalEl resumen no menciona limitaciones.

Apareció: martes, 10 de noviembre. arXiv. Preprint con versión publicada.

DOI: 10.1145/1390156.1390240

Versión publicada: Francisco S. Melo, Sean P. Meyn, M. Isabel Ribeiro. "An analysis of reinforcement learning with function approximation." Proc. 25th Int. Conf. Machine Learning, pp. 664-671, 2008

Comentario de los autores: This version of the paper corrects the one that was presented at the 25th International Conference on Machine Learning. We corrected condition (7) of Theorem 1 and its proof, as well as the importance-sampling update (11). We also made the