### Abstract

In this paper we address the problem of the stability and convergence of the stochastic approximation procedure
theta(n+1) = theta(n) + gamma(n+1)[h(theta(n)) + xi(n+1)].
The stability of such sequences {theta(n)} is known to heavily rely on the behavior of the mean field h at the boundary of the parameter set and the magnitude of the stepsizes used. The conditions typically required to ensure convergence, and in particular the boundedness or stability of {theta(n)}, are either too difficult to check in practice or not satisfied at all. This is the case even for very simple models. The most popular technique for circumventing the stability problem consists of constraining {theta(n)} to a compact subset K in the parameter space. This is obviously not a satisfactory solution, as the choice of K is a delicate one. In this paper we first prove a "deterministic" stability result, which relies on simple conditions on the sequences {xi(n)} and {gamma(n)}. We then propose and analyze an algorithm based on projections on adaptive truncation sets, which ensures that the aforementioned conditions required for stability are satisfied. We focus in particular on the case where {xi(n)} is a so-called Markov state-dependent noise. We establish both the stability and convergence with probability 1 (w.p.1) of the algorithm under a set of simple and verifiable assumptions. We illustrate our results with an example related to adaptive Markov chain Monte Carlo algorithms.

Translated title of the contribution | Stability of stochastic approximation under verifiable conditions |
---|---|

Original language | English |

Pages (from-to) | 283 - 312 |

Number of pages | 30 |

Journal | SIAM Journal on Control and Optimization |

Volume | 44 (1) |

DOIs | |

Publication status | Published - Jan 2005 |

### Bibliographical note

Publisher: Society for Industrial and Applied MathematicsOther identifier: IDS Number: 961MD

## Fingerprint Dive into the research topics of 'Stability of stochastic approximation under verifiable conditions'. Together they form a unique fingerprint.

## Cite this

Andrieu, C., Moulines, E., & Priouret, P. (2005). Stability of stochastic approximation under verifiable conditions.

*SIAM Journal on Control and Optimization*,*44 (1)*, 283 - 312. https://doi.org/10.1137/S0363012902417267