Abstract: An LMS-like algorithm is applied for estimating the time-varying parameter θ_n in the linear model y_n=φ_n^τθ_n+v_n, which is general in the sense that none of the probabilistic properties such as stationarity, Marker property, independence and ergodicity is imposed on any of the processes {y_n}, {φ_n}, {θ_n} and {v_n}. It is shown that the α-th moment of the estimation error is of order of the α-th moment of the observation noise and the parameter variation w_n≜θ_n-θ_n-1.