Recursive Least Squares — System Identification/Recursive

Recursive Least Squares

System Identification / Recursive

Estimates the parameters of any model that is linear in its parameters, one sample per step, from a regressor supplied on a port: ϑ̂ is moved along φ so that φ·ϑ̂ tracks y. The prediction is ŷ = φ′·ϑ̂, the reported error is the a priori residual e = y − ŷ, and how far ϑ̂ moves is what the Estimation Method decides.

Because the regressor arrives on a port rather than being built inside, this one block covers structures that would otherwise each need their own: driven from Tapped Delay it is an adaptive FIR identifier; given past inputs and outputs it is recursive ARX; given any measured basis it fits that basis.

Ports

• phi – the regressor, [n,1]. Its length n sets the parameter count, and so the width of the estimate: nothing else configures it, so the signal wired here cannot disagree with the block.
• y – the measured output the regressor is fitted to, scalar.
• theta – the running parameter estimate, [n,1], updated every step.
• e – the a priori prediction error y − φ′·ϑ̂, scalar, measured BEFORE the step's update. It is the quantity the estimator actually minimises, so it is the honest one to monitor.

Parameters

• Estimation Method – which update law runs. The three are genuinely different recursions, not settings of one:
  • Forgetting Factor – standard RLS. Past data is discounted geometrically, so the estimate keeps tracking a system that drifts.
  • Kalman Random Walk – treats ϑ as a random walk and adds Process Noise to the covariance each step instead of dividing it by a forgetting factor. It keeps adapting without the covariance wind-up an under-excited forgetting factor suffers.
  • Normalized Gradient – carries no covariance at all: ϑ̂ moves along φ by an amount set by Adaptation Gain, normalized by the regressor's own power. n² storage and work per step becomes n, at the cost of slower convergence.
• Forgetting Factor – λ in (0,1], used by the Forgetting Factor method only. 1 weighs every sample equally and settles on a fixed estimate; 0.95–0.99 tracks a drifting one, at the cost of a noisier estimate. Below 1 it needs a persistently exciting regressor: with one that stops exciting some direction, the covariance in that direction grows without bound (classical wind-up), which is what the Kalman method exists to avoid.
• Initial Covariance – the diagonal P is seeded with, for the two covariance methods. Large means "the seed is not trusted", so the estimate moves fast at the start; small means the opposite.
• Initial Parameters – the seed for ϑ̂. A scalar broadcasts to every parameter; an n-long vector is taken in order. This is carried into the generated code as a literal, so an exported core starts exactly where the simulation does.
• Process Noise – Q, added to P's diagonal each step by the Kalman Random Walk method only. It is the assumed per-step drift of ϑ: larger tracks faster and is noisier.
• Adaptation Gain – μ for the Normalized Gradient method only. Convergence needs 0 < μ < 2.
• Regularization – the floor added to φ′·φ in the Normalized Gradient denominator, so a momentarily zero regressor cannot divide by zero. Used by that method only.
• Sampling Time (s) – zero or less inherits the solver's rate; a positive value runs the block at that period.

Notes

• Discrete only: the estimate advances per sample and has no derivative to integrate.
• Stateful. The state is ϑ̂, plus P for the two covariance methods; the Normalized Gradient method carries none.
• There is no warm-up gate, unlike the recursive blocks that build their own regressor: φ arrives complete, so the first sample is a valid update and the estimate starts moving immediately.
• An estimate is only as identifiable as the data. A regressor that does not excite a direction leaves that parameter where it started, and no method recovers it.
• Aggressive forgetting is numerically fragile, and this is a property of the recursion rather than of this implementation. The Forgetting Factor update discounts P in every direction each step but corrects it in only the one the current regressor points along, so a direction the data stops exciting grows as λ^−k until something excites it again – classical covariance wind-up. The further λ sits below 1, the wider P wanders and the more the estimate depends on rounding: measured here, the same model exported to ten languages agreed to 1e-14 at λ = 0.997 and disagreed by up to 3 % at λ = 0.97, differently on each run. Prefer λ at or above about 0.99, and if the application genuinely needs faster tracking use Kalman Random Walk instead: adding Q to the diagonal floors the covariance every step, which is exactly the term wind-up lacks.

Code export

All ten targets: Python, MATLAB, Java, Rust, C, C++, VHDL, Verilog, SystemVerilog and PLC Structured Text. Each emits only the selected method's recursion, so the Normalized Gradient export declares no covariance array in any language. The regressor length is fixed at export time from the resolved port size, so every array bound is a constant and the cores need no allocation. The seed is baked in from the configuration rather than read from the running block, so an exported core starts where the simulation started rather than where it had got to when the export ran.

The three HDL targets are simulation-only: they carry the recursion in double arithmetic and quantize only on the way out to the fixed-point signal ports. Every method divides once per sample, and the two covariance methods span the decades between the seeded P and its settled value; neither belongs in the Q16.16 signal format. A synthesizable fixed-point estimator would need its own scaled datapath, which is separate work.

Simulink bridge

Neither direction. The counterpart would be the System Identification Toolbox's Recursive Least Squares Estimator, which is not installed here, so a bridge could not be verified against the real block even if written. The catalog entry reports the block with that reason instead of silently dropping it.