Nonlinear Least Squares — Control Systems/Optimization

Nonlinear Least Squares

Control Systems / Optimization

Finds the x that minimizes a sum of squared polynomial residuals whose coefficients arrive on a port – MATLAB's lsqnonlin at its default trust-region-reflective algorithm, solved again on every sample:

F_r(x) = Σ_t c_r,t · x₁^e(t,1) · … · x_n^e(t,n),   r = 1 … m,   minimizing F′F

The exponents e are configuration and the coefficients c are the signal, exactly as in Nelder-Mead Search and Unconstrained Minimization. What is new is that m residuals share one exponent pattern, and m need not equal n – which is what makes this a least-squares problem rather than a root find. Its square sibling is Nonlinear System Solve, whose column is the same shape at m = n.

The Jacobian is analytic: a polynomial's derivative is another polynomial, so the block reproduces lsqnonlin configured with SpecifyObjectiveGradient, not its finite-difference default.

The factory setting is Rosenbrock's function as lsqnonlin's own documented residual pair, F = [10(x₂−x₁²); 1−x₁] from [−1.2; 1]: the four terms [0 1; 2 0; 0 0; 1 0] against coefficients [10; −10; 0; 0; 0; 0; 1; −1]. It reaches the minimum in 23 iterations and 24 evaluations, which is lsqnonlin's count exactly – and the same surface the two neighbouring blocks are set to, so all three can be dropped on one diagram and compared.

Ports

• c – the term coefficients, a single column [m·T,1]: residual r occupies rows (r−1)T+1 … rT, in the row order of Term Exponents. A matrix port is not offered, for the reason its square sibling gives: the renderers reshape row-major in Python and column-major in MATLAB, and one column cannot be read two ways.
• x – the minimizer, [n,1] (n = the columns of Term Exponents).
• resnorm – F′F at that x, a scalar [1,1]. MATLAB's second output, and note it is the sum of squares, not half of it.
• exitflag – a scalar [1,1], lsqnonlin's own: 1 the gradient test was met; 3 the change in F′F fell below Function Tolerance; 2 the step fell below Step Tolerance; 0 a limit below stopped the search.

Parameters

• Options – empty (default), or the path of a Solver Options block (Home/Solver Options), MATLAB’s options argument: for the run, every option it sets replaces this block’s parameter of the same name; one it leaves at default, or one this block does not have, changes nothing.
• Term Exponents – a [T,n] matrix of whole numbers from 0 to 3: row t, column i is the power of x_i in term t. T is 1 to 5 terms, n is 1 to 4 variables, and no single term may total more than degree 3. Those caps are the export's – see Code export.
• Residual Count – m, how many residuals share that pattern: 1 to 6, and never fewer than the number of variables. That floor is lsqnonlin's own, not this block's: with fewer residuals than unknowns the algorithm stops with an error, and here it is refused at config load with the reason.
• Start Point – x0, a vector of n entries. The search is LOCAL: it descends into the valley it starts in.
• Optimality Tolerance – OptimalityTolerance, positive; default 1e-6. The first test on every pass, met when ∥J′F∥_∞ falls below it – which is exactly the number lsqnonlin reports as output.firstorderopt. Flag 1.
• Function Tolerance – FunctionTolerance, positive; default 1e-6. Met when the step is well inside the trust region, the step was a good one, and F′F changed by less than tol·(1+|F′F|). Flag 3.
• Step Tolerance – StepTolerance, positive; default 1e-6. Checked only after the two above, and only from the second iteration on. Flag 2.
• Maximum Iterations – 1 to 2000, default 400.
• Maximum Function Evaluations – same range, default 400. Whichever limit binds first gives exit flag 0.
• Sampling Time (s) – zero or less inherits the solver's rate; a positive value runs the block at that period.

Code export

All ten targets: Python, MATLAB, Java, Rust, C, C++, VHDL, Verilog, SystemVerilog and PLC Structured Text, printed from one description of the iteration, so every target does the same arithmetic in the same order as the simulation.

The exponent pattern is unrolled: a power becomes repeated multiplication, because no target can take a loop bound out of a real array. One evaluation computes all m residuals and all m·n Jacobian entries in the same pass, and the body holds two of them, which is why m, n, T and the degree are capped where they are. Everything else – the Householder QR that decides whether the Jacobian needs regularizing, the inner conjugate gradient, the subspace trust-region solve and its Brent root – is emitted as real loops with fixed trip counts and a flag that stops the arithmetic once the answer is found.

⚠ The three HDL targets run the search in simulation-only real arithmetic, quantizing only at the port boundaries: a trust-region step divides and compares across many orders of magnitude, which a Q16.16 datapath does not carry.

Simulink bridge

None (Support::None). The Optimization Toolbox ships no Simulink library at all and lsqnonlin is a MATLAB function, so there is no block to map onto; the bridge reports this block rather than dropping it silently, and it has no parity testbench. Code export verification still covers it across all ten languages.

Notes

• Stateless: the search restarts from Start Point on every sample, so the answer depends only on the coefficients present at that step.
• ⚠ A RANK-DEFICIENT JACOBIAN IS A DEFINED CASE, NOT A FAILURE. When J′J is singular the algorithm does not give up and does not report a bad step: it regularizes the preconditioner – J′J + λ²I for the first λ in 1, 4, 16, … that makes the factorization safe – and takes a Levenberg-Marquardt-like direction instead of a Gauss-Newton one. That is lsqnonlin's own behaviour, and it is common with freely varying coefficients rather than exotic.
• ⚠ resnorm is F′F, not ½F′F. MATLAB's own convention, and the source says so in as many words where it forms the predicted-reduction ratio.
• ⚠ LOCAL, and a small gradient is not a certificate. A different Start Point can reach a different minimum, and a residual system with no finite minimizer walks x outwards until a limit stops it – exit flag 0 with a large x is that case, not a defect.
• eps has no spelling in nine of the ten targets, so MATLAB's eps and sqrt(eps) are the exact binary constants 2⁻⁵² and 2⁻²⁶ here.