Incremental Classification Kernel Fit — Machine Learning/Incremental Learning

Incremental Classification Kernel Fit

Machine Learning / Incremental Learning

Fits a binary kernel classifier online, one labelled observation per step, and puts the model on a bus every step: the score is f = z(x)·β + b, where z(x) is a fixed random expansion of the predictors into D features that approximates a Gaussian kernel, and f ≥ 0 predicts the second class. This is MATLAB's incrementalClassificationKernel learner with its default scale-invariant solver (Statistics and Machine Learning Toolbox), as Simulink's IncrementalClassificationKernel Fit block runs it: standardize x (when Standardize is on), expand it, z = (1/√(o·b))·[cos T, sin T] with T the Fastfood map of x (o the Hadamard order, b the number of blocks), then fit a linear classifier on z with the labels coded −1 (first class) and +1 (second). The first Estimation Period observations are used only to estimate the predictors' means and standard deviations; every later one updates β and b. Feed the bus to an Incremental Classification Kernel Predict block that carries the same feature map.

Ports

• x – one observation of the P predictors, a row [1,P]. P is read off this width.
• y – the label of that observation, [1,1]: one of the two Class Names. A NaN in x or y skips the observation; any other label stops the run.
• mdl (bus) – the learner after this observation, a bus of eight elements: Beta [D,1], Bias [1,1], IsWarm (bool, true once Metrics Warmup Period observations have been seen after the estimation period), CanPredict (bool, false only while standardization is still being estimated), Mu [P,1] and Sigma [P,1] (the standardization, 0 and 1 when there is none), MajorityClass (u8, 1 or 2: the class seen more often, the second on a tie) and Prior [2,1] (the class proportions among the observations trained on, 0.5 each before the first).

Parameters

• Class Names – the two class labels, [0 1] by default. The learner is binary, as Simulink's block is.
• Learner – the loss the solver minimizes:
  • svm (the default) – the hinge loss of a support vector machine;
  • logistic – the logistic loss of a logistic regression.
• Standardize – off (the default) or on: center and scale each predictor by its mean and standard deviation over the estimation period, before the expansion.
• Estimation Period – how many observations the estimation uses, 1000 by default as in MATLAB. It applies only when Standardize is on; otherwise every observation trains.
• Metrics Warmup Period – how many observations after the estimation period make the model warm (IsWarm), 1000 by default.
• Num Expansion Dimensions – D, the number of expanded features and the length of Beta, the learner's NumExpansionDimensions; 16 by default, at most 256, and larger than P.
• Kernel Scale – the learner's KernelScale, a positive number; the map divides by it. 1 by default.
• Feature Map S, Feature Map G, Feature Map B and Feature Map P – the learner's random Fastfood map, four [b,o] matrices, one row per Hadamard block: o = 2^{ceil(log2 P)} and b = ceil(D/(2o)). Take them from s = toStruct(Mdl.Impl.FeatureMapper) as s.S, s.G, s.B and s.P (P one-based, each row a permutation of 1..o). The defaults are a real map MATLAB drew for 3 predictors and 16 features.
• Sampling Time (s) – zero or less inherits the solver's rate; a positive value runs the block at that period.

When a run is refused

An x that is not a row, a y that is not [1,1], Class Names that are not two different numbers, a negative or fractional period, Standardize on with an Estimation Period of 0, more than 8 predictors, a feature map that does not fit P and D, and, during the run, a label that is neither class name.

Code export

Six targets: Python, MATLAB, Java, Rust, C and C++, each carrying the learner's whole state and running the same step as the live block, in the same arithmetic order; the feature map is written out feature by feature, which is why D is capped. An exported core does not stop on a foreign label: it takes any label other than the first class name as the second. The hardware targets (VHDL, Verilog, SystemVerilog) and PLC Structured Text carry no bus, so an export to one stops and names the block.

Simulink bridge

None. Simulink's block takes its learner as InitialLearner, the name of an incrementalClassificationKernel object in the MATLAB workspace, rather than as dialog parameters, so there is nothing in its dialog to map these configs onto or read them back from.

Notes

• Stateful: the state is the solver's (four running sums per coefficient), the estimation sums, and the class counts. The feature map does not change.
• The bus at a step has already learnt that step's observation, as in Simulink: a Predict block fed the same x sees a model trained on it.
• Simulink's prior filter is reproduced: while no first-class observation has been trained on, every second-class observation after the first is dropped (neither trained on nor counted in Prior), exactly as Simulink's block drops it. Timing also follows Simulink's block rather than MATLAB's fit(): Mu and Sigma appear one step after the estimation period ends, and IsWarm turns true one step after fit() would set it.
• Kernel Scale is fixed: MATLAB's 'auto' estimates it with a random heuristic, which no other run could reproduce, so it is not offered. One observation per step; Simulink's optional weights and reset inputs are not offered.
• A NaN is skipped, where Simulink's block lets it in (measured on R2026a): there a NaN predictor turns Beta into NaN for the rest of the run and a NaN label stops the run; here either one skips the observation.
• Verified against R2026a, by the incremental_kernel_learners regression suite: driven for 120 steps with the inputs Simulink's block was run on, in four configurations (svm and logistic, each once opening on six second-class labels so the prior filter drops observations; logistic standardized; svm standardized with Class Names [2 7]), every element of the bus agrees with what Simulink's block logged within 10⁻¹¹ at the checkpoints, and IsWarm, CanPredict and MajorityClass at every step.