ECOC Classifier Predictor — Machine Learning/Classical Models

ECOC Classifier Predictor

Machine Learning / Classical Models

Evaluates a fitted multiclass error-correcting output codes model at inference – what MATLAB's [label, NegLoss, PBScore] = predict(Mdl, x) returns for a ClassificationECOC model (trained with fitcecoc) and one observation. Each of L binary learners scores x; each of K classes is scored by how well those scores agree with its row of the coding matrix M:

s_l = T(f_l(x)),   loss_k = mean_j g(M_kj·s_j),   NegLoss = −loss,   label = ClassNames(k) for the first k with the largest NegLoss – where f_l is the l-th learner's score, T the learners' score transform and g the binary loss.

Each learner is a kernel machine: z = (x − Mu_l) ./ Sigma_l, f_l = Σ_i Alpha_i·K(z/s_l, SV_i/s_l) + Bias_l, or (z/s_l)·Beta_l + Bias_l for the linear kernel. The model is baked in as configuration, pasted from the fitted object; nothing is fitted here and no model file is read.

Supported binary learners

• SVM learners (fitcecoc's default, and templateSVM) with the linear, gaussian or polynomial kernel, standardised or not, each learner with its own kernel scale, Mu and Sigma.
• Linear learners (templateLinear, ClassificationLinear): the linear kernel with Kernel Scale 1, Mu 0 and Sigma 1; Binary Score Transform logit for the logistic learner, none for the svm learner.

Not supported: kernel learners (templateKernel, a random feature expansion – a different model), tree, KNN, naive Bayes, discriminant and ensemble learners (each a different model representation), SVM learners fitted with 'FitPosterior' (their step or sigmoid transforms are per-learner functions), custom kernels and custom binary losses (a function handle cannot be pasted as a number), and the posterior-probability output (a per-observation optimisation).

Ports

• x – one observation, a column [D,1] with one predictor per row, in the order the model was trained on.
• label – the predicted class, a scalar [1,1]: one of the Class Names, as a number.
• negloss – the negated average binary loss per class, a column [K,1] in Class Names order: MATLAB's NegLoss, as a column.
• pbscore – each binary learner's positive-class score s, a column [L,1]: MATLAB's PBScore, as a column.

Parameters

• Coding Matrix – Mdl.CodingMatrix, [K,L], entries −1, 0 or +1: row k is class k, column l is learner l. Every design fitcecoc and designecoc produce is accepted (onevsone, onevsall, binarycomplete, ternarycomplete, ordinal, denserandom, sparserandom), since the design is only the matrix. Every row needs at least one nonzero entry.
• Class Names – Mdl.ClassNames, K numbers in the model's own order (the order of the coding matrix's rows). Text or categorical labels are not numbers: use their positions, 1..K, and map them downstream.
• Kernel Function – the learners' KernelParameters.Function, shared by all of them as a templateSVM makes it:
  • linear – K = a·b. The default, as in fitcecoc. The learners are then given by Beta.
  • gaussian – K = exp(−‖a − b‖²) (MATLAB's 'rbf' is the same kernel).
  • polynomial – K = (a·b + 1)^Order.
• Beta – [D,L], column l the l-th learner's Beta. Used by linear only. fitcecoc's linear SVM learners discard their support vectors and keep only this.
• Support Vectors – [N,D], every learner's SupportVectors stacked in learner order (standardised, as MATLAB stores them). Used by gaussian and polynomial.
• Alpha – N signed coefficients stacked the same way: each learner's Alpha .* SupportVectorLabels.
• Support Vector Counts – L whole numbers, how many of the stacked rows belong to each learner.
• Bias – L values, each learner's Bias.
• Kernel Scale – L values, each learner's KernelParameters.Scale ('auto' gives every learner its own), or one value for all. Strictly positive.
• Polynomial Order – a whole number from 1 to 32. Used by polynomial only.
• Mu – [L,D], row l the l-th learner's Mu: standardised one-vs-one learners each carry their own, since each saw only its own classes. One row applies to every learner; 0 means none (an unstandardised model).
• Sigma – [L,D] likewise, or one row, or 1 for none. A predictor whose Sigma is 0 is centred but not divided, as in MATLAB.
• Binary Score Transform – the learners' ScoreTransform:
  • none – s = f, unbounded. SVM learners, and svm-learner linear ones.
  • logit – s = 1/(1 + e^−f), a probability. Logistic linear learners.
• Binary Loss – g, MATLAB's 'BinaryLoss' (the model's BinaryLoss is its default: hinge for SVM learners, quadratic for logistic ones), with y = M_kj·s_j:
  • hinge – max(0, 1 − y)/2.
  • hamming – (1 − sign(y))/2; with logit scores, (1 − sign(M·(2s − 1)))/2.
  • linear – (1 − y)/2.
  • quadratic – (1 − M·(2s − 1))²/2.
  • exponential – exp(−y)/2.
  • binodeviance – log(1 + exp(−2y))/(2·log 2).
  • logit – log(1 + exp(−y))/(2·log 2).
  As in MATLAB, quadratic needs logit scores and linear, exponential, binodeviance, hinge and logit need none; hamming takes either. Any other pairing is refused.
• Decoding – MATLAB's 'Decoding':
  • lossweighted – the mean over the learners whose coding entry for the class is nonzero. The default.
  • lossbased – the mean over all L learners, a zero entry contributing g(0).
  With no zero in the coding matrix (onevsall, binarycomplete) the two agree.
• Sampling Time (s) – zero or less inherits the solver's rate; a positive value runs the block at that period.

Code export

All ten targets: Python, MATLAB, Java, Rust, C, C++, VHDL, Verilog, SystemVerilog and PLC Structured Text. The whole model is baked into the body at export time; there is no tunable parameter, so a refitted model means a re-export. The software targets evaluate exactly the arithmetic above, in the same order. The three HDL bodies are simulation-only real arithmetic in every kernel, loss and decoding, quantized at the ports only: the decoder is a mean over each class's coding row followed by an argmax, and five of the seven losses and the gaussian kernel need an exponential or a logarithm. They are not offered as synthesizable.

Simulink bridge

None. Simulink's ClassificationECOC Predict block (Statistics and Machine Learning Toolbox) exists, and its BinaryLoss and Decoding are dialog parameters, but its model parameter, TrainedLearner, is the name of a fitted model object in the MATLAB workspace. That object cannot be built from a coding matrix and the learners' coefficients – ClassificationECOC's constructor refuses direct use and its binary learners' are not public – and a parameter mapping carries one value to one parameter, so neither direction can cross. The bridge reports the block rather than dropping it silently, and it has no parity testbench, which is the documented consequence of that. The arithmetic is checked against MATLAB's own predict instead, and code export verification covers all ten languages. The Simulink block also has no SampleTime parameter.

Notes

• Algebraic and stateless: the outputs depend only on the current input.
• A tie goes to the earlier class in Class Names order – MATLAB takes the first maximum of NegLoss. Ties are common under hamming, whose losses take a handful of values.
• Prior and Cost do not enter the label. MATLAB uses the prior only for an observation whose losses are all NaN, which it labels with the most probable class; here a NaN in x makes every loss NaN and the label ClassNames(1).
• The label is discontinuous, so the three HDL targets, which quantize x to Q16.16, can disagree with the simulation on a sample within about 1.5×10⁻⁵ of a decision boundary: the label then differs by the distance between two class names. That is inherent to quantizing the input, not an export fault; take negloss downstream if an application cannot tolerate it.
• Carries no state space: the label is a discontinuous function of x.