Gaussian Mixture Model — Machine Learning/Classical Models

Gaussian Mixture Model

Machine Learning / Classical Models

A fitted Gaussian mixture, evaluated one sample at a time. Each component scores the sample by its own full-covariance Gaussian:

logp[k] = log(w[k]) − ½·d·log(2π) + ½log(det Sinv[k]) − ½·(u − mu[k])'· Sinv[k]·(u − mu[k])

This is sklearn.mixture.GaussianMixture with weights_, means_ and precisions_ pasted in. It is Gaussian Naive Bayes with a full covariance: that block's per-feature variances make the quadratic form diagonal, and carrying the off-diagonal terms is exactly the difference between "naive" and a mixture component.

Use it as a soft clustering (which component does this sample belong to, and how strongly) or as a density model (how likely is this sample at all, which is what makes a mixture a novelty detector).

Ports

• u – the sample, a [d,1] column, d being the number of COLUMNS of Component Means.
• idx – [1,1], the winning component, plus Index Base. It is always present, whatever Output is set to.
• p – [K,1], one entry per component: either their log-likelihoods or their responsibilities, per Output. K is the number of ROWS of Component Means.

Parameters

• Component Means – mu, a [K,d] matrix, one component per ROW: sklearn's means_ orientation. Its shape decides both K and d.
• Inverse Covariances – the precision matrices, STACKED VERTICALLY as [K·d, d]: component k occupies rows k·d to k·d+d−1. A configuration matrix is 2-D, so a [K,d,d] tensor needs a flattening convention and this is it. From sklearn, numpy.vstack(gm.precisions_) produces exactly this layout. Each block must be symmetric and positive definite – both are checked, because a transposed or mis-stacked paste is otherwise a silently different model.
• Component Weights – [K,1], sklearn's weights_. Every entry must be greater than zero.
• Output – what the p port carries. The two are different code paths rather than a scaling, so each is verified as its own mode:
  • Log Likelihood – the raw per-component log-likelihoods, in component order. Their differences are what matter; the winner's own value is p[idx]. This is the mode to use for novelty scoring.
  • Responsibility – the posterior probability of each component, non-negative and summing to one: the E-step of the algorithm that fitted the model. This is the mode to use for soft clustering.
• Index Base – what the first component is numbered, spelled exactly as Argmax Decision, K Means Assign and both Naive Bayes blocks spell it:
  • Zero-based (PyTorch, numpy) – the first component is 0.
  • One-based (MATLAB) – the first component is 1.
• Sampling Time (s) – zero or less inherits the solver's rate; a positive value runs the block at that period.

Code export

All ten targets: Python, MATLAB, Java, Rust, C, C++, VHDL, Verilog, SystemVerilog and PLC Structured Text. Every component's quadratic form is unrolled at export time, and the constant baked in front of it is the FOLDED one: the log weight, the d/2 log(2π) and the half log-determinant together, computed once when the configuration loads. No logarithm and no determinant reaches the generated code.

The two modes differ in the HDL targets: Log Likelihood is genuine synthesizable Q16.16 – a subtract, two multiplies and an add per term, then a comparison chain. Responsibility exponentiates and divides by a sum that depends on the sample, neither of which belongs in a Q16.16 datapath, so the three HDL targets carry that mode in simulation-only real arithmetic.

In that mode the HDL cores also exponentiate the quantized log-likelihood, since the scores pass through the fixed-point output vector on their way; the difference is about one quantum in the exponent and stays far inside the verification band.

Simulink bridge

None. Simulink's mixture and clustering blocks live in the Statistics and Machine Learning Toolbox, which is not installed on this machine, so there is nothing here to map onto and a bridge could not be verified even if it were written. Those blocks also take a fitted model OBJECT rather than parameter matrices. The block therefore has no parity testbench, which is the documented consequence of Support::None.

Notes

• Stateless and algebraic.
• The maximum is subtracted before exponentiating, exactly as Softmax does and for the same reason: a component log-likelihood of −40 is entirely ordinary, and exp(−40) underflows to zero in single precision and in Q16.16 alike. Subtracting the winner makes the largest term exactly 1 and costs nothing, since the winner has already been found for the idx port.
• Ties go to the LOWEST component index – Argmax Decision's rule, spelled identically in every backend.
• The index is a DISCONTINUOUS function of the input, so the three HDL targets can rank two near-equal components the other way round; one Q16.16 quantum is about 1.5×10⁻⁵. The p output is continuous and does not have that property.
• No matrix is inverted here. The precisions are configuration, as they are for Mahalanobis Distance – no block body inverts a matrix, and sklearn hands you precisions_ already.
• For a single component this block is a log-density, and Mahalanobis Distance is the same quadratic form without the normalizer. For a DIAGONAL covariance use Gaussian Naive Bayes, which carries d terms instead of d².