Gaussian Naive Bayes — Machine Learning/Classical Models

Gaussian Naive Bayes

Machine Learning / Classical Models

A fitted Gaussian naive Bayes classifier, evaluated one sample at a time. Each class scores the sample by its own per-feature Gaussian, and the highest score wins:

logp[k] = log(prior[k]) − ½ Σ_j [ log(2π·var[k][j]) + (u[j] − mu[k][j])² ÷ var[k][j] ]

It is also quadratic discriminant analysis. Set Covariance to Full (QDA), paste one inverse covariance per class into Class Precisions, and the per-feature sum above becomes the full quadratic form

logp[k] = log(prior[k]) − ½ log|Σ_k| − ½·d·log(2π) − ½ (u − mu_k)^T Σ_k⁻¹ (u − mu_k)

– which is sklearn.discriminant_analysis.QuadraticDiscriminantAnalysis written out. Naive Bayes is the special case where every Σ_k is diagonal, so the two settings are one block rather than two, and they agree exactly where they overlap: give the Full mode a diagonal precision and it lands on the Diagonal mode's numbers, term for term.

This is sklearn.naive_bayes.GaussianNB with theta_, var_ and class_prior_ pasted in. It is the cheapest probabilistic classifier there is – no matrix inverse, no kernel, no tree walk – and it reports a calibrated-ish score per class rather than only a decision, which is what lets a controller refuse to act when nothing scored well.

Everything logarithmic is folded at configuration time, so the sample loop is subtract, square and multiply-accumulate and nothing else. That is what keeps this block synthesizable in fixed point.

Ports

• u – the sample, a [d,1] column, where d is the number of COLUMNS of Class Means. One sample per step; it is checked rather than broadcast.
• idx – [1,1], the winning class, plus Index Base. This is what a controller switches on.
• logp – [K,1], every class's log-score in class order, where K is the number of ROWS of Class Means. The winner's own score is logp[idx], so nothing downstream has to recompute it; feed the whole vector to Softmax for posteriors, or to Confidence Gate or Top K for a decision with a threshold.

Parameters

• Class Means – mu, a [K,d] matrix, one class per ROW. This is sklearn's theta_ orientation, so it pastes in without transposing. Its shape decides both K and d.
• Class Variances – var, [K,d] to match: sklearn's var_. Per class AND per feature – that the features are treated as independent within a class is exactly what makes the model "naive". Read only when Covariance is Diagonal.
• Covariance – which of the two models this is:
  • Diagonal (naive Bayes) – each feature has its own variance within a class and the features are treated as independent. Reads Class Variances; Class Precisions is not read at all.
  • Full (QDA) – each class has a whole covariance, so features may correlate. Reads Class Precisions; Class Variances and Variance Floor are not read at all, and are left alone rather than having to be cleared.
• Class Precisions – the K inverse covariances Σ_k⁻¹, each [d,d], stacked one under another in class order as a [K·d,d] matrix: rows d·k to d·k+d−1 are class k. Read only when Covariance is Full.
  • It is the inverse, and this block inverts nothing – the same convention Mahalanobis Distance uses, and for the same reason: inverting a fitted covariance is a job for wherever the model was fitted, done once against the data, not once per configuration load in a block that must also write out ten literal tables.
  • Each block must be symmetric and positive definite, which the inverse of a genuine covariance always is. A non-positive determinant is refused with the class named rather than producing a score that is not a log-probability.
  • Only the upper triangle is used, with each off-diagonal coefficient carrying both of the symmetric pair – the same number for the symmetric matrix a precision always is, and half as many multiplies in hardware.
• Class Priors – [K,1], sklearn's class_prior_. Every entry must be greater than zero; a class with a zero prior can never win, and the log of it is not a number.
• Variance Floor – the smallest variance used, applied before the reciprocal is taken. sklearn's var_smoothing exists for the same reason: a feature that never varied in training has zero variance and would divide by zero here. It is applied ONCE at configuration load, so it costs the generated code nothing. Read only when Covariance is Diagonal.
• Index Base – what the first class is numbered, spelled exactly as Argmax Decision and K Means Assign spell it:
  • Zero-based (PyTorch, numpy) – the first class is 0.
  • One-based (MATLAB) – the first class is 1.
• Sampling Time (s) – zero or less inherits the solver's rate; a positive value runs the block at that period.

Code export

All ten targets: Python, MATLAB, Java, Rust, C, C++, VHDL, Verilog, SystemVerilog and PLC Structured Text. Every class's score is unrolled at export time, and the constants baked into it are the FOLDED ones – one c[k] per class carrying the log prior and all d log normalizers together, and one −½/var[k][j] per feature. No logarithm, no division and no exponential survives into the generated code.

Full (QDA) costs the generated code no new KIND of arithmetic, only more of the same. A class's score is a list of terms coefficient · (u[i] − mu[i]) · (u[j] − mu[j]); Diagonal is the case where every term has i = j. The log determinant folds into the same c[k] the log normalizers already folded into, so it too is gone before any code is written. The cost is the term count: d terms per class in Diagonal, d(d+1)/2 in Full, since only the upper triangle is emitted.

The three HDL targets are therefore genuine synthesizable Q16.16 in both settings, not the simulation-only real arithmetic this family's activation and distance blocks need: what is left in the datapath is a subtract, a multiply and an add per term, then a comparison chain.

Simulink bridge

None. Simulink's classifier blocks live in the Statistics and Machine Learning Toolbox, which is not installed on this machine – so there is no ClassificationNaiveBayes predict block here to map onto, and a bridge could not be verified even if it were written, since the parity suite would need that toolbox on the machine running it. Independently of that, those blocks take a fitted model OBJECT rather than parameter matrices, which no configuration value can carry across the bridge. The block therefore has no parity testbench, which is the documented consequence of Support::None. Code export verification still covers it across all ten languages.

Notes

• Stateless and algebraic: the answer depends on the current sample only.
• No state space: the score is quadratic in the input and the decision is a comparison, so no A/B/C/D describes it and model reduction correctly refuses the block.
• Ties go to the LOWEST class index. The scan keeps its running best and replaces it only on a strictly greater score – the same rule Argmax Decision states and K Means Assign mirrors for its argmin. Every backend spells the identical scan, because the samples that tie are exactly the ones where two implementations that both "find the maximum" disagree.
• The index is a DISCONTINUOUS function of the input. The three HDL targets compare Q16.16 values where the reference compares doubles, so two classes whose scores sit within one quantum (about 1.5×10⁻⁵) of each other can rank the other way round, and the index then differs by a whole step. That is a property of the block, not of the generated code; the logp output is continuous and does not have it.
• "Naive" is the Diagonal setting only. There the features are assumed independent within a class. When they are not, switch Covariance to Full (QDA) – or use Mahalanobis Distance for an unsupervised distance, or a Gaussian Mixture Model for several components per class.
• A linear discriminant (LDA) is not a setting here, and does not need to be. LDA is QDA with one covariance SHARED by every class, so it is this block with the same precision block pasted K times – and since the quadratic term is then identical for every class, it cancels in the comparison and the discriminant is linear, exactly as the textbook says.