Quantized Dense Layer — Machine Learning/Neural Networks

Quantized Dense Layer

Machine Learning / Neural Networks

A dense layer evaluated the way a deployed int8 network evaluates it: the input is quantized to an 8-bit code, the products are accumulated as integers, and the result is requantized and scaled back to a real signal.

It is lossy on purpose. That is the whole reason to choose it over Dense Layer, which multiplies doubles: the question this block answers is whether a network still behaves once its weights are eight bits and its activations saturate – and on the PLC and HDL targets, whether it fits on the part at all.

The arithmetic, in order:

• nᵢ = clamp(round(uᵢ / Su), −128−Zu, 127−Zu) – the int8 input code.
• accᵢ = Bᵢ + Σ⫺ (Wᵢ⫺ − Zw) · n⫺ – an integer accumulator.
• yᵢ = Sy · clamp(round(Mᵢ·accᵢ), −128−Zy, 127−Zy), with Mᵢ = Su·Swᵢ/Sy – requantize, then back to a real signal.

Ports

• u – the real feature column, [m,1], where m is the NUMBER OF COLUMNS of Weights (int8).
• Output – y, a column [p,1], where p is the NUMBER OF ROWS of Weights (int8). It is a real signal again, and it takes only the 255 distinct values the output quantization allows.

Parameters

• Weights (int8) – [p,m], one output per ROW, and every entry a whole number in [−128, 127]. These are the quantized weights exactly as a framework exports them – not the real-valued ones, which belong in Dense Layer.
• Weight Zero Point – Zw, a whole number in [−128, 127]. Symmetric int8 weights – what TensorFlow Lite and PyTorch both produce – have Zw = 0, which is the default; the parameter exists for the asymmetric case.
• Weight Scale – Sw, strictly positive: either one scale for the whole tensor (a scalar) or one per output row ([p,1], which is per-channel quantization, the modern default).
• Bias (int32) – [p,1], whole numbers, expressed in accumulator units rather than in the output's units. That is how a quantized graph stores it: the bias is pre-scaled at Su·Swᵢ, so it adds straight into the integer accumulator with no conversion.
• Input Scale / Input Zero Point – Su (> 0) and Zu, the quantization of the incoming signal. Su is the size of one input code step, so it sets what range of input the layer can see before it saturates.
• Output Scale / Output Zero Point – Sy (> 0) and Zy, the quantization of the result. Sy is the size of one output step, and therefore the resolution of everything downstream.
• Sampling Time (s) – zero or less inherits the solver's rate; a positive value runs the block at that period.

Code export

All ten targets: Python, MATLAB, Java, Rust, C, C++, VHDL, Verilog, SystemVerilog and PLC Structured Text. Every parameter is structural and baked in; there is no tunable parameter, and the layer is fully unrolled.

Rounding is half away from zero in all ten – the convention Quantizer, MATLAB and TensorFlow Lite's reference kernels all use. Python and Java get an explicit form because np.round rounds halves to even and Math.round is a floor on a long. The zero points do not appear in the emitted arithmetic at all: they fold into the clamp bounds, exactly and not approximately.

The three HDL targets divide the accumulator by a power of two, because the integer accumulator does not fit: Q16.16 saturates at ±32768 and a single int8 product already reaches 127×255 = 32385. Each weight and the bias are divided by K, the smallest power of two that brings this layer's worst case under that ceiling, and Mᵢ·K is applied once at the end. Because K is a power of two, every divided weight, every product and the whole sum stay exact in Q16.16 – a small layer gets K = 1 and is bit-exact. There is still exactly one multiplier per weight and no divider anywhere.

What remains is that the input arrives already quantized to Q16.16, so a sample sitting within one quantum of a rounding tie can round the other way and shift that output by one Sy. That is inherent to a discontinuous output in fixed point – the same exposure Quantizer and Argmax Decision carry – rather than a codegen defect.

Simulink bridge

None. Simulink's Deep Learning blocks take a trained network object, and its fixed-point tooling quantizes a model rather than accepting an already-quantized one as parameters. The bridge reports the block rather than dropping it silently, and it has no parity testbench, which is the documented consequence of Support::None rather than a gap.

Notes

• Algebraic and stateless: the output depends only on the current input, so the block cannot break an algebraic loop.
• Nonlinear and discontinuous – rounding and saturation both are – and it deliberately carries no state space.
• Saturation is a result, not an error. An input beyond ±128·Su clamps, and so does an output beyond its own range; that is what the deployed network does, and seeing it here is the point of simulating the quantized form rather than the exact one.
• The output takes 255 distinct values. If a downstream block needs finer resolution than Sy, the answer is a smaller Sy or an unquantized Dense Layer, not a filter.