Softmax — Machine Learning/Neural Networks
Machine_Learning/Neural_Networks/Softmax · 1 input / 1 output port(s) at insert · exports to Python, MATLAB, Java, Rust, C, C++, VHDL, Verilog, SystemVerilog, PLC Structured Text
Description#
The block's own DESCRIPTION_HTML, rendered verbatim — the same text the config dialog's info panel and the library navigator show. Fix a wrong sentence in the block's .cpp (R-D9), never here.
Softmax
Machine Learning / Neural Networks
Turns a column of logits into a probability vector:
yi = ezi ÷ Σj ezj, where zi = (ui − max(u)) ÷ T
Every output is in (0, 1) and they sum to exactly 1, so this is the head of a classifier: put it after the final Dense Layer, and follow it with Argmax Decision if what you need is the class rather than the distribution.
Subtracting max(u) changes nothing mathematically – softmax is exactly invariant to a constant added to every logit – and is what keeps the exponentials from overflowing on a saturated network. See Notes.
Ports
- u – the logits, a column [m,1] with m ≥ 1. The whole column is reduced, so every entry affects every output.
- Output – y, the same [m,1] column, summing to 1.
Parameters
- Temperature – T, the divisor applied to the logits before exponentiating. 1.0 is the plain softmax. Values below 1 sharpen the distribution towards a one-hot vector; values above 1 flatten it towards uniform. It must be strictly positive – a zero or negative temperature is reported rather than run, since T → 0 is the argmax limit and is what the Argmax Decision block is for.
- Sampling Time (s) – zero or less inherits the solver's rate; a positive value runs the block at that period.
Code export
All ten targets: Python, MATLAB, Java, Rust, C, C++, VHDL, Verilog, SystemVerilog and PLC Structured Text. The temperature is baked into the body at export time; there is no tunable parameter.
The three HDL targets are simulation-only: they carry the
exponentials and the division in real arithmetic and quantize only
at the port boundary. Neither belongs in a Q16.16 datapath.
Simulink bridge
None. Simulink's softmax lives inside its Deep Learning blocks, which
take a trained network object rather than a signal, and no config value
can carry an object across the bridge. The bridge reports the block rather than
dropping it silently, and it has no parity testbench, which is the
documented consequence of Support::None rather than a gap. Code
export verification still covers it across all ten languages.
Notes
- Algebraic and stateless: the output depends only on the current input, so the block cannot break an algebraic loop.
- Nonlinear and coupling, and deliberately carries no state space: every output depends on every input through the reduction, so no A/B/C/D describes it and model reduction correctly refuses the block.
- Why the max is subtracted.
expoverflows to infinity beyond an argument of about 709 in double precision, and much sooner in the narrower arithmetics an exported core can land in. Subtracting the largest logit bounds every exponent argument at 0 and makes the largest term exactly 1, without changing the result at all. Skipping it does not produce a slightly wrong number – it produces ∞÷∞, which is NaN. - A constant input column comes out uniform: every logit equal means every exponent equal, so each output is 1÷m. That is the defined behaviour, not a degenerate case.
Code facts#
| Fact | Value |
|---|---|
| registered type | Machine_Learning/Neural_Networks/Softmax |
| family | Machine_Learning/Neural_Networks |
| solver environment class | ICoreBlock_0_Machine_Learning_1_Neural_Networks_2_Softmax |
| source | src/ICoreSDK/ICoreBlockLibrary/Blocks/Machine_Learning/Neural_Networks/Softmax/ICoreBlock_0_Machine_Learning_1_Neural_Networks_2_Softmax.cpp |
| header | src/ICoreSDK/ICoreBlockLibrary/Blocks/Machine_Learning/Neural_Networks/Softmax/ICoreBlock_0_Machine_Learning_1_Neural_Networks_2_Softmax.h |
| default size on canvas | 110 × 80 px |
| ports at insert | 1 in, 1 out |
| code generators implemented | Python, MATLAB, Java, Rust, C, C++, VHDL, Verilog, SystemVerilog, PLC Structured Text |
Ports#
| # | Direction | Signal type | Description label |
|---|---|---|---|
| 1 | in | ICoreDouble | u |
| 2 | out | ICoreDouble | y |
Ports the constructor creates. A block whose port list changes with its configuration adds or removes ports at load time; the count above is the one a freshly inserted block has.
Configuration variables#
| Config variable | Default | Simulink parameter |
|---|---|---|
Temperature | 1.0 | — |
Every block also carries Sampling Time (s) from ICoreBlockSolverEnvironment: zero or less inherits the solver's rate, a positive value runs the block at that period.
Simulink bridge#
| support | Support::None |
| Simulink path | — |
| port-count rule | PortsParam::None |
SampleTime parameter | yes |
Caveat (shown to the user): no Simulink equivalent as a signal block: its softmax exists only inside the Deep Learning blocks, which take a trained network OBJECT rather than a logits signal, and no config value crosses the bridge as an object
Catalog contract: src/ICoreSDK/ICoreCoder/ICoreCommandSystem/SimulinkBridge/ICoreSimulinkBlockCatalog.h
Description vs code#
The lists agree. check_block_descriptions.py finds no disagreement between the description's Ports, Parameters, Code export and Simulink bridge lists and the code's.
The verdict above is
tools/docs/check_block_descriptions.py(P7.1), which compares LISTS. It cannot read a sentence: "stateless" on a block with a state, an initial-value semantic the recursion does not implement, a "not synthesizable" caveat the HDL banner contradicts. That is the agent audit (P7.3) on BLOCK_DESCRIPTION_AUDIT.md, and this tool's green is not a substitute for one.
File banner (developer view)#
The top comment of the block's .cpp — the maths, the realization and the export strategy, addressed to whoever changes it. It must not contradict the description above (P7.5).
Softmax — logits to a probability vector, with a temperature z_i = (u_i - max(u)) / T, y_i = exp(z_i) / sum_j exp(z_j)
Not elementwise: the denominator reduces the whole column, so this is its own block rather than a mode of Activation_Function. See the header for why the max subtraction is a correctness requirement rather than a nicety.
The three HDL targets are SIMULATION-ONLY
realarithmetic, as with the rest of this family: an exponential and a division per sample do not belong in a Q16.16 datapath. There is no way to declare arealscratch variable in a VHDL block body, so the sum of exponentials is INLINED once per output -- m exp() calls per output, m^2 in the emitted text. That is acceptable at the widths a classifier head has and is stated here rather than discovered by whoever reads the generated VHDL.
Sample results#
| t | in ICoreDouble-Out-0 | out ICoreDouble-Out-0 |
|---|---|---|
| 0 | -2 | 1 |
| 0.4 | 0.5 | 1 |
| 0.8 | -2 | 1 |
| 1.2 | 0.5 | 1 |
| 1.6 | -2 | 1 |
| 2 | 0.5 | 1 |
| 2.4 | -2 | 1 |
| 2.8 | 0.5 | 1 |
| 3.2 | -2 | 1 |
| 3.6 | 0.5 | 1 |
| 4 | -2 | 1 |
| 4.4 | 0.5 | 1 |
| 4.8 | -2 | 1 |
| 5.2 | 0.5 | 1 |
Every 4th of 60 samples, from the table stimulus.
The same rig also ran:
| Stimulus | What it is | Output range |
|---|---|---|
impulse | Impulse: one sample of 1 at k = 5, 0 elsewhere (Repeating Sequence Stair) | 1 … 1 |
ramp | Ramp: slope 1 from t = 0 | 1 … 1 |
sine | Sine Wave: amplitude 1, 2 rad/s, no phase, no bias | 1 … 1 |
step | Step: 0 -> 1 at t = 1 s | 1 … 1 |
Plotted: table — Repeating Sequence Stair: [-2 -1 -0.5 0 0.5 1 2 3], one entry per sample
Category static · sample time 0.1 · 60 steps · commit ccf005c8 · produced by docsSample --out <folder> --steps 60 · data docs/generated/samples/Machine_Learning__Neural_Networks__Softmax.json · the SVG is generated from those numbers by tools/docs/plot_svg.py, so it is a run and not a drawing (R-D10).