Simple RNN Cell — Machine Learning/Neural Networks

Simple RNN Cell

Machine Learning / Neural Networks

One recurrent step of an Elman network, evaluated at inference:

h[k] = tanh( W_x·u[k] + W_h·h[k−1] + b )

This is torch.nn.RNNCell / Keras SimpleRNNCell with the trained weights pasted in. The hidden state is also the output, so a single block is a complete recurrent layer: feed it a sample per step and it carries context between them, which is what a plain Dense Layer cannot do.

Stateful and discrete by nature. A network step advances per sample and has no derivative to integrate, so the block declares itself discrete-only and always steps at its own rate rather than being pushed through the continuous solver.

Ports

• u – the input sample, a column [m,1]. m must equal the number of COLUMNS of Input Weights, and it is checked rather than broadcast.
• Output – h, a column [n,1] where n is the number of ROWS of Input Weights: the new hidden state. The layer's width is read off W_x alone, and the block resizes its output when you change it.

Parameters

• Input Weights – W_x, an [n,m] matrix. cell.weight_ih in PyTorch.
• Recurrent Weights – W_h, an [n,n] matrix. cell.weight_hh. This is what makes the block recurrent; a zero matrix here reduces it to a Dense Layer with a tanh.
• Bias – b, an [n,1] column. PyTorch keeps two bias vectors (bias_ih and bias_hh); they are only ever added together, so paste their sum.
• Initial Hidden State – h[−1], an [n,1] column: where the recurrence starts, re-applied at the start of every run. Zero is the usual choice for a freshly started sequence.
• Sampling Time (s) – zero or less inherits the solver's rate; a positive value runs the block at that period. Being discrete-only, a non-positive value falls back to the model's global sampling time rather than to the surrounding rate.

Code export

All ten targets: Python, MATLAB, Java, Rust, C, C++, VHDL, Verilog, SystemVerilog and PLC Structured Text. Both weight matrices, the bias and the initial state are baked in as literals at full setprecision(17), and each backend carries the hidden state in its own persistent form – a struct field, a core state member, a registered signal, a retained ST variable – seeded with Initial Hidden State.

The three HDL targets are simulation-only: they carry the step in real arithmetic and quantize only at the port boundary, because tanh is transcendental and has no place in a Q16.16 datapath.

PLC Structured Text has no TANH, so it is emitted through 2 ÷ (1 + EXP(−2z)) − 1 – the same function written in terms of EXP alone.

Simulink bridge

None. Simulink's recurrent layers live inside its Deep Learning blocks, which take a trained network object rather than weight matrices, and no config value can carry an object across the bridge. The bridge reports the block rather than dropping it silently, and it has no parity testbench, which is the documented consequence of Support::None rather than a gap. Code export verification still covers it across all ten languages.

Notes

• Stateful, so unlike every other block in this family it is not algebraic: the same input gives a different output depending on what came before. Chain it with a Dense Layer readout to build a small recurrent model.
• No state space, deliberately: tanh makes the recurrence nonlinear, so no A/B/C/D describes it and model reduction correctly refuses the block.
• The state is read before it is written. h[k] depends on every element of h[k−1], so an implementation that updated the vector in place would feed already-updated components into later rows. The generated code computes the new vector into a temporary and copies it back – except on the three HDL targets, where the state is a registered signal and pre-clock reads give this for free.
• Ten backends, one seed. The initial state written into every exported core comes from the configuration, never from wherever the in-app run had got to when you pressed export.