LSTM Cell — Machine Learning/Neural Networks

LSTM Cell

Machine Learning / Neural Networks

One long short-term memory step, evaluated at inference – torch.nn.LSTMCell:

i = σ(W_iiu + b_ii + W_hih + b_hi)   f = σ(W_ifu + b_if + W_hfh + b_hf)
g = tanh(W_igu + b_ig + W_hgh + b_hg)   o = σ(W_iou + b_io + W_hoh + b_ho)
c' = f·c + i·g    h' = o·tanh(c')

The cell state c is the long-term path: the forget gate scales what is already there and the input gate decides how much of the candidate joins it, with no squashing in between. That additive path is why an LSTM holds information over far more steps than a Simple RNN Cell, whose state passes through a tanh every step.

Ports

• u – the input sample, a column [m,1], m being the COLUMNS of Input Weights.
• h – the new hidden state, [n,1], where n is the ROWS of Input Weights divided by four. This is the cell's output in the usual sense.
• c – the new cell state, [n,1]. Exposed because LSTMCell returns it and a stacked cell needs it; ignore it if you are using one cell as a layer.

Parameters

• Input Weights – [4n,m], the four input matrices stacked i, f, g, o – cell.weight_ih's own layout, so it pastes in without splitting.
• Recurrent Weights – [4n,n], stacked the same way (cell.weight_hh).
• Input Bias – [4n,1] (cell.bias_ih).
• Recurrent Bias – [4n,1] (cell.bias_hh). Unlike GRU Cell, these two could be added together without changing the answer – see Notes – but they are taken separately so a PyTorch export pastes straight in.
• Initial Hidden State – [n,1].
• Initial Cell State – [n,1]. A trained model almost always starts both at zero; set them otherwise only if you are resuming a sequence.
• Sampling Time (s) – zero or less inherits the solver's rate; a positive value runs the block at that period. Being discrete-only, a non-positive value falls back to the model's global sampling time.

Code export

All ten targets: Python, MATLAB, Java, Rust, C, C++, VHDL, Verilog, SystemVerilog and PLC Structured Text. Every weight is baked in at full setprecision(17); both state vectors are carried in each backend's own persistent form, seeded from the two initial-state configs.

The three HDL targets are simulation-only: sigmoid and tanh are transcendental. PLC Structured Text has no TANH, so it is emitted through 2 ÷ (1 + EXP(−2z)) − 1.

Simulink bridge

None. Simulink's LSTM lives inside its Deep Learning blocks, which take a trained network object rather than weight matrices, and no config value can carry an object across the bridge. The bridge reports the block rather than dropping it silently, and it has no parity testbench, which is the documented consequence of Support::None.

Notes

• Why the biases are separate even though they need not be. Every LSTM gate only ever adds its two biases, so folding them is exact – unlike GRU Cell, where the reset gate multiplies one of them and folding is wrong. They are kept apart here for convenience and consistency, not correctness: a PyTorch export gives you bias_ih and bias_hh and you paste both.
• h' depends on the NEW c, not the old one. That is the one ordering this block must get right, and every backend computes c' first and feeds it into tanh for h'.
• Stateful and discrete by nature, so the block declares itself discrete-only, and carries no state space – the gates are nonlinear.
• Both states are read before either is written. The generated code builds the whole new h and c before storing either, except on the three HDL targets where registered state gives that for free.