theanets.layers.recurrent.Recurrent

class theanets.layers.recurrent.Recurrent(**kwargs)

A recurrent network layer incorporates some dependency on past values.

In many respects, a recurrent network layer is much like a basic feedforward layer: both layers take an input signal, apply some transformation to it, and produce an output signal. Recurrent layers, however, additionally preserve the previous state(s) of the layer’s output and incorporate them into the transformation of the current input.

This layer type is actually just a base class for the many different types of recurrent network layers, for example RNN or LSTM.

Recurrent layer types can only be included in theanets models that use recurrent inputs and outputs, i.e., theanets.recurrent.Autoencoder, theanets.recurrent.Predictor, theanets.recurrent.Classifier, or theanets.recurrent.Regressor.

Parameters:

radius : float, optional

If given, rescale the initial weights for the recurrent units to have this spectral radius. No scaling is performed by default.

direction : {None, ‘back’, ‘backwards’}, optional

If given, this string indicates whether the recurrency for this layer should run “backwards”, with future states influencing the current state. The default is None, which runs the recurrency forwards in time so that past states influence the current state of the layer.

bptt_limit : int, optional

If given, limit backpropagation of gradient information in scans (loops) to the given number of time steps. Defaults to -1, which imposes no limit.

__init__(**kwargs)

Methods

__init__(**kwargs)
add_weights(name, nin, nout[, mean, std, ...])	Helper method to create a new weight matrix.
initial_state(name, batch_size)	Return an array of suitable for representing initial state.

Attributes

input_size	For networks with one input, get the input size.
num_params	Total number of learnable parameters in this layer.
params	A list of all parameters in this layer.

add_weights(name, nin, nout, mean=0, std=0, sparsity=0, radius=0, diagonal=0)

Helper method to create a new weight matrix.

Parameters:

name : str

Name of parameter to define.

nin : int, optional

Size of “input” for this weight matrix. Defaults to self.nin.

nout : int, optional

Size of “output” for this weight matrix. Defaults to self.nout.

mean : float, optional

Mean of initial matrix values. Defaults to 0.

std : float, optional

Standard deviation of initial matrix values. Defaults to \(1 / sqrt(n_i + n_o)\).

sparsity : float, optional

Fraction of weights to set randomly to zero. Defaults to 0.

radius : float, optional

If nonzero, rescale initial weights to have this spectral radius. Defaults to 0.

initial_state(name, batch_size)

Return an array of suitable for representing initial state.

Parameters:

name : str

Name of the variable to return.

batch_size : int

Number of elements in a batch. This can be symbolic.

Returns:

initial : theano shared variable

A variable containing the initial state of some recurrent variable.