theanets.layers.recurrent.MRNN

class theanets.layers.recurrent.MRNN(factors=None, **kwargs)

Define a recurrent network layer using multiplicative dynamics.

The formulation of MRNN implemented here uses a factored dynamics matrix as described in Sutskever, Martens & Hinton, ICML 2011, “Generating text with recurrent neural networks.” This paper is available online at http://www.icml-2011.org/papers/524_icmlpaper.pdf.

__init__(factors=None, **kwargs)

Methods

__init__([factors])
add_weights(name, nin, nout[, mean, std, ...])	Helper method to create a new weight matrix.
initial_state(name, batch_size)	Return an array of suitable for representing initial state.
setup()	Set up the parameters and initial values for this layer.
to_spec()	Create a specification dictionary for this layer.
transform(inputs)	Transform the inputs for this layer into an output for the layer.

Attributes

input_size	For networks with one input, get the input size.
num_params	Total number of learnable parameters in this layer.
params	A list of all parameters in this layer.

setup()

Set up the parameters and initial values for this layer.

to_spec()

Create a specification dictionary for this layer.

Returns:

spec : dict

A dictionary specifying the configuration of this layer.

transform(inputs)

Transform the inputs for this layer into an output for the layer.

Parameters:

inputs : dict of theano expressions

Symbolic inputs to this layer, given as a dictionary mapping string names to Theano expressions. See base.Layer.connect().

Returns:

outputs : dict of theano expressions

A map from string output names to Theano expressions for the outputs from this layer. This layer type generates a “factors” output that gives the activation of the hidden weight factors given the input data (but not incorporating influence from the hidden states), a “pre” output that gives the unit activity before applying the layer’s activation function, and an “out” output that gives the post-activation output.

updates : list of update pairs

A sequence of updates to apply inside a theano function.