theanets.graph.Network

class theanets.graph.Network(layers, weighted=False, sparse_input=False)

The network class encapsulates a network computation graph.

In addition to defining standard functionality for common types of feedforward nets, there are also many options for specifying topology and regularization, several of which must be provided to the constructor at initialization time.

Parameters:

layers : sequence of int, tuple, dict, or Layer

A sequence of values specifying the layer configuration for the network. For more information, please see Specifying Layers.

weighted : bool, optional

If True, the network will require an additional input during training that provides weights for the target outputs of the network; the weights will be the last input argument to the network, and they must be the same shape as the target output.

This can be particularly useful for recurrent networks, where the length of each input sequence in a minibatch is not necessarily the same number of time steps, or for classifier networks where the prior proabibility of one class is significantly different than another. The default is not to use weighted outputs.

sparse_input : bool

If True, create an input variable that can hold a sparse matrix. Defaults to False, which assumes all arrays are dense.

Attributes

inputs	(list of theano variables) A list of the symbolic inputs this network expects during training.
layers	(list of Layer) A list of the layers in this network model.
weighted	(bool) True iff this network expects target weight inputs during training.

__init__(layers, weighted=False, sparse_input=False)

Methods

__init__(layers[, weighted, sparse_input])
add_layer(layer[, is_output])	Add a layer to our network graph.
build_graph(**kwargs)	Connect the layers in this network to form a computation graph.
error(outputs)	Build a theano expression for computing the network error.
feed_forward(x, **kwargs)	Compute a forward pass of all layers from the given input.
find(layer, param)	Get a parameter from a layer in the network.
load(filename)	Load a saved network from disk.
loss(**kwargs)	Return a variable representing the loss for this network.
monitors(**kwargs)	Return expressions that should be computed to monitor training.
output_name()	Get the fully-scoped name of the output for this network model.
predict(x)	Compute a forward pass of the inputs, returning the network output.
save(filename)	Save the state of this network to a pickle file on disk.
score(x, y[, w])	Compute R^2 coefficient of determination for a given labeled input.
updates(**kwargs)	Return expressions to run as updates during network training.

Attributes

num_params	Number of parameters in the entire network model.
params	A list of the learnable theano parameters for this network.

add_layer(layer, is_output=False)

Add a layer to our network graph.

Parameters:

layer : int, tuple, dict, or Layer

A value specifying the layer to add. For more information, please see Specifying Layers.

is_output : bool, optional

True iff this is the output layer for the graph. This influences the default activation function used for the layer: output layers in most models have a linear activation, while output layers in classifier networks default to a softmax activation.

build_graph(**kwargs)

Connect the layers in this network to form a computation graph.

Parameters:

noise : dict mapping str to float, optional

A dictionary that maps layer output names to standard deviation values. For an output “layer:output” in the graph, white noise with the given standard deviation will be added to the output. Defaults to 0 for all layer outputs.

dropout : dict mapping str to float in [0, 1], optional

A dictionary that maps layer output names to dropout values. For an output “layer:output” in the graph, the given fraction of units in the output will be randomly set to 0. Default to 0 for all layer outputs.

Returns:

outputs : list of theano variables

A list of expressions giving the output of each layer in the graph.

updates : list of update tuples

A list of updates that should be performed by a theano function that computes something using this graph.

error(outputs)

Build a theano expression for computing the network error.

Parameters:

outputs : dict mapping str to theano expression

A dictionary of all outputs generated by the layers in this network.

Returns:

error : theano expression

A theano expression representing the network error.

feed_forward(x, **kwargs)

Compute a forward pass of all layers from the given input.

All keyword arguments are passed directly to build_graph().

Parameters:

x : ndarray (num-examples, num-variables)

An array containing data to be fed into the network. Multiple examples are arranged as rows in this array, with columns containing the variables for each example.

Returns:

layers : list of ndarray (num-examples, num-units)

The activation values of each layer in the the network when given input x. For each of the hidden layers, an array is returned containing one row per input example; the columns of each array correspond to units in the respective layer. The “output” of the network is the last element of this list.

find(layer, param)

Get a parameter from a layer in the network.

Parameters:

layer : int or str

The layer that owns the parameter to return.

If this is an integer, then 0 refers to the input layer, 1 refers to the first hidden layer, 2 to the second, and so on.

If this is a string, the layer with the corresponding name, if any, will be used.

param : int or str

Name of the parameter to retrieve from the specified layer, or its index in the parameter list of the layer.

Returns:

param : theano shared variable

A shared parameter variable from the indicated layer.

Raises:

KeyError

If there is no such layer, or if there is no such parameter in the specified layer.

classmethod load(filename)

Load a saved network from disk.

Parameters:

filename : str

Load the state of a network from a pickle file at the named path. If this name ends in ”.gz” then the input will automatically be gunzipped; otherwise the input will be treated as a “raw” pickle.

loss(**kwargs)

Return a variable representing the loss for this network.

The loss includes both the error for the network as well as any regularizers that are in place.

Parameters:

weight_l1 : float, optional

Regularize the L1 norm of unit connection weights by this constant.

weight_l2 : float, optional

Regularize the L2 norm of unit connection weights by this constant.

hidden_l1 : float, optional

Regularize the L1 norm of hidden unit activations by this constant.

hidden_l2 : float, optional

Regularize the L2 norm of hidden unit activations by this constant.

contractive : float, optional

Regularize model using the Frobenius norm of the hidden Jacobian.

noise : float, optional

Standard deviation of desired noise to inject into input.

dropout : float in [0, 1], optional

Proportion of input units to randomly set to 0.

Returns:

loss : theano expression

A theano expression representing the loss of this network.

monitors(**kwargs)

Return expressions that should be computed to monitor training.

Returns:

monitors : list of (name, expression) pairs

A list of named monitor expressions to compute for this network.

num_params

Number of parameters in the entire network model.

output_name()

Get the fully-scoped name of the output for this network model.

Returns:

name : str

A name of the form “{layer}:{output}” that indicates the overall output for the network.

params

A list of the learnable theano parameters for this network.

predict(x)

Compute a forward pass of the inputs, returning the network output.

Parameters:

x : ndarray (num-examples, num-variables)

An array containing data to be fed into the network. Multiple examples are arranged as rows in this array, with columns containing the variables for each example.

Returns:

y : ndarray (num-examples, num-variables)

Returns the values of the network output units when given input x. Rows in this array correspond to examples, and columns to output variables.

save(filename)

Save the state of this network to a pickle file on disk.

Parameters:

filename : str

Save the state of this network to a pickle file at the named path. If this name ends in ”.gz” then the output will automatically be gzipped; otherwise the output will be a “raw” pickle.

score(x, y, w=None)

Compute R^2 coefficient of determination for a given labeled input.

Parameters:

x : ndarray (num-examples, num-inputs)

An array containing data to be fed into the network. Multiple examples are arranged as rows in this array, with columns containing the variables for each example.

y : ndarray (num-examples, num-outputs)

An array containing expected target data for the network. Multiple examples are arranged as rows in this array, with columns containing the variables for each example.

Returns:

r2 : float

The R^2 correlation between the prediction of this netork and its target output.

updates(**kwargs)

Return expressions to run as updates during network training.

Returns:

updates : list of (parameter, expression) pairs

A list of named parameter update expressions for this network.