theanets.graph.Network

class theanets.graph.Network(layers=(), loss='mse', weighted=False, rng=13)

The network class encapsulates a network computation graph.

Parameters:

layers : int, tuple, dict, or Layer

A sequence of values specifying the layer configuration for the network. For more information, please see Specifying Layers.

loss : str or Loss

The name of a loss function to optimize when training this network model.

weighted : bool, optional

If True, optimize this model using a “weighted” loss. Weighted losses typically require an additional array as input during optimization. For more information, see Using Weighted Targets. Defaults to False.

rng : int or RandomState, optional

A seed or numpy RandomState instance for generating randomness in the model. Defaults to 13.

Notes

Computation graphs are organized into layers. Each layer receives one or more arrays of input data, transforms them, and generates one or more arrays of output data.

Outputs in a computation graph are named according to their layer and output type, so the ‘pre’ output of a layer named ‘hid1’ would be named ‘hid1:pre’. The ‘out’ output is the default output for a layer. By default the last layer in a network is named ‘out’.

The parameters in a network graph are optimized by minimizing a loss function with respect to some set of training data. Typically the value produced by ‘out:out’ is compared to some target value, creating an error value of some sort. This error value is then propagated back through the computation graph to update the parameters in the model.

Attributes:

layers : list of Layer

A list of the layers in this network model.

losses : list of Loss

A list of losses to be computed when optimizing this network model.

__init__(layers=(), loss='mse', weighted=False, rng=13)

x.__init__(…) initializes x; see help(type(x)) for signature

Methods

__init__([layers, loss, weighted, rng])	x.__init__(…) initializes x; see help(type(x)) for signature
add_layer([layer])	Add a layer to our network graph.
add_loss([loss])	Add a loss function to the model.
build_graph([regularizers])	Connect the layers in this network to form a computation graph.
feed_forward(x, **kwargs)	Compute a forward pass of all layers from the given input.
find(which, param)	Get a parameter from a layer in the network.
itertrain(train[, valid, algo, subalgo, …])	Train our network, one batch at a time.
load(filename_or_handle)	Load a saved network from disk.
loss(**kwargs)	Return a variable representing the regularized loss for this network.
monitors(**kwargs)	Return expressions that should be computed to monitor training.
predict(x, **kwargs)	Compute a forward pass of the inputs, returning the network output.
save(filename_or_handle)	Save the state of this network to a pickle file on disk.
score(x, y[, w])	Compute R^2 coefficient of determination for a given labeled input.
set_loss(*args, **kwargs)	Clear the current loss functions from the network and add a new one.
train(*args, **kwargs)	Train the network until the trainer converges.
updates(**kwargs)	Return expressions to run as updates during network training.

Attributes

DEFAULT_OUTPUT_ACTIVATION	Default activation for the output layer.
INPUT_NDIM	Number of dimensions for holding input data arrays.
OUTPUT_NDIM	Number of dimensions for holding output data arrays.
inputs	A list of Theano variables for feedforward computations.
params	A list of the learnable Theano parameters for this network.
variables	A list of Theano variables for loss computations.

DEFAULT_OUTPUT_ACTIVATION = 'linear'

Default activation for the output layer.

INPUT_NDIM = 2

Number of dimensions for holding input data arrays.

OUTPUT_NDIM = 2

Number of dimensions for holding output data arrays.

add_layer(layer=None, **kwargs)

Add a layer to our network graph.

Parameters:

layer : int, tuple, dict, or Layer

A value specifying the layer to add. For more information, please see Specifying Layers.

add_loss(loss=None, **kwargs)

Add a loss function to the model.

Parameters:

loss : str, dict, or theanets.losses.Loss

A loss function to add. If this is a Loss instance, it will be added immediately. If this is a string, it names a loss function to build and add. If it is a dictionary, it should contain a 'form' key whose string value names the loss function to add. Other arguments will be passed to theanets.losses.Loss.build().

build_graph(regularizers=())

Connect the layers in this network to form a computation graph.

Parameters:

regularizers : list of theanets.regularizers.Regularizer

A list of the regularizers to apply while building the computation graph.

Returns:

outputs : list of Theano variables

A list of expressions giving the output of each layer in the graph.

updates : list of update tuples

A list of updates that should be performed by a Theano function that computes something using this graph.

feed_forward(x, **kwargs)

Compute a forward pass of all layers from the given input.

All keyword arguments are passed directly to build_graph().

Parameters:

x : ndarray (num-examples, num-variables)

An array containing data to be fed into the network. Multiple examples are arranged as rows in this array, with columns containing the variables for each example.

Returns:

layers : list of ndarray (num-examples, num-units)

The activation values of each layer in the the network when given input x. For each of the hidden layers, an array is returned containing one row per input example; the columns of each array correspond to units in the respective layer. The “output” of the network is the last element of this list.

find(which, param)

Get a parameter from a layer in the network.

Parameters:

which : int or str

The layer that owns the parameter to return.

If this is an integer, then 0 refers to the input layer, 1 refers to the first hidden layer, 2 to the second, and so on.

If this is a string, the layer with the corresponding name, if any, will be used.

param : int or str

Name of the parameter to retrieve from the specified layer, or its index in the parameter list of the layer.

Returns:

param : Theano shared variable

A shared parameter variable from the indicated layer.

Raises:

KeyError

If there is no such layer, or if there is no such parameter in the specified layer.

inputs

A list of Theano variables for feedforward computations.

itertrain(train, valid=None, algo='rmsprop', subalgo='rmsprop', save_every=0, save_progress=None, **kwargs)

Train our network, one batch at a time.

This method yields a series of (train, valid) monitor pairs. The train value is a dictionary mapping names to monitor values evaluated on the training dataset. The valid value is also a dictionary mapping names to values, but these values are evaluated on the validation dataset.

Because validation might not occur every training iteration, the validation monitors might be repeated for multiple training iterations. It is probably most helpful to think of the validation monitors as being the “most recent” values that have been computed.

After training completes, the network attribute of this class will contain the trained network parameters.

Parameters:

train : Dataset or list

A dataset to use when training the network. If this is a downhill.Dataset instance, it will be used directly as the training datset. If it is a list of numpy arrays or a list of callables, it will be converted to a downhill.Dataset and then used as the training set.

valid : Dataset or list, optional

If this is provided, it will be used as a validation dataset. If not provided, the training set will be used for validation. (This is not recommended!)

algo : str, optional

An optimization algorithm to use for training our network. If not provided, RMSProp will be used.

subalgo : str, optional

An optimization algorithm to use for a trainer that requires a “sub-algorithm,” sugh as an unsupervised pretrainer. Defaults to RMSProp.

save_every : int or float, optional

If this is nonzero and save_progress is not None, then the model being trained will be saved periodically. If this is a float, it is treated as a number of minutes to wait between savings. If it is an int, it is treated as the number of training epochs to wait between savings. Defaults to 0.

save_progress : str or file handle, optional

If this is not None, and save_progress is nonzero, then save the model periodically during training. This parameter gives either (a) the full path of a file to save the model, or (b) a file-like object where the model should be saved. If it is a string and the given name contains a “{}” format specifier, it will be filled with the integer Unix timestamp at the time the model is saved. Defaults to None, which does not save models.

Yields:

training : dict

A dictionary of monitor values computed using the training dataset, at the conclusion of training. This dictionary will at least contain a ‘loss’ key that indicates the value of the loss function. Other keys may be available depending on the trainer being used.

validation : dict

A dictionary of monitor values computed using the validation dataset, at the conclusion of training.

classmethod load(filename_or_handle)

Load a saved network from disk.

Parameters:

filename_or_handle : str or file handle

Load the state of this network from a pickle file. If this parameter is a string, it names the file where the pickle will be saved. If it is a file-like object, this object will be used for reading the pickle. If the filename ends in “.gz” then the output will automatically be gunzipped.

loss(**kwargs)

Return a variable representing the regularized loss for this network.

The regularized loss includes both the loss computation for the network as well as any regularizers that are in place.

Keyword arguments are passed directly to theanets.regularizers.from_kwargs().

Returns:

loss : Theano expression

A Theano expression representing the loss of this network.

monitors(**kwargs)

Return expressions that should be computed to monitor training.

Returns:

monitors : list of (name, expression) pairs

A list of named monitor expressions to compute for this network.

params

A list of the learnable Theano parameters for this network.

predict(x, **kwargs)

Compute a forward pass of the inputs, returning the network output.

All keyword arguments end up being passed to build_graph().

Parameters:

x : ndarray (num-examples, num-variables)

An array containing data to be fed into the network. Multiple examples are arranged as rows in this array, with columns containing the variables for each example.

Returns:

y : ndarray (num-examples, num-variables)

Returns the values of the network output units when given input x. Rows in this array correspond to examples, and columns to output variables.

save(filename_or_handle)

Save the state of this network to a pickle file on disk.

Parameters:

filename_or_handle : str or file handle

Save the state of this network to a pickle file. If this parameter is a string, it names the file where the pickle will be saved. If it is a file-like object, this object will be used for writing the pickle. If the filename ends in “.gz” then the output will automatically be gzipped.

score(x, y, w=None, **kwargs)

Compute R^2 coefficient of determination for a given labeled input.

Parameters:

x : ndarray (num-examples, num-inputs)

An array containing data to be fed into the network. Multiple examples are arranged as rows in this array, with columns containing the variables for each example.

y : ndarray (num-examples, num-outputs)

An array containing expected target data for the network. Multiple examples are arranged as rows in this array, with columns containing the variables for each example.

Returns:

r2 : float

The R^2 correlation between the prediction of this netork and its target output.

set_loss(*args, **kwargs)

Clear the current loss functions from the network and add a new one.

All parameters and keyword arguments are passed to add_loss() after clearing the current losses.

train(*args, **kwargs)

Train the network until the trainer converges.

All arguments are passed to itertrain().

Returns:

training : dict

A dictionary of monitor values computed using the training dataset, at the conclusion of training. This dictionary will at least contain a ‘loss’ key that indicates the value of the loss function. Other keys may be available depending on the trainer being used.

validation : dict

A dictionary of monitor values computed using the validation dataset, at the conclusion of training.

updates(**kwargs)

Return expressions to run as updates during network training.

Returns:

updates : list of (parameter, expression) pairs

A list of named parameter update expressions for this network.

variables

A list of Theano variables for loss computations.