theanets.losses.CrossEntropy

class theanets.losses.CrossEntropy(target, weight=1.0, weighted=False, output_name='out')

Cross-entropy (XE) loss function for classifiers.

Parameters:

target : int

Number of dimensions required to store the target values for computing the loss.

weight : float, optional

The importance of this loss for the model being trained. Defaults to 1.

weighted : bool, optional

If True, a floating-point array of weights with the same dimensions as out_dim will be required to compute the “weighted” loss. Defaults to False.

output_name : str, optional

Name of the network output to tap for computing the loss. Defaults to ‘out:out’, the name of the default output of the last layer in a linear network.

Notes

The cross-entropy between a “true” distribution over discrete classes \(p(t)\) and a “model” distribution over predicted classes \(q(x)\) is the expected number of bits needed to store the model distribution, under the expectation of the true distribution. Mathematically, this loss computes:

\[\mathcal{L}(x, t) = - \sum_{k=1}^K p(t=k) \log q(x=k)\]

The loss value is similar to the KL divergence between \(p\) and \(q\), but it is specifically aimed at classification models. When using this loss, targets are assumed to be integers in the half-open interval \([0, k)\); internally, the loss is computed by first taking the log of the model distribution and then summing up only the entries in the resulting array corresponding to the true class.

Attributes

weight	(float, optional) The importance of this loss for the model being trained.
output_name	(str) Name of the network output to tap for computing the loss.

__init__(target, weight=1.0, weighted=False, output_name='out')

Methods

__init__(target[, weight, weighted, output_name])
accuracy(outputs)	Build a Theano expression for computing the accuracy of graph output.
log()	Log some diagnostic info about this loss.

Attributes

variables

A list of Theano variables used in this loss.

accuracy(outputs)

Build a Theano expression for computing the accuracy of graph output.

Parameters:

outputs : dict of Theano expressions

A dictionary mapping network output names to Theano expressions representing the outputs of a computation graph.

Returns:

acc : Theano expression

A Theano expression representing the accuracy of the output compared to the target data.