# theanets.losses.CrossEntropy¶

class theanets.losses.CrossEntropy(target, weight=1.0, weighted=False, output_name='out')

Cross-entropy (XE) loss function for classifiers.

Parameters: target : int Number of dimensions required to store the target values for computing the loss. weight : float, optional The importance of this loss for the model being trained. Defaults to 1. weighted : bool, optional If True, a floating-point array of weights with the same dimensions as out_dim will be required to compute the “weighted” loss. Defaults to False. output_name : str, optional Name of the network output to tap for computing the loss. Defaults to ‘out:out’, the name of the default output of the last layer in a linear network.

Notes

The cross-entropy between a “true” distribution over discrete classes $$p(t)$$ and a “model” distribution over predicted classes $$q(x)$$ is the expected number of bits needed to store the model distribution, under the expectation of the true distribution. Mathematically, this loss computes:

$\mathcal{L}(x, t) = - \sum_{k=1}^K p(t=k) \log q(x=k)$

The loss value is similar to the KL divergence between $$p$$ and $$q$$, but it is specifically aimed at classification models. When using this loss, targets are assumed to be integers in the half-open interval $$[0, k)$$; internally, the loss is computed by first taking the log of the model distribution and then summing up only the entries in the resulting array corresponding to the true class.

Attributes

 weight (float, optional) The importance of this loss for the model being trained. output_name (str) Name of the network output to tap for computing the loss.
__init__(target, weight=1.0, weighted=False, output_name='out')

Methods

 __init__(target[, weight, weighted, output_name]) accuracy(outputs) Build a Theano expression for computing the accuracy of graph output. log() Log some diagnostic info about this loss.

Attributes

 variables A list of Theano variables used in this loss.
accuracy(outputs)

Build a Theano expression for computing the accuracy of graph output.

Parameters: outputs : dict of Theano expressions A dictionary mapping network output names to Theano expressions representing the outputs of a computation graph. acc : Theano expression A Theano expression representing the accuracy of the output compared to the target data.