Using a Model¶
Once you’ve trained a model, you will probably want to do something useful with it. If you are working in a production environment, you might want to use the model to make predictions about incoming data; if you are doing research, you might want to examine the parameters that the model has learned.
Predicting New Data¶
For most neural network models, you can compute the “natural” output of the model
layer by calling Network.predict()
:
results = exp.network.predict(new_dataset)
For regression
and
autoencoding
models, this method
returns the output of the network when passed the given input dataset. For
classification
models, this method
returns the predicted classification of the inputs. (To get the actual output of
the network—the posterior class probabilities—for a classifier model, use
predict_proba()
.)
Regardless of the model, you pass to predict()
a numpy
array containing
data examples along the rows, and the method returns an array containing one row
of output predictions for each row of input data.
You can also compute the activations of all layer outputs in the network using
the Network.feed_forward()
method:
for name, value in exp.network.feed_forward(new_dataset).items():
print(abs(value).sum(axis=1))
This method returns a dictionary that maps layer output names to their
corresponding values for the given input. Like predict()
, each output array
contains one row for every row of input data.
Getting Learned Parameters¶
The parameters in each layer of the model are available using
Network.find()
. This method takes
two query terms—either integer index values or string names—and returns a
theano shared variable for the given parameter. The first query term finds a
layer in the network, and the second finds a parameter within that layer.
The find()
method returns a Theano shared variable. To get a numpy array
of the current values of the variable, call get_value()
on the result from
find()
, like so:
values = network.find('hid1', 'w').get_value()
For “encoding” layers in the network, this value array contains a feature vector in each column, and for “decoding” layers (i.e., layers connected to the output of an autoencoder), the features are in each row.
Visualizing Weights¶
Many times it is useful to create a plot of the features that the model learns; this can be useful for debugging model performance, but also for interpreting the dataset through the “lens” of the learned features.
For example, if you have a model that takes as input a 28×28 MNIST digit, then you could plot the weight vectors attached to each unit in the first hidden layer of the model to see what sorts of features the hidden unit detects:
img = np.zeros((28 * 10, 28 * 10), dtype='f')
for i, pix in enumerate(exp.network.find('hid1', 'w').get_value().T):
r, c = divmod(i, 10)
img[r * 28:(r+1) * 28, c * 28:(c+1) * 28] = pix.reshape((28, 28))
plt.imshow(img, cmap=plt.cm.gray)
plt.show()
Here we’ve taken the weights from the first hidden layer of the model
(exp.network.find('hid1', 'w')
) and plotted them as though they were 28×28
grayscale images. This is a useful technique for processing images (and, to some
extent, other types of data) because visually inspecting features can give you a
quick sense of how the model interprets its input. In addition, this can serve
as a sanity check—if the features in the model look like TV snow, for example,
the model probably hasn’t adapted its weights properly, so something might be
wrong with the training process.