Table Of Contents

Previous topic

Getting ffnet

Next topic

API docs

http://images.sourceforge.net/images/project-support.jpg

http://sflogo.sourceforge.net/sflogo.php?group_id=126615&type=8

Using ffnet

Architecture definition

Network architecture is defined simply by list of neuron connections:

>>> conec = [(1, 3), (2, 3), (0, 3),
             (1, 4), (2, 4), (0, 4),
             (3, 5), (4, 5), (0, 5)]

0 (zero) in conec is a special unit representing bias. If there is no connection from 0, bias is not considered in the node. In general any feed-forward (directed acyclic) graph is allowed.

Generation of conec is left to the user, however there are several functions provided to facilitate this task. See description of the following functions:

mlgraph(arch[, biases]) Creates standard multilayer network architecture.
tmlgraph(arch[, biases]) Creates multilayer network full connectivity list.
imlgraph(arch[, biases]) Creates multilayer architecture with independent outputs.

Network creation

Network is created by means of ffnet class in the following way:

>>> from ffnet import ffnet
>>> net = ffnet(conec)

ffnet‘s constructor makes check for cycles in the provided conec and raises TypeError if any.

All nodes in the network are assumed to have sigmoid activation function of the form y(x) = \frac{1}{1 + e^{-x}}, exept of the identity input nodes. Weights of the connections are automatically initialized at the network creation. They can be reinitialized later with ffnet.randomweights method.

Training network

Training algorithms are implemented as methods of the ffnet class. These methods are listed below:

ffnet.train_momentum(input, target[, eta, ...]) Simple backpropagation training with momentum.
ffnet.train_rprop(input, target[, a, b, ...]) Rprop training algorithm.
ffnet.train_cg(input, target, **kwargs) Train network with conjugate gradient algorithm.
ffnet.train_bfgs(input, target, **kwargs) Train network with constrained version of BFGS algorithm.
ffnet.train_tnc(input, target[, nproc])
Parameters:
ffnet.train_genetic(input, target, **kwargs) Global weights optimization with genetic algorithm.

The only non-keyword arguments of these methods – input and target – are numpy arrays of the shapes (p, i), and (p, o), where i – number of inputs, o – number of outputs, p – number of training patterns. For single network input or output training data can be 1D arrays of length p. Basic checks of data consistency are performed and ValueError is raised if any problem occur.

Note

If you store your training data in plain text files, you can use ffnet.readdata to load them into numpy arrays. Equivalently you can use numpy.loadtxt.

The simplest training call is, for example:

>>> net.train_tnc(input, target)

See deccription on the above methods for information about training parameters.

Data normalization

Normalization is handled automatically by ffnet so you can use simply your raw data arrays. Linear mapping to the range (0.15, 0.85) is performed for each input and output node.

Normalization ranges are retrieved from training data at first call of the training method, then these ranges are used at all subsequent training calls and at network recalling. However if you set:

>>> net.renormalize = True

the normalization info will be retrieved again at next training call (and net.renormalize will be set back to False).

Testing network

Look at the destription of ffnet.test method:

ffnet.test(input, target[, iprint, filename]) Calculates output and parameters of regression.

Recalling network

Usage of the trained network is as simple as function call:

>>> ans = net(inp)

or, alternatively:

>>> ans = net.call(inp)

If inp is 1D array of length i it is interpreted as single input pattern. In this case ans is also a 1D array of length o. If inp is 2D array of the shape (p, i) then ans will be calculated for all p patterns and will have shape (p, o)

There is also possibility to get exact partial derivatives of network outputs vs. its inputs. Derivative information is useful for example in analysing sensitivity of the network. Derivatives are calculated by calling:

>>> deriv = net.derivative(inp)

For single input pattern deriv will have shape (o, i). In mathematical notation this array can be represented as:

\mathbf{D} =
\left[
\begin{array}{ccc}
\frac{\partial y_1}{\partial x_1} & \ldots & \frac{\partial y_1}{\partial x_n} \\
\vdots                            & \ddots & \vdots                             \\
\frac{\partial y_m}{\partial x_1} & \ldots & \frac{\partial y_m}{\partial x_n}
\end{array}
\right]

where x_i, y_j are network inputs and outputs, appropriately. For 2D array of p input patterns, the generated deriv will have shape (p, o, i).

Loading/saving/exporting network

Look at the description of these utility functions:

savenet(net, filename) Dumps network to a file using cPickle.
loadnet(filename) Loads network pickled previously with savenet.
exportnet(net, filename[, name, lang, ...]) Exports network to a compiled language source code.