PyBrain

An introduction
What's in PyBrain
Neural networks

Reinforcement learning

Black-box optimization

Supervised learning

...


General RL framework:

Agent interacts with an Environment
Task provides rewards
Experiment connects them
An agent generally chooses its actions according
to a parametrized Policy .
(by default parameters is a continuous vector)
A Learner adapts the parameters of the policy in
order to maximize reward.
Examples of Environments that work with Tasks:
FlexCubeEnvironment
RollingUpTask
WalkDirectionTask
JumpTask
CaptureGame
GoMokuGame
MaximizeWinTask
HandicapTask
MazeEnvironment
CheeseMazeTask
TMazeTask
A different perspective on the same scenario:
Black-box optimization .
Class hierarchy for Learners
Policies use Modules :
black box
transform (vector-) input into (vector-) output
transformation depends on parameters

Modules can be differentiable :
back-propagate output-error towards the input
compute the partial derivatives w.r.t. each
parameter


Example Modules:

ActionValueTable
SigmoidLayer
BernoulliLayer
BiasUnit
LSTMLayer
GateLayer
LinearLayer
SoftmaxLayer


Modules can be combined in Networks :
linked with Connections (which can have parameters)
Networks are Modules themselves
Network-parameters are the concatenation of the
component parameters.


Example Networks:

FeedForwardNetwork
RecurrentNetwork
ConvolutionalNetwork
BidirectionalNetwork

Example Connections:
FullConnection
IdentityConnection
PermutationConnection
SharedConnection


Example: A simple Recurrent Neural Network


4 Module components (1 in, 1 out) without parameters
4 Connection components with parameters
1 of which is a recurrent Connection (pink)


# initialize a new network
from pybrain import *
n = RecurrentNetwork()

# add all modules (in any order)
n.addOutputModule(SigmoidLayer(2, name='out'))
n.addModule(TanhLayer(10, name='h'))
n.addModule(BiasUnit(name='bias'))
n.addInputModule(LinearLayer(4, name='in'))

# add all connections (in any order)
n.addConnection(FullConnection(n['h'], n['out']))
n.addConnection(FullConnection(n['bias'], n['h']))
n.addConnection(FullConnection(n['in'], n['h']))
n.addRecurrentConnection(FullConnection(n['h'], n['h']))

# build the network structure
n.sortModules()

# use it to map an input to an output
output = n.activate([0,1,2,-2])
General supervised learning framework:

DataSet provides training examples: inputs with
associated known targets
Module provides model that estimates targets
from inputs
Trainer tries to improve the model
Another view on the problem shows the analogy to
reinforcement learning
A DataSet essentially consists of a collection of 2d-arrays
(=fields ), holding the patterns to learn from.
Each ML problem solved by PyBrain has a custom DataSet .
Simplest case:
SupervisedDataSet
DataSets can be
saved/read
assembled pattern by pattern
constructed from arrays
transformed, split, scaled
extended with custom fields
...
The Trainer's function is to adjust parameters of the Module
directly or (most of the time) iteratively, to produce the best
model for the data. Schematically, this works as follows:
Module with initial parameters
Module is activated with an
input pattern and produces
output
Error is calculated by comparing
with target, and back-
propagated through the module
Parameters are adjusted by
some gradient descend or other
scheme
Further information
Web page: http://www.pybrain.org

Source: http://github.com/pybrain/pybrain

Mailing list: http://groups.google.com/group/pybrain/

Documentation: http://www.pybrain.org/docs/