.. 1, ..

1,2

,
2
. .. ,
klakhman@gmail.com

*
,
.
.. [1] - ..
[2]. ,
.
: , , , ,

.
,
.
.
[3] . -,
. *

, 11-04-12174--2011 10-01-00786-.

 , , -
, c [4].
[5]. ,
[6, 7]

, .
,
[8, 9].

.

,
[10]
[11]. , ,
, , .

,
, [1, 12].

, , :
1
E t e t , ,e env t , e t 0,1 .

, , , n env . , :

ai n1 , q1 ,, nki , qki

 n j , q j () , k i .
,
.
:

Cf

NA

1
2 ki ki env
2n
i 1

ki

N A , k i ,
( ). , ,
.
, . , , . ,
Tr .
, .
, .


,
( ). ,
,
( 0,5),
.
,
. -

, 1 0
.

NEAT
[13], ,
.

, . , NEAT . .


.

( [1]) ( ).

,
.
, .

.

.
. ,
. [14], .

 -
. .

.
.

. , , ,
(. 1).

. 1. ( 20 10
, * - t- - p = 0,01)

, ,
(.1)
-

. , , , , . ,
,
.

, .
, (. 2),
, .

. 2.

,
, ,
.
, . .3 , -

 .
. 30 ,
15 -, 611 .
, , , .
, , ,
. ,

.

13

6
1

1
4

10

2
2
7
12

11

13

. 3. A. ( ,
/ ); . -
( , )

, , , , ,
4- .
, , ,
.

, -, -

 , -, . . 4 , .
,
, , , , . , . 4,
2.
5
5

1
6

10

10

5
8

7
3

. 4.

. 5. -

 (. 5)

,
, ,
(
).
. 1 ,
.
. 1.

n env
-
Tev

Np

250

5000

-
Tlife

250

Tr

Dm

30

Pm
Pa-syn

0,6

Pd-syn

0,05

Pdup

0,007

Pen- var

0,08

0,1

0,0085

, .. .
, .
, .
, -

. . , ,
.

1. .. . .: ,
1975.
2. .. .
. .: , 1995.
3. Kaelbling L.P., Littman M.L., Moore A.W. Reinforcement Learning. A Survey //
Journal of Artificial Intelligence Research. Vol. 4, Pp. 237-285, 1996.
4. Singh S., Lewis R., Barto A. G. Where Do Rewards Come From? // Proceedings
of the Cognitive Science Society (CogSci). Pp. 2601-2606, 2009.
5. Botvinick M.M., Niv Y., Barto A.G. Hierarchically organized behavior and its
neural foundations. A reinforcement learning perspective // Cognition. Vol.113, Is.
3, Pp. 262-280, 2009.
6. Sandamirskaya Y., Schoner G. An embodied account of serial order: How instabilities drive sequence generation // Neural Networks, Vol. 23, No. 10, Pp. 1164-1179,
2010
7. Komarov M. A., Osipov G. V., Burtsev M. S. Adaptive functional systems: Learning with chaos // Chaos, Vol. 20, Is.4, 04511, 2010.
8. Floreano D., Mondana F. Automatic Creation of an Autonomous Agent: Genetic
Evolution of a Neural-Network Driven Robot // Proceedings of the third international conference on Simulation of adaptive behavior: From Animals to Animats 3,
Pp. 421-430, 1994.
9. Floreano D., Durr P., Mattiussi C. Neuroevolution from architectures to learning
// Evolutionary Intelligence. Vol. 1, No.1, Pp. 47-62, 2008.
10. Hochreiter S., Bengio Y., Frasconi P., Schmidhuber J. Gradient Flow in Recurrent
Nets - the Difficulty of Learning Long-Term Dependencies // A Field Guide to
Dynamical Recurrent Neural Networks. IEEE Press, Pp. 237-243, 2001.
11. Botvinick M.M., Plaut D.C. Short-Term Memory for Serial Order: A Recurrent
Neural Network Model // Psychological Review. Vol. 113, No. 2, Pp. 201-233,
2006.
12. Edelman G. Neural Darwinism: The theory of neuronal group selection. NY: Basic
Books, 1987.
13. Kenneth S., Miikkulainen R. Evolving Neural Network through Augmenting Topologies // Evolutionary Computation. Vol. 10(2), Pp. 99-127, 2002.
14. Jog M.S., Kubota K, Connolly C.I., Hillegaart V., Graybiel A.M. Building neural
representations of habits // Science. Vol. 286, Pp. 1745-1749, 1999.