A fuzzy logic approach to neuronal modeling

W. Senny, M. Jakoby , K. Wylery, J. Kleinle,y, H.-R. Luscherz , H.-J. Meyy, L. Mullery April 22, 1996
Fuzzy methods are commonly used to produce a desired output of a complex system which can be steered by known expert rules. But fuzzy methods can also be applied to the reversed problem: if the input-output behavior of a system is given one wishes to nd the rules by which this behavior is evoked. We introduce this reversed method to neuronal modeling and, as an example, apply it to a biological neuron of the HodgkinHuxley type. Going out from measured data of the membran potential, the sodiumand potassium-current, we rebuild the membran behavior by fuzzy rules and show how these rules can lead to a better understanding of the underlying mechanisms. The appeal of the approach is that once the system variables are identi ed, the analysis runs comletely automatically. De ning in advance the width of the membership functions the number and speci ty of the outcoming rules can be tuned.

Abstract

Keywords Neuro-fuzzy system, neuronal modeling, Hodgkin-Huxley model, rule-based

estimator

Introduction
One of the characteristics of biological neural networks are there functional plasticity and there robustness in solving a computational task. To guarantee such properties the computation is distributed to a great number of single processing units which are driven by a few basic mechanisms. Robustness is believed to be accompanied with some degree of irrelevance in the output of a particular processing unit. The basic mechanisms of a unit may be superposed by additional e ects due to the biological implementation as e.g. energy constraints which, however, should be fade out in a rst attempt to understand the pure information theoretical principles. The robustness in solving the computational task leads us to describe a unit by means of simple fuzzy rules which should t the basic mechanisms of the unit relevant to the information processing of the network. The idea is to extract some elementary fuzzy rules obtained from observing a single unit during the management of its tasks. The functional properties of the unit then may be condensed to these rules and this allows to abstract from the bulk of implementational details. The aim of this work is to introduce the method of fuzzy logic to neuronal modeling. Although the essential properties of a single unit may only be investigated in a functional
y z

e-mail: wsenn@iam.unibe.ch Institut fur Informatik u. angewandte Mathematik, Universitat Bern, Neubruckstr. 10, CH-3012 Bern Physiologisches Institut, Universitat Bern, Buhlplatz 5, CH-3012 Bern

context of the whole neural tissue, the characteristics of the new method is best explained on a simple, well understood object. Hereto we chose a single neuron as far as it is described by the well known Hodgkin-Huxley model and try to infer appropriate rules from experimental data to explain its behavior. In a technical terminology we solve a reversed engineering problem. While an engineer typically starts with rules intended to steer his plant we start with the plant, in our case the neuron, and deduce the rules according to which it works. This technique is also known as system identi cation, in our case the identi cation of a neuron with the appropriate variables and the corresponding dynamic rules.

1 The concept of fuzzy modeling

Let us explain the concept with the example of an axon segment which is trated more detailed in the following sections. The electro-physiological properties of such an axon compartment are determined by sodium- and potassium channels in the cell membran. We are interested in `dumb rules' describing the change of the membran potential as a function of e.g. sodium and potassium conductances of the membran. A typical example of such a rule is: IF the Na+ -conductance is large AND the membran potential is low THEN the change of membran potential is positive large. The linguistic variables as large and low are interpreted as fuzzy sets and are xed by appropriate membership functions over the real numbers. Such a rule with its membership functions may be deduced from the experimentally measured data, in this case from the values of the Na+ -conductances and the membran potential. Having available a representative collection of measurements the fuzzy-rules and the membership functions can be obtained essentially by a statistical correlation analysis. Suppose we had a train of triples (gNa gK V ) recorded every half millisecond and giving the values of sodium- and potassium-conductances and the membran potential at each time step. Each rule corresponds to a spherical cluster in this three-dimensional data space while the membership functions correspond to the projection of the sphere to the coordinate axes (cf. Figue 4). In the example of the rule above (which is of the Mamdani type) these axes specify the Na+ -conductance, the membran potential and, in a further dimension, the change of the latter one. The fuzzy system can be seen as an arti cial neural network where each neuron evaluates a rule. This view enables us to adapt the membership functions with help of back propagation. In case of a system modeling a Hodgkin-Huxley compartment the network has three layers: an input layer with input gNa , gK and V , a hidden layer with a unit for each rule and an output layer calculating the aggregation of the rule-outputs followed by a defuzzi cation, i.e. a conversion to a real (also called crisp) number. The output of the network is the change of the system variables, dgNa , dgK and dV , during a unit time. If we have available a supplementary time series of (gNa gK V ) we could ne tune the membersip functions obtained in a rst shot from clustering the data space. This is done by calculating (dgNa dgK dV ) for the new series once as output of the fuzzy network and once as change in the measured data. The error of the network output then is distributed backwards to the membership functions at the di erent nodes representing the fuzzy rules. 2

The computational power of a fuzzy neuron Microcircuits (Shepherd, 1990) implementing boolean functions in spines. More natural on a subthreshold level would be to consider not binary logic but ruther fuzzy logic rules. Binary logic would lead to the same limitation as Minsky and Papert run into when they tried to generalize the learning algorithm of the perceptron to multiple layers. Solution: Backpropagation with smooth transfer function, cf. (Rumelhart et al., 1986). This aspect will become particularly important if we consider `physiological back propagation' as it is suggested by recent experiments on the back propagating action potential and its induced long term potentiation. For instance, the AND-NOT gate on a dendritic spine proposed in (Shepherd, 1990, Fig. 3) could translate to the fuzzy rule: IF the presynaptic transmitter release at time t is large AND the release caused by the inhibitory interneuron during the next 4t milliseconds is low THEN the change of Na+ -conductance of the spiny membran is positive large during the same time interval 4t.

To evaluate this linguistic rule numerically one has to specify the meaning of large, low and positive large, respectively, in terms of membership function. The technique to adapt these function is back propagation of the error which is obtained by tting the data with the fuzzy rules system. If measured data about the fuzzy variables, in this case the transmitter release and the Na+ -conductance, are available the back propagation of the membership functions evolves over one layer. If e.g. no direct data are available on the transmitter release, the error has to be propagated over several layers back to known input data. Traditionally, the synaptic e ects are modeled by a set of di erential equation, see e.g. (Migliore et al., 1995). Not a direct link to the logical function. In the same way as McCulloch-Pitts neurons can realize any Boolean function (i.e. any function from f0 1gn to f0 1gn ) fuzzy neurons can approximate any continuous real function e.g. on the hypercube 0 1] , cf. the universal approximation theorem in (Wang, 1994, (2.50)). Equivalence with radial basis function networks may be seen as a special case of adaptive fuzzy systems (Wang, 1994, ch. 7.3) - any smooth function can be aproximated....
Comparison with other methods of neuronal modeling Our approach to neuronal modeling may be seen as a top-down approach leading from observed data to rules generating these data in terms of elementary processes. This approach is opposed to the classical one going back to Hodgkin and Huxley (Hodgkin and Huxley, 1952). In this classical approach one proceeds from bottom to top by describing rst the elementary processes of neural activity by means of ordinary di erential equations, cf. Figure 1. This prerequires a careful dissection of the problem into its elementary processes each of which is modeled by a dynamic parameter. The time dependent parameters together form a system of ordinary di erential equation tting the original data. In case of the Hodgkin-Huxley model the elementary processes are the gating dynamics of the Na+ - and K+ -channels which are described by the dynamic parameters m, h and n, respectively. The nal quantity which is approximated by these parameters is again the membran potential (cf. Appendix A). Additional mechanisms as e.g. inhomogeneous distribution of channels or spatially di erent ion-concentrations may be modeled in the setting of di erential equations. Two

emerging behavior model-free estimators (biological) System to be analyzed model-based estimators

adaptive fuzzy system

Ordinary Differential equations

elementary processes

Figure 1: Two di erent methods to rebuild a biological system. Describing the system by di erential equations requires a sound analysis of the object. The elementary processes have to be identi ed and the global behavior emerges by their interaction. This is opposed to the neuro-fuzzy method which applies to the unstructured data and leads to structured knowledge in terms of linguistic rules. Additional relations between the parameters could be incorporated to uncover the elementary processes. problems, however, arise: The rst is the intellectual one of nding the appropriate di erential equation modeling the extended situation. The second is the computational one of managing the enlarged system of di erential equation in a reasonable CPU-time. A remedy against both problems is not trying to analyze the situation into details but rather to t directly the output by new rules including the additional parameters. Of course, the price of such a direct approach is that we have not yet structured our object of consideration and that the rules we will extract from the available data perhaps do not tell us much comprehensible about the object itself. Any functional relations between the input parameters, however, could be incorporated into the rule-based description making it to hybrid model with bene ts of both extremes, the complete mathematical description and the direct t of the input-output relation by rules (cf. the discussion in Section 4). Apart from the categorization top-down versus bottom-up there is a categorization according to the prerequired knowledge. The method of extracting fuzzy rules leads to a model-free estimator while the method of Hodgkin and Huxley leads to a model-based estimator e.g. of the membran potential. Other model-free estimators of neuronal activity are e.g. the method of approximating the output spike train by applying a linear lter procedure on the stimulus train (Gabbiani, 1996) or the neural network approach of (Doya and Selverston, 1994) tting a complex biological neuron by a recurrent multilayer-network. The common characteristics of the mentioned model-free estimators is to see the neuron in a rst instance as a black box and to catch the main characteristics of the input-output relationship in terms of there speci c language, i.e. in terms of lter characteristics, of connectionists weight distribution and of fuzzy rules, respectively. In a second instance one attempts to draw conclusions on the black box content by analyzing how the best t of the input-output relationship is obtained within the speci c language. This method of rst imitating the black box is the essential di erence to the classical method which bases 4

on a formal treatment of the object. Let us nally distinguish between our fuzzy approach and the cited arti cial neural network approach. Actually, since we want to exibly adapt the fuzzy rules, we designed a neuro-fuzzy system in order to back propagate errors of the estimator's output to the rules. In this sense our approach combines the advantages of both concepts, the plasticity of neural networks and the structuring of knowledge by the fuzzy-rules. The type of rules emerging are reminiscent to the way an experimentalist would describe the e ects of certain parameters to his observed quantity. The approach therefore should be suitable as well to support experimentalists in their analysis of data (cf. the results in Section 4).

2 An example: the membran of a squid axon

Experimentally one is able to measure locally the membran potential during a spike of the neuron and the ionic current involved in this process, cf. Figure 2. The technique of dynamic clamp ....
dentrites Na + Na + Na +

IK

soma

axon

INa

K+

K+

K+

to the muscle

Figure 2: Left: A motoneuron from the spinal cord of a rat. Right: Schematical draw of a piece of axon with Na+ - and K+ -channels as it is modeled by Hodgkin and Huxley. Inward owing Na+ -ions raise the membran potential while out owing K+ -ions lower it again.

3 Generation of the fuzzy system 4 Results of the analysis and discussion

Mean square error of the tted experimental data: .... ANFIS test results. The bene t of the analysis is not primarely the high precision in reproducing the experimental data but rather to show the interaction and the importance of the di erent rules. A modest step into the direction of an integration in a neural tissue is the connection of several hundreds of our compartments to a complex neuron. We implemented our fuzzy model as a look-up table in the software package NEURON allowing to simulate biological neurons and small networks. 5

120

1 I_K 0.8 0.6

100

Membran potential V [mV]

80

V(t)

0.4 0.2 I_Na + I_K 0 0.2 0.4 0.6

60

40

20

0
0.8 I_Na

20 0

Current [A]

10

20

30 time [ms]

40

50

60

1 0

10

20

30 t [ms]

40

50

60

Figure 3: These time courses can be recorded with 3 di erent experiments. Left: The time course of the membran potential after a super threshold current injection. This action potential is propagated through the axon without attenuation. Right: The time course of the Na+ -current and the K+ -current which ows through the membran at the same time at which the action potential on the left is generated. At the beginning Na+ ions ow fast inwards (indicated by the negative sign of the current) and this leads to the raise of the membran potential. The potential is shunted by a slow out ow of K+ -ions in the second half of the time course neutralizing the slow Na+ -in ow (cf. Figure 2).

Appendix A The Hodgkin-Huxley model of an axon segment B Fuzzy inference system of the Sugeno type
(Wang, 1994, ch. 7.3)

V sgno
Rule 1. 2. 3. 4. 5. 6. 7.

Input

V
neg.small pos.mid pos.large pos.mid pos.large pos.small pos.small

INa
zero neg.large neg.large neg.large neg.mid neg.small neg.small

IK
zero pos.large pos.large pos.large pos.mid zero pos.small

Output Change of Output @dV dV p = @dV q = @I r = @dV @V @IK Na 8.4375 0.3854 -286.0440 -134.6368 -5.0381 -0.0332 -32.3923 -35.8855 -4.8002 -0.0096 -29.7728 -32.3776 -4.5231 -0.0906 -104.2911 -98.5410 -2.7864 -0.5821 -17.6550 -37.9492 12.9417 -0.2253 -138.9352 -23.3895 -10.2220 -0.8661 100.3190 186.5755

Acknowledgment WS and KW were supported by the Swiss National Science Foundation under the research grant no. 5002-03793 (Schwerpunktprogramm Biotechnologie).
6

1 0.8 0.6

Input3: I_K

0.4 0.2 0 0.2 0 0.2 0.4 0.6 0.8 Input2: I_Na 20 0 20 40 Input1: V 60 80 100 120

Figure 4: Input data in the phase space (V INa IK ). Spheres generated by a cluster algorithm. Projection leads to the indicated membership functions on the 3 axes.

References
Doya, K. & Selverston, A. I. (1994). Dimension Reduction of Biological Neuron Models by Arti cial Neural Networks. Neural Computation, 6(4):696{717. Gabbiani, F. (1996). Coding of time-varying signals in spike trains of linear and half-wave rectifying neurons. Network: Computation in Neural Systems, 7:61{85. Hodgkin, A. L. & Huxley, A. F. (1952). A Quantitative Description of Membrane Current and Its Application to Conduction and Excitation in Nerve. J. Physiol. (London), 117:500{544. Migliore, M., Alicata, F., & Ayala, G. (1995). A Model for Long-Term Potentiation and Depression. J. comp. Neurosci, 2:335{343. Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). Learning internal representations by erroe propagation. In: Parallel Distributed Processing: Explorations in the Microstructures of Cognition, D. E. Rumelhart & J. L. McClelland, ed., volume 1, pages 318{362. MIT Press. Shepherd, G. M. (1990). The Signi cance of Real Neuron Architectures for Neural Network Simulations. In: Computational Neuroscience, E. L. Schwartz, ed., chapter 8, pages 82{96. A Bradford book, MIT Press. Wang, L.-X. (1994). Adaptive fuzzy systems and control. Prentice Hall. 7

40

20

Output: dV

R1

R6

R5

R3
20

R4 R7

R2

40 0.2 0 0.2 0.4 0.6 0.8 Input2: I_Na 1 20 0 Input1: V 20 60 40 80 100

120

Figure 5: Fuzzy rules of the Sugeno-type approximate the output linearly in the cluster centers.

120

Generated rules V I_Na Z0 NB0 NB1 NB2 NM0 NS0 NS1 I_K Z0 PB0 PB1 PB2 PM0 Z1 PS0 output 8.4375 5.0381 4.8002 4.5231 2.7864 12.9417 10.2220

Membran potential V, generated by the fuzzysystem

100
Regel1 Regel2 NS0 PM0 PB0 PM1 PB1 PS0 PS1

80

Regel3 Regel4

60

Regel5 Regel6

40

Regel7

20

20 0

20

40

60 time [ms]
1

80

100

120 150

Degree of firing of the rules for V 1 5 0.9 0.8 0.7 3 2 4 7 6

100

0.6 0.5 0.4 0.3 0.2 0.1 0 0

Influence dV_i of rule i on dV

Degree of firing

50 dV 1 0 3 2 5 4

20

40

60 Sample time

80

100

120

50 0

7 20 40 60 time t [ms] 80 100 120

Figure 6: Top left: An action potential as generated by the neuro-fuzzy system after training. The number refer to the rules with maximal degree of ring at the corresponding times. Top right: The rules generated by di erent runs through the training data. Bottom left: The degree of ring indicates when and with the corresponding rule is used. Bottom right: The curve show the e ective in uence of the rules on the (change of) membran potential during the action potential. It is calculated by the di erence of the system output with and without the rule.

Clamps at different voltage levels 0.5

Clamps of the FuzzySystem at different voltage levels 0.2

0 V=120

V=0 V=20

V=120

0.2

0.5

0.4

I_Na

V=100

I_Na

V=100

0.6
1.5

0.8
V=40 2 V=80 V=60 0.5 1 1.5 2 2.5 3 time[seconds] 3.5 4 4.5 x 10 5
3

V=80 1 V=60 V=40

2.5 0

1.2 0

0.5

1.5

2.5 3 time[seconds]

3.5

4.5 x 10

5
3

Figure 7: Left: The time course of the Na+ -current for di erent voltage clamps. The data are generated by the Hodgkin-Huxley equations and represent with high precision the true Na+ -current of a clamped membran of the squid axon. Right: The Na+ -current at xed membran potential generated by the neuro-fuzzy system. Since only the time course of the action potential in Figure 3 is learned the voltage clamp experiments may not be tted exactly. The quality of tting is a measure for the generalization capability of the neuro-fuzzy system.

10