NEURAL NETWORKS

Dr. Atif AL-

nsour
Presented by:
Abdullah smadi (2004974044)
ohammed bonyan (2004974094)
ustafa al !adi (200497404")
Ali rababah (20049740#$)
Abstract:
This report is an introduction to Artificial Neural
Networks. The different types of neural networks are
explained and shown, applications of neural
networks like ANNs in medicine are described, and
historical background is provided. The connection
between the artificial and the real thing is also
explained. Finally, the mathematical models
involved are presented.
2
Contents:
1. Introduction to Neural Networks
1.1 What is a neural network?
1.2 Historical background
1.3 Why use neural networks?
1. Neural networks !ersus con!entional co"#uters $ a co"#arison

2. Hu"an and Arti%icial Neurons $ in!estigating the si"ilarities
2.1 How the Hu"an &rain 'earns?
2.2 (ro" Hu"an Neurons to Arti%icial Neurons

3. An )ngineering a##roach
3.1 A si"#le neuron $ descri#tion o% a si"#le neuron
3.2 (iring rules $ How neurons "ake decisions
3.3 *attern recognition $ an e+a"#le
3. A "ore co"#licated neuron
. Architecture o% neural networks
.1 (eed$%orward ,associati!e- networks
.2 (eedback ,auto associati!e- networks
.3 Network layers
. *erce#tions
.. /he 'earning *rocess
..1 /rans%er (unction
..2 An )+a"#le to illustrate the abo!e teaching #rocedure
..3 /he &ack$*ro#agation Algorith"
.. /he &ack$*ro#agation Algorith" ,non$linear-
0. A##lications o% neural networks
0.1 Neural networks in #ractice
0.2 Neural networks in "edicine
0.2.1 1odeling and 2iagnosing the Cardio!ascular 3yste"
0.2.2 )lectronic noses $ detection and reconstruction o% odors by ANNs
0.2.3 Instant *hysician $ a co""ercial neural net diagnostic #rogra"
0.3 Neural networks in business
0.3.1 1arketing
0.3.2 Credit e!aluation
4. Conclusion

5e%erences
3
1. Introduction to neural networks
"." %hat is a &eural &et'or()
An Artificial Neural Network (ANN is an information processing paradigm that is
inspired by the way biological nervous systems works, such as the brain, process
information. The key element of this paradigm is the structure of the information
processing system. !t is composed of a large number of highly interconnected
processing elements (neurons working together to solve specific problems.
ANNs,its "ust like people, learn by example. An ANN is designed for a specific
application, such as a data classification, through a learning process. #earning in
biological systems involves ad"ustments to the synaptic connections that exist
between the neurons. This is true of ANNs as well.
".2 *istori+al ba+(!round
Neural network simulations appear to be a recent development. $owever, this field
was established before the advent of computers, and has survived at least one
ma"or setback and several eras.
%any important advances have been boosted by the use of inexpensive computer
emulations. Following an initial period of enthusiasm, the field survived a period
of frustration and disrepute. &uring this period when funding and professional
support was minimal, important advances were made by relatively few researchers.
These pioneers were able to develop convincing technology which surpassed the
limitations identified by %insky and 'apert. %insky and 'apert, published a book
(in ()*) in which they summed up a general feeling of frustration (against neural
networks among researchers, and was thus accepted by most without further
analysis. +urrently, the neural network field en"oys a resurgence of interest and a
corresponding increase in funding.
The first artificial neuron was produced in (),- by the neurophysiologist .arren
%c+ulloch and the logician .alter 'its. /ut the technology available at that time
did not allow them to do too much.
4
"., %hy use neural net'or(s)
Neural networks, with their ability to derive meaning from complicated or
imprecise data, can be used to get patterns and detect trends that are too complex to
be noticed by either humans or other computer techni0ues. A trained neural
network can be thought of as an 1expert1 in the category of information it has been
given to analy2e. This expert can then be used to provide pro"ections given new
situations of interest and answer 1what if1 0uestions.
3ther advantages include4
1. Adaptive learning: An ability to learn how to do tasks based
on the data given for training or initial experience.
2. Self-rgani!ation: An A"" can create its own organi!ation or
representation of the infor#ation it receives d$ring learning
ti#e.
3. %eal &i#e peration: A"" co#p$tations #ay be carried o$t
in parallel' and special hardware devices are being designed
and #an$fact$red which take advantage of this capability.
4. (a$lt &olerance via %ed$ndant )nfor#ation *oding: +artial
destr$ction of a network leads to the corresponding
degradation of perfor#ance. ,owever' so#e network
capabilities #ay be retained even with #a-or network
da#age.
".4 &eural net'or(s -ersus +on-entional +om.uters
Neural networks take a different approach to problem solving than that of
conventional computers. +onventional computers use an algorithmic approach i.e.
the computer follows a set of instructions in order to solve a problem. 5nless the
specific steps that the computer needs to follow are known the computer cannot
solve the problem. That restricts the problem solving capability of conventional
computers to problems that we already understand and know how to solve. /ut
computers would be so much more useful if they could do things that we don6t
exactly know how to do.
.
Neural networks process information in a similar way the human brain does.
The network is composed of a large number of highly interconnected
processing elements (neurones working in parallel to solve a specific
problem. Neural networks learn by example. They cannot be programmed to
perform a specific task. The examples must be selected carefully otherwise
useful time is wasted or even worse the network might be functioning
incorrectly. The disadvantage is that because the network finds out how to
solve the problem by itself, its operation can be unpredictable.
3n the other hand, conventional computers use a cognitive approach to
problem solving7 the way the problem is to solved must be known and stated
in small unambiguous instructions. These instructions are then converted to a
high level language program and then into machine code that the computer
can understand. These machines are totally predictable7 if anything goes
wrong is due to a software or hardware fault.
Neural networks and conventional algorithmic computers are not in
competition but complement each other. There are tasks are more suited to an
algorithmic approach like arithmetic operations and tasks that are more suited
to neural networks. 8ven more, a large number of tasks, re0uire systems that
use a combination of the two approaches (normally a conventional computer
is used to supervise the neural network in order to perform at maximum
efficiency.
/
2. Hu"an and Arti%icial Neurons $ in!estigating the
si"ilarities
2." *o' the *uman /rain Learns)
%uch is still unknown about how the brain trains itself to process information, so
theories abound. !n the human brain, a typical neuron collects signals from others
through a host of fine structures called dendrites. The neuron sends out signals of
electrical activity through a long, thin stand known as an axon, which splits into
thousands of branches. At the end of each branch, a structure called a synapse
converts the activity from the axon into electrical effects that inhibit or excite
activity in the connected neurons. .hen a neuron receives excitatory input that is
sufficiently large compared with its inhibitory input, it sends a spike of electrical
activity down its axon. #earning occurs by changing the effectiveness of the
synapses so that the influence of one neuron on another changes.


+omponents of a neuron

The synapse

0
2.2 0rom *uman &eurons to Artifi+ial &eurons
.e conduct these neural networks by first trying to find the essential features of
neurones and their interconnections. .e then typically program a computer to
simulate these features. $owever because our knowledge of neurones is
incomplete and our computing power is limited, our models are necessarily gross
ideali2ations of real networks of neurones.

/he neuron "odel .
1
6 3. An engineering a##roach
,." A sim.le neuron
An artificial neuron is a device with many inputs and one output. The neuron has
two modes of operation7 the training mode and the using mode. !n the training
mode, the neuron can be trained to fire (or not, for particular input patterns. !n the
using mode, when a taught input pattern is detected at the input, its associated
output becomes the current output. !f the input pattern does not belong in the
taught list of input patterns, the firing rule is used to determine whether to fire or
not.
A simple neuron
,.2 0irin! rules
The firing rule is an important concept in neural networks and accounts for their
high flexibility. A firing rule determines how one calculates whether a neuron
should fire for any input pattern. !t relates to all the input patterns, not only the
ones on which the node was trained.
A simple firing rule can be implemented by using $amming distance techni0ue.
The rule goes as follows4
Take a collection of training patterns for a node, some of which cause it to fire (the
(9taught set of patterns and others which prevent it from doing so (the :9taught
set. Then the patterns not in the collection cause the node to fire if, on comparison
, they have more input elements in common with the 6nearest6 pattern in the (9
taught set than with the 6nearest6 pattern in the :9taught set. !f there is a tie, then the
pattern remains in the undefined state.
2
For example, a -9input neuron is taught to output ( when the input (;(,;< and ;-
is ((( or (:( and to output : when the input is ::: or ::(. Then, before applying
the firing rule, the truth table is7
31: 4 4 4 4 1 1 1 1
32: 4 4 1 1 4 4 1 1
33: 4 1 4 1 4 1 4 1

5&:
4 4 461 461 461 1 461 1
As an example of the way the firing rule is applied, take the pattern :(:. !t differs
from ::: in ( element, from ::( in < elements, from (:( in - elements and from
((( in < elements. Therefore, the 6nearest6 pattern is ::: which belongs in the :9
taught set. Thus the firing rule re0uires that the neuron should not fire when the
input is ::(. 3n the other hand, :(( is e0ually distant from two taught patterns that
have different outputs and thus the output stays undefined (:=(.
/y applying the firing in every column the following truth table is obtained7
31: 4 4 4 4 1 1 1 1
32: 4 4 1 1 4 4 1 1
33: 4 1 4 1 4 1 4 1

5&:
4 4 4 461 461 1 1 1
The difference between the two truth tables is called the generalisation of the
neuron. Therefore the firing rule gives the neuron a sense of similarity and enables
it to respond 6sensibly6 to patterns not seen during training.
14

,., Pattern 1e+o!nition - an e2am.le
An important application of neural networks is pattern recognition. 'attern
recognition can be implemented by using a feed9forward (figure ( neural
network that has been trained accordingly. &uring training, the network is
trained to associate outputs with input patterns. .hen the network is used, it
identifies the input pattern and tries to output the associated output pattern.
The power of neural networks comes to life when a pattern that has no output
associated with it, is given as an input. !n this case, the network gives the
output that corresponds to a taught input pattern that is least different from
the given pattern.
Figure (.
For example4
The network of figure ( is trained to recogni2e the patterns T and $. The
associated patterns are all black and all white respectively as shown next
11
!f we represent black s0uares with : and white s0uares with ( then the truth tables
for the - neurons after generali2ation are7
;((4 : : : : ( ( ( (
;(<4 : : ( ( : : ( (
;(-4 : ( : ( : ( : (
35T4 : : ( ( : : ( (
/o# neuron
;<(4 : : : : ( ( ( (
;<<4 : : ( ( : : ( (
;<-4 : ( : ( : ( : (
35T4 ( :=( ( :=( :=( : :=( :
1iddle neuron
;<(4 : : : : ( ( ( (
;<<4 : : ( ( : : ( (
;<-4 : ( : ( : ( : (
35T4 ( : ( ( : : ( :
&otto" neuron
12
From the tables it can be seen the following associations can be extracted4
!n this case, it is obvious that the output should be all blacks since the input pattern
is almost the same as the 6T6 pattern.
$ere also, it is obvious that the output should be all whites since the input pattern
is almost the same as the 6$6 pattern.
13
$ere, the top row is < errors away from the T and - from an $. >o the top output is
black. The middle row is ( error away from both T and $ so the output is random.
The bottom row is ( error away from T and < away from $. Therefore the output is
black. The total output of the network is still in favour of the T shape.
,.4 A more +om.li+ated neuron
The previous neuron doesn6t do anything that conventional computers don6t do
already. A more complicated neuron (figure < is the %c+ulloch and 'itts model
(%+'. The difference from the previous model is that the inputs are 6weighted6,
the effect that each input has at decision making is dependent on the weight of the
particular input. The weight of an input is a number which when multiplied with
the input gives the weighted input. These weighted inputs are then added together
and if they exceed a pre9set threshold value, the neuron fires. !n any other case the
neuron does not fire?.
Figure <. An %+' neuron
!n mathematical terms, the neuron fires if and only if7
;(.( @ ;<.< @ ;-.- @ ... A T
The addition of input weights and of the threshold makes this neuron a very
flexible and powerful one. The %+' neuron has the ability to adapt to a particular
situation by changing its weights and=or threshold. Barious algorithms exist that
cause the neuron to 6adapt67 the most used ones are the &elta rule and the back error
propagation. The former is used in feed9forward networks and the latter in
feedback networks.
14
Architecture o% neural networks
4." 0eed-for'ard net'or(s
Feed9forward ANNs (figure ,9( allow signals to travel one way only7 from input
to output. There is no feedback (loops i.e. the output of any layer does not affect
that same layer. Feed9forward ANNs tend to be straight forward networks that
associate inputs with outputs. They are widely used in pattern recognition. This
type of organi2ation is also referred to as bottom9up or top9down.
4.2 0eedba+( net'or(s
Feedback networks (figure ,9< can have signals travelling in both directions by
introducing loops in the network. Feedback networks are very powerful and can
get extremely complicated. Feedback networks are dynamic7 their 6state6 is
changing continuously until they reach a balanced point. They remain at the
balanced point until the input changes and a new e0uilibrium needs to be found.
Feedback architectures are also referred to as interactive or recurrent, although the
latter term is often used to denote feedback connections in single9layer
organi2ations.
1.
Figure ,.( An example of a simple
feedforward network
Figure ,.< An example of a complicated
network
4., &et'or( layers
The commonest type of artificial neural network consists of three groups, or layers,
of units4 a layer of 1in#ut1 units is connected to a layer of 1hidden1 units, which is
connected to a layer of 7out#ut1 units. (Figure ,.(
The activity of the input units represents the raw information that is fed into the
network.
The activity of each hidden unit is determined by the activities of the input units
and the weights on the connections between the input and the hidden units.
The behavior of the output units depends on the activity of the hidden units and
the weights between the hidden and output units.
1/
This simple type of network is interesting because the hidden units are free to
construct their own representations of the input. The weights between the input and
hidden units determine when each hidden unit is active, and so by modifying these
weights, a hidden unit can choose what it represents.
.e also distinguish single9layer and multi9layer architectures. The single9layer
organi2ation, in which all units are connected to one another, to form the most
general case and is of more potential computational power than hierarchically
structured multi9layer organi2ations. !n multi9layer networks, units are often
numbered by layer, instead of following a global numbering.
10
4.4 Per+e.tions
The most effective thing on the work on neural nets in the *:6s went under the
heading of 6perceptrons6 a term coined by Frank Cosenblatt. The perceptron (figure
,., turns out to be an %+' model ( neuron with weighted inputs with some
additional, fixed, pre99processing. 5nits labelled A(, A<, A" , Ap are called
association units and their task is to extract specific, locali2ed featured from the
input images. 'erceptrons mimic the basic idea behind the mammalian visual
system. They were mainly used in pattern recognition even though their
capabilities extended a lot more.
Figure ,.,
!n ()*) %insky and 'apert wrote a book in which they described the limitations of
single layer 'erceptrons. The impact that the book had was tremendous and caused
a lot of neural network researchers to loose their interest. The book was very well
written and showed mathematically that single layer perceptrons could not do
some basic pattern recognition operations like determining the parity of a shape or
determining whether a shape is connected or not. .hat they did not realised, until
the D:6s, is that given the appropriate training, multilevel perceptrons can do these
operations.
11
.. /he 'earning *rocess
The memori2ation of patterns and the subse0uent response of the network can be
categori2ed into two general paradigms4
Associati!e "a##ing in which the network learns to produce a particular pattern
on the set of input units whenever another particular pattern is applied on the set of
input units. The associative mapping can generally be broken down into two
mechanisms4
Auto-association4 an input pattern is associated with itself and the states of
input and output units coincide. This is used to provide pattern completition, ie to
produce a pattern whenever a portion of it or a distorted pattern is presented. !n the
second case, the network actually stores pairs of patterns building an association
between two sets of patterns.
Hetero-association4 is related to two recall mechanisms4
Nearest-neighbor recall, where the output pattern produced corresponds to the
input pattern stored, which is closest to the pattern presented, and
Interpolative recall, where the output pattern is a similarity dependent
interpolation of the patterns stored corresponding to the pattern presented. Eet
another paradigm, which is a variant associative mapping is classification, ie when
there is a fixed set of categories into which the input patterns are to be classified.

5egularity detection in which units learn to respond to particular properties of
the input patterns. .hereas in associative mapping the network stores the
relationships among patterns, in regularity detection the response of each unit has a
particular 6meaning6. This type of learning mechanism is essential for feature
discovery and knowledge representation.
8very neural network possesses knowledge which is contained in the values of the
connections weights. %odifying the knowledge stored in the network as a function
of experience implies a learning rule for changing the values of the weights.
12

!nformation is stored in the weight matrix . of a neural network. #earning is the
determination of the weights. Following the way learning is performed, we can
distinguish two ma"or categories of neural networks4
(i+ed networks in which the weights cannot be changed, ie d.=dtF:. !n such
networks, the weights are fixed a priori according to the problem to solve.
Ada#ti!e networks which are able to change their weights, ie d.=dt notF :.
All learning methods used for adaptive neural networks can be classified into two
ma"or categories4
3u#er!ised learning which incorporates an external teacher, so that each output
unit is told what its desired response to input signals ought to be. &uring the
learning process global information may be re0uired. 'aradigms of supervised
learning include error9correction learning, reinforcement learning and stochastic
learning.
An important issue conserving supervised learning is the problem of error
convergence, ie the minimi2ation of error between the desired and computed unit
values. The aim is to determine a set of weights which minimi2es the error. 3ne
well9known method, which is common to many learning paradigms, is the least
mean s0uare (#%> convergence.
24
8nsu#er!ised learning uses no external teacher and is based upon only local
information. !t is also referred to as self9organi2ation, in the sense that it self9
organi2es data presented to the network and detects their emergent collective
properties. 'aradigms of unsupervised learning are $ebbian learning and
competitive learning.
Ano<.< From $uman Neurons to Artificial Neuronesther aspect of learning
concerns the distinction or not of a separate phase, during which the network is
trained, and a subse0uent operation phase. .e say that a neural network learns off9
line if the learning phase and the operation phase are distinct. A neural network
learns on9line if it learns and operates at the same time. 5sually, supervised
learning is performed off9line, whereas unsupervised learning is performed on9line.
5.1 Transfer Function
The behavior of an ANN (Artificial Neural Network depends on both the weights
and the input9output function (transfer function that is specified for the units. This
function typically falls into one of three categories4
#inear (or ramp
Threshold
>igmoid
For linear units, the output activity is proportional to the total weighted output.
For threshold units, the output are set at one of two levels, depending on whether
the total input is greater than or less than some threshold value.
For sig"oid units, the output varies continuously but not linearly as the input
changes. >igmoid units bear a greater resemblance to real neurones than do linear
or threshold units, but all three must be considered rough approximations.
To make a neural network that performs some specific task, we must choose how
the units are connected to one another (see figure ,.(, and we must set the weights
on the connections appropriately. The connections determine whether it is possible
for one unit to influence another. The weights specify the strength of the influence.
21
.e can teach a three9layer network to perform a particular task by using the
following procedure4
1. 7e present the network with training exa#ples' which
consist of a pattern of activities for the inp$t $nits together
with the desired pattern of activities for the o$tp$t $nits.
2. 7e deter#ine how closely the act$al o$tp$t of the network
#atches the desired o$tp$t.
3. 7e change the weight of each connection so that the
network prod$ces a better approxi#ation of the desired
o$tp$t.
5.2 An Example to illustrate the above teaching
procedure:
Assume that we want a network to recogni2e hand9written digits. .e might use an
array of, say, <G* sensors, each recording the presence or absence of ink in a small
area of a single digit. The network would therefore need <G* input units (one for
each sensor, (: output units (one for each kind of digit and a number of hidden
units.
For each kind of digit recorded by the sensors, the network should produce high
activity in the appropriate output unit and low activity in the other output units.
To train the network, we present an image of a digit and compare the actual activity
of the (: output units with the desired activity. .e then calculate the error, which
is defined as the s0uare of the difference between the actual and the desired
activities. Next we change the weight of each connection so as to reduce the error.
.e repeat this training process for many different images of each different images
of each kind of digit until the network classifies every image correctly.
To implement this procedure we need to calculate the error derivative for the
weight (8. in order to change the weight by an amount that is proportional to the
rate at which the error changes as the weight is changed. 3ne way to calculate the
8. is to perturb a weight slightly and observe how the error changes. /ut that
method is inefficient because it re0uires a separate perturbation for each of the
many weights.
22
Another way to calculate the 8. is to use the /ack9propagation algorithm which
is described below, and has become nowadays one of the most important tools for
training neural networks.
5.3 The ac!"#ropagation Algorithm
!n order to train a neural network to perform some task, we must ad"ust the weights
of each unit in such a way that the error between the desired output and the actual
output is reduced. This process re0uires that the neural network compute the error
derivative of the weights ()W. !n other words, it must calculate how the error
changes as each weight is increased or decreased slightly. The back propagation
algorithm is the most widely used method for determining the )W.
The back9propagation algorithm is easiest to understand if all the units in the
network are linear. The algorithm computes each )W by first computing the )A,
the rate at which the error changes as the activity level of a unit is changed. For
output units, the )A is simply the difference between the actual and the desired
output. To compute the )A for a hidden unit in the layer "ust before the output
layer, we first identify all the weights between that hidden unit and the output units
to which it is connected. .e then multiply those weights by the )As of those
output units and add the products. This sum e0uals the )A for the chosen hidden
unit. After calculating all the )As in the hidden layer "ust before the output layer,
we can compute in like fashion the )As for other layers, moving from layer to
layer in a direction opposite to the way activities propagate through the network.
This is what gives back propagation its name. 3nce the )A has been computed for
a unit, it is straight forward to compute the )W for each incoming connection of
the unit. The )W is the product of the 8A and the activity through the incoming
connection.
Note that for non9linear units, the back9propagation algorithm includes an extra
step. /efore back9propagating, the )A must be converted into the )I, the rate at
which the error changes as the total input received by a unit is changed.

23
../he back$#ro#agation Algorith" $ a "athe"atical a##roach ,non$linear-
5nits are connected to one another. +onnections correspond to the edges of the
underlying directed graph. There is a real number associated with each connection,
which is called the weight of the connection. .e denote by .i" the weight of the
connection from unit ui to unit u". !t is then convenient to represent the pattern of
connectivity in the network by a weight matrix . whose elements are the weights
.i". Two types of connection are usually distinguished4 excitatory and inhibitory.
A positive weight represents an excitatory connection whereas a negative weight
represents an inhibitory connection. The pattern of connectivity characteri2es the
architecture of the network.
A unit in the output layer determines its activity by following a two step procedure.
First, it computes the total weighted input x", using the formula4
24
where yi is the activity level of the "th unit in the previous layer and .i" is the
weight of the connection between the ith and the "th unit.
Next, the unit calculates the activity y" using some function of the total weighted
input. Typically we use the sigmoid function4
3nce the activities of all output units have been determined, the network computes
the error 8, which is defined by the expression4
.here y" is the activity level of the "th unit in the top layer and d" is the desired
output of the "th unit.

The back9propagation algorithm consists of four steps4
(. +ompute how fast the error changes as the activity of an output unit is changed.
This error derivative (8A is the difference between the actual and the desired
activity.
<. +ompute how fast the error changes as the total input received by an output unit
is changed. This 0uantity (8! is the answer from step ( multiplied by the rate at
which the output of a unit changes as its total input is changed.
-. +ompute how fast the error changes as a weight on the connection into an output
unit is changed. This 0uantity (8. is the answer from step < multiplied by the
activity level of the unit from which the connection emanates.
2.
,. +ompute how fast the error changes as the activity of a unit in the previous layer
is changed. This crucial step allows back propagation to be applied to multilayer
networks. .hen the activity of a unit in the previous layer changes, it affects the
activates of all the output units to which it is connected. >o to compute the overall
effect on the error, we add together all these separate effects on output units. /ut
each effect is simple to calculate. !t is the answer in step < multiplied by the weight
on the connection to that output unit.
/y using steps < and ,, we can convert the 8As of one layer of units into 8As for
the previous layer. This procedure can be repeated to get the 8As for as many
previous layers as desired. 3nce we know the 8A of a unit, we can use steps < and
- to compute the 8.s on its incoming connections.
2/
#. A..li+ations of neural net'or(s
#." &eural &et'or(s in Pra+ti+e
Hiven this description of neural networks and how they work, what real world
applications are they suited forI Neural networks have broad applicability to real
world business problems. !n fact, they have already been successfully applied in
many industries.
>ince neural networks are best at identifying patterns or trends in data, they are
well suited for prediction or forecasting needs including4
.eather forecasting
!ndustrial process control
+ustomer research
&ata validation
Cisk management
Target marketing
/ut to give you some more specific examples7 ANN are also used in the following
specific paradigms4 recognition of speakers in communications7 diagnosis of
hepatitis7 recovery of telecommunications from faulty software7 interpretation of
multimeaning +hinese words7 undersea mine detection7 texture analysis7 three9
dimensional ob"ect recognition7 hand9written word recognition7 and facial
recognition. .
20

#.2 &eural net'or(s in medi+ine
Artificial Neural Networks (ANN are currently a 6hot6 research area in medicine
and it is believed that they will receive extensive application to biomedical systems
in the next few years. At the moment, the research is mostly on modeling parts of
the human body and recogni2ing diseases from various scans (e.g. cardiograms,
+AT scans, ultrasonic scans, etc..
Neural networks are ideal in recogni2ing diseases using scans since there is no
need to provide a specific algorithm on how to identify the disease. Neural
networks learn by example so the details of how to recogni2e the disease are not
needed. .hat is needed is a set of examples that are representative of all the
variations of the disease. The 0uantity of examples is not as important as the
60uantity6. The examples need to be selected very carefully if the system is to
perform reliably and efficiently.
$.2.1 %odeling and &iagnosing the 'ardiovascular ()stem
Neural Networks are used experimentally to model the human cardiovascular
system. &iagnosis can be achieved by building a model of the cardiovascular
system of an individual and comparing it with the real time physiological
measurements taken from the patient. !f this routine is carried out regularly,
potential harmful medical conditions can be detected at an early stage and thus
make the process of combating the disease much easier.
A model of an individual6s cardiovascular system must mimic the relationship
among physiological variables (i.e., heart rate, systolic and diastolic blood
pressures, and breathing rate at different physical activity levels. !f a model is
adapted to an individual, then it becomes a model of the physical condition of that
individual. The simulator will have to be able to adapt to the features of any
individual without the supervision of an expert. This calls for a neural network.
Another reason that "ustifies the use of ANN technology is the ability of ANNs to
provide sensor fusion which is the combining of values from several different
sensors. >ensor fusion enables the ANNs to learn complex relationships among the
21
individual sensor values, which would otherwise be lost if the values were
individually analy2ed. !n medical modeling and diagnosis, this implies that even
though each sensor in a set may be sensitive only to a specific physiological
variable, ANNs are capable of detecting complex medical conditions by fusing the
data from the individual biomedical sensors.
$.2.2 Electronic noses
ANNs are used experimentally to implement electronic noses. 8lectronic noses
have several potential applications in telemedicine. Telemedicine is the practice of
medicine over long distances via a communication link. The electronic nose would
identify odours in the remote surgical environment. These identified odours would
then be electronically transmitted to another site where a door generation system
would recreate them. /ecause the sense of smell can be an important sense to the
surgeon, telesmell would enhance telepresent surgery.
$.2.3 *nstant #h)sician
An application developed in the mid9()D:s called the 1instant physician1 trained
an autoassociative memory neural network to store a large number of medical
records, each of which includes information on symptoms, diagnosis, and
treatment for a particular case. After training, the net can be presented with input
consisting of a set of symptoms7 it will then find the full stored pattern that
represents the 1best1 diagnosis and treatment.

#., &eural &et'or(s in business
/usiness is a diverted field with several general areas of speciali2ation such as
accounting or financial analysis. Almost any neural network application would fit
into one business area or financial analysis.
There is some potential for using neural networks for business purposes, including
resource allocation and scheduling. There is also a strong potential for using neural
networks for database mining that is, searching for patterns implicit within the
explicitly stored information in databases. %ost of the funded work in this area is
classified as proprietary. Thus, it is not possible to report on the full extent of the
work going on. %ost work is applying neural networks, such as the $opfield9Tank
network for optimi2ation and scheduling.
22
$.3.1 %ar!eting
There is a marketing application which has been integrated with a neural network
system. The Airline %arketing Tactician (a trademark abbreviated as A%T is a
computer system made of various intelligent technologies including expert
systems. A feed forward neural network is integrated with the A%T and was
trained using back9propagation to assist the marketing control of airline seat
allocations. The adaptive neural approach was amenable to rule expression.
Additionally, the application6s environment changed rapidly and constantly, which
re0uired a continuously adaptive solution. The system is used to monitor and
recommend booking advice for each departure. >uch information has a direct
impact on the profitability of an airline and can provide a technological advantage
for users of the system. J$utchison K >tephens, ()DLM
.hile it is significant that neural networks have been applied to this problem, it is
also important to see that this intelligent technology can be integrated with expert
systems and other approaches to make a functional system. Neural networks were
used to discover the influence of undefined interactions by the various variables.
.hile these interactions were not defined, they were used by the neural system to
develop useful conclusions. !t is also noteworthy to see that neural networks can
influence the bottom line.
$.3.2 'redit Evaluation
The $N+ +ompany, founded by Cobert $echt9Nielsen, has developed several
neural network applications. 3ne of them is the +redit >coring system which
increases the profitability of the existing model up to <LN. The $N+ neural
systems were also applied to mortgage screening. A neural network automated
mortgage insurance underwriting system was developed by the Nestor +ompany.
This system was trained with G:,D applications of which <G)L were certified. The
data related to property and borrower 0ualifications. !n a conservative mode the
system agreed on the underwriters on )LN of the cases. !n the liberal model the
system agreed D,N of the cases. This is system run on an Apollo &N-::: and used
<G:O memory while processing a case file in approximately ( sec.
34
4. Conclusion
The computing world has a lot to gain from neural networks. Their ability to learn
by example makes them very flexible and powerful. Furthermore there is no need
to devise an algorithm in order to perform a specific task7 i.e. there is no need to
understand the internal mechanisms of that task. They are also very well suited for
real time systems because of their fast response and computational times which are
due to their parallel architecture.
Neural networks also contribute to other areas of research such as neurology and
psychology. They are regularly used to model parts of living organisms and to
investigate the internal mechanisms of the brain.
'erhaps the most exciting aspect of neural networks is the possibility that some day
6conscious6 networks might be produced. There is a number of scientists arguing
that consciousness is a 6mechanical6 property and that 6conscious6 neural networks
are a realistic possibility.
Finally, ! would like to state that even though neural networks have a huge
potential we will only get the best of them when they are integrated with
computing, A!, fu22y logic and related sub"ects.
31
5e%erences:
1. An introd$ction to ne$ral co#p$ting. Aleksander' ). and
8orton' ,. 2nd edition
2. "e$ral "etworks at +acific "orthwest "ational 9aboratory
http:66www.e#sl.pnl.gov:24146docs6cie6ne$ral6ne$ral.ho#ep
age.ht#l
3. )nd$strial Applications of "e$ral "etworks :research reports
;sprit' ).(.*roall' <.+.8ason=
4. A "ovel Approach to 8odelling and >iagnosing the
*ardiovasc$lar Syste#
http:66www.e#sl.pnl.gov:24146docs6cie6ne$ral6papers26keller.
wcnn2..abs.ht#l
.. Artificial "e$ral "etworks in 8edicine
http:66www.e#sl.pnl.gov:24146docs6cie6techbrief6"".techbrief
.ht
/. "e$ral "etworks by ;ric >avalo and +atrick "ai#
0. 9earning internal representations by error propagation by
%$#elhart' ,inton and 7illia#s :121/=.
1. ?li#asa$skas' **. :1212=. &he 1212 "e$ro *o#p$ting
@ibliography.
2. >A%+A "e$ral "etwork St$dy :ctober' 1210-(ebr$ary'
1212=. 8)& 9incoln 9ab. "e$ral "etworks' ;ric >avalo and
+atrick "ai#
14.Assi#ov' ) :1214' 12.4=' %obot' @allatine' "ew Aork.
11.;lectronic "oses for &ele#edicine
http:66www.e#sl.pnl.gov:24146docs6cie6ne$ral6papers26keller.
ccc2..abs.ht#l
12.+attern %ecognition of +athology )#ages
http:66kopernik-eth.npac.syr.ed$:12446&ask46pattern.ht#l

32