A Seminar Report on

Artificial Neural Networks

In the partial fulfillment for the degree of B.Tech.

Seminar Presentation (8CS9)

Jodhpur Institute of Engineering & Technology (JIET)

Department of Computer Science & Engineering (YEAR)

Guide
Er. S.K Soni (Depty HOD CSE)

Submitted by
Sonu Singhal Batch: B-2

Acknowledgement
I take this opportunity to express my sincere thanks and deep gratitude to all those people who extended their wholehearted co-operation and helped me in completing this seminar report successfully. I express my deep and sincere gratitude to Prof. S.K. Soni, Deputy HOD at JIET, Jodhpur who provided me an opportunity, inspiration and requisite facility to complete this report in spite of his busy schedule and patiently solving my rather amateurish queries. I am also earnestly thankful to all the concerned faculties who supported and furnished information to make this endeavor a success. SONU SINGHAL B. tech final year Jodhpur Institute of Engineering & Tech.

PREFACE

An artificial neural network (ANN), usually called neural network (NN), is a mathematical model or computational model that is inspired by the structure and/or functional aspects of biological neural networks. A neural network consists of an interconnected group of artificial neurons, and it processes information using a connectionist approach to computation. In most cases an ANN is an adaptive system that changes its structure based on external or internal information that flows through the network during the learning phase. Modern neural networks are non-linear statistical data model in tools. They are usually used to model complex relationships between inputs and outputs or to find patterns in data.A neural network is a parallel system, capable of resolving paradigms that linear computing cannot. The complexity of real neurons is highly abstracted when modelling artificial neurons. These basically consist of inputs (like synapses), which are multiplied by weights (strength of the respective signals), and then computed by a mathematical function which determines the activation of the neuron. Another function (which may be the identity) computes the output of the artificial neuron (sometimes in dependence of a certain threshold). ANNs combine artificial neurons in order to process information

Index
1. Introduction to Artificial Neural Network.

1.1Introduction 1.2Historical Background 1.3Biological Inspiration 1.4Neural N/W v/s Conventional N/W 1.5What are Computer System good atnot so good at
2. Neural Architecture ...

2.1 2.2 2.3 2.4 3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8

Analogy to the brain Artificial Neurons and how they work Electronic Implementation of artificial neurons Artificial Network operation Models Biological Model Mathematical Model Classification of NN Probabilistic Neural Network Generalized Neural Regression Network Linear Network SOFM Network

3. Neural Model and Classification..

4. ANN Processing 4.1 Gathering data for Neural Network.. 4.2 Learning 4.3 Learning Process 4.4 Transfer Function
5. Application of Neural Network

5.1 5.2 5.3 5.4

Area where we can use it? What can do with and what not? Who is concern with NN Software available

6. Advantages. 7. Disadvantages. 8. References 1 Introduction to the Artificial Neural Network

1.1 Introduction
Artificial Neural Networks are relatively crude electronic models based on the neural structure of the brain. The brain basically learns from experience. It is natural proof that some problems that are beyond the scope of current computers are indeed solvable by small energy efficient packages. This brain modeling also promises a less technical way to develop machine solutions. This new approach to computing also provides a more graceful degradation during system overload than its more traditional counterparts. These biologically inspired methods of computing are thought to be the next major advancement in the computing industry. Even simple animal brains are capable of functions that are currently impossible for computers. Computers do rote things well, like keeping ledgers or performing complex math. But computers have trouble recognizing even simple patterns much less generalizing those patterns of the past into actions of the future. Now, advances in biological research promise an initial understanding of the natural thinking mechanism. This research shows that brains store information as patterns. Some of these patterns are very complicated and allow us the ability to recognize individual faces from many different angles. This process of storing information as patterns, utilizing those patterns, and then solving problems encompasses a new field in computing. This field, as mentioned before, does not utilize traditional programming but involves the creation of massively parallel networks and the training of those networks to solve specific problems. This field also utilizes words very different from traditional computing, words like behave, react, self-organize, learn, generalize, and forget. Artificial neural networks (ANN) are among the newest signal-processing technologies in the engineer's toolbox. The field is highly interdisciplinary, but our approach will restrict the view to the engineering perspective. In engineering, neural networks serve two important functions: as pattern classifiers and as nonlinear adaptive filters. We will provide a brief overview of the theory, learning rules, and applications of the most important neural network models. Definitions and Style of Computation An Artificial Neural Network is an adaptive, most often nonlinear system that learns to perform a function (an input/output map) from data. Adaptive means that the system parameters are changed during operation, normally called the training phase. After the training phase the Artificial Neural Network parameters are fixed and the system is deployed to solve the problem at hand (the testing phase). The Artificial Neural Network is built with a systematic step-by-step procedure to optimize a performance criterion or to follow some implicit internal constraint, which is commonly referred to as the learning rule. The input/output training data are fundamental in neural network technology, because they convey the necessary information to "discover" the optimal operating point

An input is presented to the neural network and a corresponding desired or target response set at the output (when this is the case the training is called supervised). . The error information is fed back to the system and adjusts the system parameters in a systematic fashion (the learning rule).The process are repeated until the performance is acceptable. It is clear from this description at the performance hinges heavily on the data. If one does not have data that cover a significant portion of the operating conditions or if they are noisy, then Neural network Technology is probably not the right solution. On the other hand, if there is plenty of data and the problem is poorly understood to derive an approximate model, then neural network technology is a good choice. This operating procedure should be contrasted with the traditional engineering design, made of exhaustive subsystem specifications and intercommunication protocols. In artificial neural networks, the designer chooses the network topology, the performance function, the learning rule, and the criterion to stop the training phase, but the system automatically adjusts the parameters. So, it is difficult to bring a priori information into the design, and when the system does not work properly it is also hard to incrementally refine the solution. But ANN-based solutions are extremely efficient in terms of development time and resources, and in many difficult problems artificial neural networks provide performance that is difficult to match with other technologies. Danker 10 years ago said that "artificial neural networks are the second best way to implement a solution "motivated by the simplicity of their design and because of their universality, only shadowed by the traditional design obtained by studying the physics of the problem. At present, artificial neural networks are emerging as the technology of choice for many applications, such as pattern recognition, prediction, system identification, and control.

1.2 Historical Background

Neural network simulations appear to be a recent development. However, this field was established before the advent of computers, and has survived at least one major setback and several eras. Many important advances have been boosted by the use of inexpensive computer emulations. Following an initial period of enthusiasm, the field survived a period of frustration and disrepute. During this period when funding and professional support was minimal, relatively few researchers made important advances. The first artificial neuron was produced in 1943 by then euro physiologist Warren McCulloch and the logician Walter Pits. But the technology available at that time did not allow them to do too much. 1. First Attempts: There were some initial simulations using formal logic. McCulloch and Pitts (1943) developed models of neural networks based on their understanding of neurology. These models made several assumptions about how neurons worked. Their networks were based on simple neurons, which were considered to be binary devices with fixed thresholds. The results of their model were simple logic functions such as "a or b" and "a and b". Another attempt was by using computer simulations. Two groups (Farley and Clark, 1954; Rochester, Holland, Haibit and Duda, 1956). The first group (IBM researchers) maintained closed contact with neuroscientists at McGill University. So when ever their models did not work, they consulted the neuroscientists. This interaction established a multidisciplinary trend, which continues to the present day. 2. Promising & Emerging Technology: Not only was neuroscience influential in the development of neural networks, but psychologists and engineers also contributed to the progress of neural network simulations. Rosenblatt (1958) stirred considerable interest and activity in the field when he designed and developed the Perception. The Perception had three layers with the middle layer known as the association layer. This system could learn to connect or associate a given input to a random output unit. Another system was the ADALINE (Adaptive Linear Element), which was developed in1960 by Widrow and Hoff (of Stanford University). The ADALINE was an analogue electronic device made from simple components. The method used for learning was different to that of the Perception; it employed the Least-Mean-Squares (LMS) learning rule. 3. Period of Frustration & Disrepute: In 1969 Minsky and Papert wrote a book in which they generalized the limitations of single layer Perceptions to multilayered systems. In the book they said: "...our intuitive judgment that the extension (to multilayer systems) is sterile". The significant result of their book was to eliminate funding for research with neural network simulations. The conclusions supported the disenchantment of researchers in the field. As a result, considerable prejudice against this field was activated. 4. Innovation: Although public interest and available funding were minimal, several researchers continued working to develop neuromorphically based computational methods for problems such as pattern recognition. During this period several paradigms were generated which modern work continues to enhance. Grossberg's (Steve Grossberg and Gail Carpenter in 1988) influence founded a school of thought, which explores resonating algorithms. They developed the ART (Adaptive Resonance Theory) networks based on biologically plausible models. Anderson and Kohonen developed associative techniques independent of each other. (A. Henry Klopf) in 1972 developed a basis for learning in artificial neurons based on a biological

principle for neuronal learning called homeostasis. The original network was published in 1975 and was called the Cognition . 5. Re-Emergence: Progress during the late 1970s and early 1980s was important to the reemergence on interest in the neural network field. Several factors influenced this movement. For example, comprehensive books and conferences provided a forum for people in diverse fields with specialized technical languages, and the response to conferences and publications was quite positive. The news media picked up on the increased activity and tutorials helped disseminate the technology. Academic programs appeared and courses were introduced at most major Universities (in US and Europe).Attention is now focused on funding levels throughout Europe, Japan and the US and as this funding becomes available, several new commercial with applications in industry and financial institutions are emerging. 6. Today: Significant progress has been made in the field of neural networks-enough to attract a great deal of attention and fund further research. Advancement beyond current commercial applications appears to be possible, and research is advancing the field on many fronts. Neutrally based chips are emerging and applications to complex problems developing. Clearly, today is a period of transition for neural network technology

1.3 Biological Inspiration

Neural networks grew out of research in Artificial Intelligence; specifically, attempts to mimic the fault-tolerance and capacity to learn of biological neural systems by modeling the low-level structure of the brain (see Patterson, 1996). The main branch of Artificial Intelligence research in the 1960s -1980s produced Expert Systems. These are based upon a high-level model of reasoning processes (specifically, the concept that our reasoning processes are built upon manipulation of symbols). It became rapidly apparent that these systems, although very useful in some domains, failed to capture certain key aspects of human intelligence. According to one line of speculation, this was due to their failure to mimic the underlying structure of the brain. The axons of one cell connect to the dendrites of another via a synapse. When a neuron is activated, it fires an electrochemical signal along the axon. This signal crosses the synapses to other neurons, which may in turn fire. A neuron fires only if the total signal received at the cell body from the dendrites exceeds a certain level (the firing threshold). 1. The strength of the signal received by a neuron (and therefore its chances of firing)critically depends on the efficacy of the synapses. Each synapse actually contains agap, with neurotransmitter chemicals poised to transmit a signal across the gap. Oneof the most influential researchers into neurological systems (Donald Hebb) postulated that learning consisted principally in altering the "strength" of synapticconnections. For example, in the classic Pavlovian conditioning experiment, where a bell is rung just before dinner is delivered to a dog, the dog rapidly learns to associatethe ringing of a bell with the eating of food. 2. The synaptic connections between the appropriate part of the auditory cortex and thesalivation glands are strengthened, so that when the auditory cortex is stimulated bythe sound of the bell the dog starts to salivate. Recent research in cognitive science, in particular in the area of non-conscious information processing, have further demonstrated the enormous capacity of the human mind to infer ("learn") simpleinput-output co variations from extremely complex stimuli (e.g., see Lewicki, Hill,and Czyzewska, 1992). 3. Thus, from a very large number of extremely simple processing units (each performing a weighted sum of its inputs, and then firing a binary signal if the total input exceeds a certain level) the brain manages to perform extremely complex tasks. Of course, there is a great deal of complexity in the brain which has not been discussed here, but it is interesting that artificial neural network scan achieve some remarkable results using a model not much more complex than this 1.4 Neural networks versus conventional computers Neural networks take a different approach to problem solving than that of conventionalcomputers. Conventional computers use an algorithmic approach i.e. the computer follows a setof instructions in order to solve a problem. Unless the specific steps that the computer needs tofollow are known the computer cannot solve the problem. That restricts the problem solvingcapability of conventional computers to problems that we already understand and know how tosolve. But computers would be so much more useful if they could do things

that we don't exactlyknow how to do. Neural networks process information in a similar way the human brain does. The network iscomposed of a large number of highly interconnected processing elements(neurones) working in parallel to solve a specific problem. Neural networks learn by example. They cannot be programmed to perform a specific task. The examples must be selected carefully otherwiseuseful time is wasted or even worse the network might be functioning incorrectly. Thedisadvantage is that because the network finds out how to solve the problem by itself, itsoperation can be unpredictable.On the other hand, conventional computers use a cognitive approach to problem solving; the waythe problem is to solved must be known and stated in small unambiguous instructions. Theseinstructions are then converted to a high level language program and then into machine code thatthe computer can understand. These machines are totally predictable; if anything goes wrong isdue to a software or hardware fault. Neural networks and conventional algorithmic computers are not in competition but complementeach other. There are tasks are more suited to an algorithmic approach like arithmetic operationsand tasks that are more suited to neural networks. Even more, a large number of tasks, requiresystems that use a combination of the two approaches (normally a conventional computer is usedto supervise the neural network) in order to perform at maximum efficiency.

1.5 What are (everyday) computer systems good at... .....and not so good at?

Advantages: A neural network can perform tasks that a linear program can not. When an element of the neural network fails, it can continue without any problem by their parallel nature. A neural network learns and does not need to be reprogrammed. It can be implemented in any application. It can be implemented without any problem Disadvantages: The neural network needs training to operate. The architecture of a neural network is different from the architecture of microprocessors therefore needs to be emulated. Requires high processing time for large neural networks 2 Neural Architecture 2.1 Analogy to the Brain The exact workings of the human brain are still a mystery. Yet, some aspects of this amazing processor are known. In particular, the most basic element of the human brain is a specific type of cell which, unlike the rest of the body, doesn't appear to regenerate. Because this type of cell is the only part of the body that isn't slowly replaced, it is assumed that these cells are what provide us with our abilities to remember, think, and apply previous experiences to our every action. These cells, all 100 billion of them, are known as neurons. Each of these neurons can connect with up to 200,000 other neurons, although 1,000 to 10,000 are typical. The power of the human mind comes from the sheer numbers of these basic components and the multiple connections between them. It also comes from genetic programming and learning. The individual neurons are complicated. They have a myriad of parts, sub-systems, and control

mechanisms. They convey information via a host of electrochemical pathways. There are over one hundred different classes of neurons, depending on the classification method used. Together these neurons and their connections form a process which is not binary, not stable, and not synchronous. In short, it is nothing like the currently available electronic computers, or even artificial neural networks. These artificial neural networks try to replicate only the most basic elements of this complicated, versatile, and powerful organism. They do it in a primitive way. But for the software engineer who is trying to solve problems, neural computing was never about replicating human brains. It is about machines and a new way to solve problems. 2.2 Artificial Neurons and How They Work The fundamental processing element of a neural network is a neuron. This building block of human awareness encompasses a few general capabilities. Basically, a biological neuron receives inputs from other sources, combines them in some way, performs a generally nonlinear operation on the result, and then outputs the final result. Figure 2.2.1 shows the relationship of these four parts. Figure 2.2.1 A Simple Neuron. Within humans there are many variations on this basic type of neuron, further complicating man's attempts at electrically replicating the process of thinking. Yet, all natural neurons have the same four basic components. These components are known by their biological names - dendrites,

Soma, axon and synapses. Dendrites are hair-like extensions of the soma which act like input channels. These input channels receive their input through the synapses of other neurons. The soma then processes these incoming signals over time. The soma then turns that processed value into an output, which is sent out to other neurons through the axon and the synapses. Recent experimental data has provided further evidence that biological neurons are structurally more complex than the simplistic explanation above. They are significantly more complex than the existing artificial neurons that are built into today's artificial neural networks. As biology provides a better understanding of neurons, and as technology advances, network designers can continue to improve their systems by building upon man's understanding of the biological brain. But currently, the goal of artificial neural networks is not the grandiose recreation of the brain. On the contrary, neural network researchers are seeking an understanding of nature's capabilities for which people can engineer solutions to problems that have not been solved by traditional computing. To do this, the basic unit of neural networks,

the artificial neurons, simulates the four basic functions of natural neurons. Figure 2.2.2 shows a fundamental representation of an artificial neuron. Figure 2.2.2 A Basic Artificial Neuron. In Figure 2.2.2, various inputs to the network are represented by the mathematical symbol, x(n).Each of these inputs is multiplied by a connection weight. These weights are represented byw(n). In the simplest case, these products are simply summed, fed through a transfer function to generate a result, and then output. This process lends itself to physical implementation on a large scale in a small package. This electronic implementation is still possible with other network structures which utilize different summing functions as well as different transfer functions. 14

Some applications require "black and white," or binary, answers. These applications include there cognition of text, the identification of speech, and the image deciphering of scenes. These applications are required to turn real-world inputs into discrete values. These potential values are limited to some known set, like the ASCII characters or the most common 50,000 English words. Because of this limitation of output options, these applications don't always utilize networks composed of neurons that simply sum up, and thereby smooth, inputs. These networks may utilize the binary properties of ORing and ANDing of inputs. These functions, and many others, can be built into the summation and transfer functions of a network. Other networks work on problems where the resolutions are not just one of several known values. These networks need to be capable of an infinite number of responses. Applications of this type include the "intelligence" behind robotic movements. This "intelligence" processes inputs and then creates outputs, which actually cause some device to move. That movement can span an infinite number of very precise motions. These networks do indeed want to smooth their inputs which, due to limitations of sensors, come in non-continuous bursts, say thirty times a second. To do that, they might accept these inputs, sum that data, and then produce an output by, for example, applying a hyperbolic tangent as a transfer function. In this manner, output values from the network are continuous and satisfy more real world interfaces. Other applications might simply sum and compare to a threshold, thereby producing one of two possible outputs, a zero or a one. Other functions scale the outputs to match the application, such as the values minus one and one. Some functions even integrate the input data over time, creating timedependent networks.

2.3 Electronic Implementation of Artificial Neurons In currently available software packages these artificial neurons are called "processing elements" and have many more capabilities than the simple artificial neuron described above. Those capabilities will be discussed later in this report. Figure 2.2.3 is a more detailed schematic of this still simplistic artificial neuron.