Neural Network Programming with Java Create and

unleash the power of neural networks by

implementing professional Java code 1st Edition
Souza download
https://textbookfull.com/product/neural-network-programming-with-
java-create-and-unleash-the-power-of-neural-networks-by-
implementing-professional-java-code-1st-edition-souza/

Download more ebook from https://textbookfull.com

We believe these products will be a great fit for you. Click
the link to download now, or visit textbookfull.com
to discover even more!

Artificial Neural Networks with Java - Tools for

Building Neural Network Applications 1st Edition Igor
Livshin

https://textbookfull.com/product/artificial-neural-networks-with-
java-tools-for-building-neural-network-applications-1st-edition-
igor-livshin/

Learning Network Programming with Java 1st Edition

Reese Richard M

https://textbookfull.com/product/learning-network-programming-
with-java-1st-edition-reese-richard-m/

Java Network Programming Fourth Edition Harold Elliotte

https://textbookfull.com/product/java-network-programming-fourth-
edition-harold-elliotte/

Learning Java Beginning programming with java for

dummies First Edition John Bach

https://textbookfull.com/product/learning-java-beginning-
programming-with-java-for-dummies-first-edition-john-bach/
Learning Java An Introduction to Real World Programming
with Java Marc Loy

https://textbookfull.com/product/learning-java-an-introduction-
to-real-world-programming-with-java-marc-loy/

A Functional Approach to Java: Augmenting Object-

Oriented Java Code with Functional Principles 1st
Edition Ben Weidig

https://textbookfull.com/product/a-functional-approach-to-java-
augmenting-object-oriented-java-code-with-functional-
principles-1st-edition-ben-weidig/

Learning Java An Introduction to Real World Programming

with Java 5th Edition Marc Loy

https://textbookfull.com/product/learning-java-an-introduction-
to-real-world-programming-with-java-5th-edition-marc-loy/

Motivation Effort and the Neural Network Model Theodore

Wasserman

https://textbookfull.com/product/motivation-effort-and-the-
neural-network-model-theodore-wasserman/

Deep Learning with JavaScript: Neural networks in

TensorFlow.js 1st Edition Shanqing Cai

https://textbookfull.com/product/deep-learning-with-javascript-
neural-networks-in-tensorflow-js-1st-edition-shanqing-cai/
Neural Network Programming
with Java

Unleash the power of neural networks by implementing

professional Java code

Fábio M. Soares
Alan M.F. Souza

BIRMINGHAM - MUMBAI
Neural Network Programming with Java

Copyright © 2016 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval
system, or transmitted in any form or by any means, without the prior written
permission of the publisher, except in the case of brief quotations embedded in
critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy
of the information presented. However, the information contained in this book is
sold without warranty, either express or implied. Neither the authors, nor Packt
Publishing, and its dealers and distributors will be held liable for any damages
caused or alleged to be caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the
companies and products mentioned in this book by the appropriate use of capitals.
However, Packt Publishing cannot guarantee the accuracy of this information.

First published: January 2016

Production reference: 1060116

Published by Packt Publishing Ltd.

Livery Place
35 Livery Street
Birmingham B3 2PB, UK.

ISBN 978-1-78588-090-2

www.packtpub.com
 Credits

Authors Project Coordinator

Fábio M. Soares Kinjal Bari
Alan M.F. Souza
Proofreader
Reviewer Safis Editing
Saeed Afzal
Indexer
Commissioning Editor Hemangini Bari
Amarabha Banerjee
Graphics
Acquisition Editor Disha Haria
Rahul Nair
Production Coordinator
Content Development Editor Nilesh Mohite
Riddhi Tuljapurkar
Cover Work
Technical Editor Nilesh Mohite
Vivek Pala

Copy Editor
Tani Kothari
 About the Authors

Fábio M. Soares holds a master's degree in applied computing from UFPA and
is currently a PhD candidate at the same university. He has been designing neural
network solutions since 2004 and has developed applications with this technique in
several fields, ranging from telecommunications to chemistry process modeling, and
his research topics cover supervised learning for data-driven modeling.
He is also self-employed, offering services such as IT infrastructure management as
well as database administration to a number of small- and medium-sized companies
in northern Brazil. In the past, he has worked for big companies such as Albras, one
of the most important aluminium smelters in the world, and Eletronorte, a great
power supplier in Brazil. He also has experience as a lecturer, having worked at the
Federal Rural University of Amazon and as a Faculty of Castanhal, both in the state
of Pará, teaching subjects involving programming and artificial intelligence.
He has published a number of works, many of them available in English, all
including the topics of artificial intelligence applied to some problem. His
publications include conference proceedings, such as the TMS (The Minerals Metals
and Materials Society), Light Metals and the Intelligent Data Engineering and
Automated Learning. He has also has published two book chapters for Intech.

I would like to give a special acknowledgement to God for having

given me the opportunity to get access to rich knowledge on this
theme, which I simply love doing research on. Special thanks to my
family, my father, Josafá, and mother, Maria Alice (in memoriam),
who would be very proud of me for this book, and also my brother,
Flávio, my aunt, Maria Irenice, as well as all my relatives who always
supported me in some way during my studies. I would also like to
thank the support of my advisor, Prof. Roberto Limão. I am very
grateful to him for having invited me to work with him on many
projects regarding artificial intelligence and neural networks. Also,
special thanks to my partners and former partners from Exodus
Sistemas, who have helped me in my challenges in programming and
IT infrastructure. Finally, I'd like to thank my friend Alan Souza, who
wrote this book with me, for having extended to me this authorship.
Alan M.F. Souza is computer engineer from Instituto de Estudos Superiores
da Amazônia (IESAM). He holds a post-graduate degree in project management
software and a master's degree in industrial processes (applied computing)
from Universidade Federal do Pará (UFPA). He has been working with neural
networks since 2009 and has worked with IT Brazilian companies developing in
Java, PHP, SQL, and other programming languages since 2006. He is passionate
about programming and computational intelligence. Currently, he is a professor at
Universidade da Amazônia (UNAMA) and a PhD candidate at UFPA.

Since I was a kid, I thought about writing a book. So, this book is a
dream come true and the result of hard work. I'd like to thank God
for giving me this opportunity. I'd also like to thank my father, Célio,
my mother, Socorro, my sister, Alyne, and my amazing wife, Tayná,
for understanding my absences and worries at various moments. I
am grateful to all the members of my family and friends for always
supporting me in difficult times and wishing for my success. I'd like
to thank all the professors who passed through my life, especially
Prof. Roberto Limão for introducing me the very first neural network
concept. I must register my gratitude to Fábio Soares for this great
partnership and friendship. Finally, I must appreciate the tireless
team at Packt Publishing for the invitation and for helping us in the
production process as a whole.
 About the Reviewer

Saeed Afzal, also known as Smac Afzal, is a professional software engineer and
technology enthusiast based in Pakistan. He specializes in solution architecture and
the implementation of scalable high-performance applications.

He is passionate about providing automation solutions for different business needs

on the Web. His current research and work includes the futuristic implementation of
a next-generation web development framework, which reduces development time
and cost and delivers productive websites with many necessary and killer features by
default. He is hopeful of launching his upcoming technology in 2016.

He has also worked on the book Cloud Bees Development by Packt Publishing.

You can found out more about his skills and experience at http://sirsmac.com.
He can be contacted at sirsmac@gmail.com.

I would like to thank the Allah Almighty, my parents, and my wife,

Dr. H. Zara Saeed, for all their encouragement.
 www.PacktPub.com

Support files, eBooks, discount offers, and more

For support files and downloads related to your book, please visit www.PacktPub.com.

Did you know that Packt offers eBook versions of every book published, with PDF
and ePub files available? You can upgrade to the eBook version at www.PacktPub.
com and as a print book customer, you are entitled to a discount on the eBook copy.
Get in touch with us at service@packtpub.com for more details.

At www.PacktPub.com, you can also read a collection of free technical articles, sign
up for a range of free newsletters and receive exclusive discounts and offers on Packt
books and eBooks.
TM

https://www2.packtpub.com/books/subscription/packtlib

Do you need instant solutions to your IT questions? PacktLib is Packt's online digital
book library. Here, you can search, access, and read Packt's entire library of books.

Why subscribe?
• Fully searchable across every book published by Packt
• Copy and paste, print, and bookmark content
• On demand and accessible via a web browser

Free access for Packt account holders

If you have an account with Packt at www.PacktPub.com, you can use this to access
PacktLib today and view 9 entirely free books. Simply use your login credentials for
immediate access.
 Table of Contents
Preface vii
Chapter 1: Getting Started with Neural Networks 1
Discovering neural networks 2
Why artificial neural network? 2
How neural networks are arranged 4
The very basic element – artificial neuron 5
Giving life to neurons – activation function 5
The fundamental values – weights 6
An important parameter – bias 7
The parts forming the whole – layers 7
Learning about neural network architectures 8
Monolayer networks 8
Multilayer networks 9
Feedforward networks 9
Feedback networks 10
From ignorance to knowledge – learning process 10
Let the implementations begin! Neural networks in practice 11
Summary 18
Chapter 2: How Neural Networks Learn 19
Learning ability in neural networks 19
How learning helps to solve problems 20
Learning paradigms 20
Supervised learning 20
Unsupervised learning 21
Systematic structuring – learning algorithm 22
Two stages of learning – training and testing 23
The details – learning parameters 24
Error measurement and cost function 25

[i]
Table of Contents

Examples of learning algorithms 26

Perceptron 26
Delta rule 27
Coding of the neural network learning 27
Learning parameter implementation 27
Learning procedure 29
Class definitions 30
Two practical examples 37
Perceptron (warning system) 37
ADALINE (traffic forecast) 41
Summary 46
Chapter 3: Handling Perceptrons 47
Studying the perceptron neural network 48
Applications and limitations of perceptrons 48
Linear separation 48
Classical XOR case 50
Popular multilayer perceptrons (MLPs) 52
MLP properties 52
MLP weights 53
Recurrent MLP 54
MLP structure in an OOP paradigm 55
Interesting MLP applications 56
Classification in MLPs 56
Regression in MLPs 58
Learning process in MLPs 60
Simple and very powerful learning algorithm – Backpropagation 61
Elaborate and potent learning algorithm – Levenberg–Marquardt 63
Hands-on MLP implementation! 65
Backpropagation in action 68
Exploring the code 68
Levenberg–Marquardt implementation 72
Practical application – types of university enrolments 75
Summary 78
Chapter 4: Self-Organizing Maps 79
Neural networks' unsupervised way of learning 80
Some unsupervised learning algorithms 80
Competitive learning or winner takes all 82

[ ii ]
 Table of Contents

Kohonen self-organizing maps (SOMs) 84

One-Dimensional SOM 85
Two-Dimensional SOM 86
Step-by-step of SOM learning 88
How to use SOMs 89
Coding of the Kohonen algorithm 90
Exploring the Kohonen class 92
Kohonen implementation (clustering animals) 95
Summary 98
Chapter 5: Forecasting Weather 99
Neural networks for prediction problems 100
No data, no neural net – selecting data 101
Knowing the problem – weather variables 102
Choosing input and output variables 103
Removing insignificant behaviors – Data filtering 103
Adjusting values – data preprocessing 104
Equalizing data – normalization 105
Java implementation for weather prediction 107
Plotting charts 107
Handling data files 108
Building a neural network for weather prediction 109
Empirical design of neural networks 112
Choosing training and test datasets 112
Designing experiments 113
Results and simulations 113
Summary 116
Chapter 6: Classifying Disease Diagnosis 117
What are classification problems, and how can neural networks
be applied to them? 118
A special type of activation function – Logistic regression 119
Multiple classes versus binary classes 120
Comparing the expected versus produced results – the
confusion matrix 121
Classification measures – sensitivity and specificity 122
Applying neural networks for classification 123
Disease diagnosis with neural networks 126
Using ANN to diagnose breast cancer 126
Applying NN for an early diagnosis of diabetes 131
Summary 134

[ iii ]
Table of Contents

Chapter 7: Clustering Customer Profiles 135

Clustering task 136
Cluster analysis 137
Cluster evaluation and validation 138
External validation 138
Applied unsupervised learning 139
Neural network of radial basis functions 139
Kohonen neural network 140
Types of data 141
Customer profiling 142
Preprocessing data 142
Implementation in Java 143
Card credit analysis for customer profiling 143
Summary 149
Chapter 8: Pattern Recognition (OCR Case) 151
What is pattern recognition all about? 152
Definition of classes among tons of data 152
What if the undefined classes are undefined? 153
External validation 154
How to apply neural networks in pattern recognition 154
Preprocessing the data 155
The OCR problem 156
Simplifying the task – digit recognition 156
Approach to digit representation 157
Let the coding begin! 157
Generating data 158
Building the neural network 160
Testing and redesigning – trial and error 161
Results 163
Summary 166
Chapter 9: Neural Network Optimization and Adaptation 167
Common issues in neural network implementations 168
Input selection 168
Data correlation 169
Dimensionality reduction 170
Data filtering 171
Structure selection 171

[ iv ]
 Table of Contents

Online retraining 172

Stochastic online learning 174
Implementation 174
Application 176
Adaptive neural networks 179
Adaptive resonance theory 179
Implementation 180
Summary 182
Appendix A: Setting up the NetBeans Environment 183
Download and install NetBeans 183
Setting up the NetBeans environment 188
Importing a project 191
Programming and running code with NetBeans 194
Debugging with NetBeans 195
Appendix B: Setting Up the Eclipse Environment 199
Download and install Eclipse 199
Setting up the Eclipse environment 204
Importing a project 205
Programming and running code with the Eclipse IDE 210
Debugging with the Eclipse IDE 211
Appendix C: References 213
Chapter 1 – Getting Started with Neural Networks 213
Chapter 2 – How Neural Networks Learn 213
Chapter 3 – Working with Perceptrons 213
Chapter 4 – Self-Organizing Maps 214
Chapter 5 – Forecasting the Weather 214
Chapter 6 – Disease Diagnosis 214
Chapter 7 – Clustering Customer Profiles 215
Chapter 8 – Pattern Recognition (the OCR Case) 215
Chapter 9 – Neural Network Optimization and Adaptation 215
Index 217

[v]
 Preface
The life of a programmer can be described as a continual never-ending learning
pathway. A programmer always faces challenges regarding new technology or new
approaches. Generally, during our lives, although we become used to repeated
things, we are always subjected to learn something new. The process of learning is
one of the most interesting topics in science, and there are a number of attempts to
describe or reproduce the human learning process.

The writing of this book was guided by the challenge of facing new content and
then mastering it. While the name neural networks may appear strange or even give
an idea that this book is about neurology, we strived to simplify these nuances by
focusing on your reasons for deciding to purchase this book. We intended to build
a framework that shows you that neural networks are actually simple and easy to
understand, and absolutely no prior knowledge on this topic is required to fully
understand the concepts we present here.

So, we encourage you to explore the content of this book to the fullest, beholding
the power of neural networks when confronting big problems but always with the
point of view of a beginner. Every concept addressed in this book is explained in easy
language, and also with a technical background. Our mission in this book is to give
you an insight into intelligent applications that can be written using a simple language.

Finally, we would like to thank all those who directly or indirectly have contributed
to this book and supported us from the very beginning, right from the Federal
University of Pará, which is the university that we graduated from, to the data and
component providers INMET (Brazilian Institute of Meteorology), Proben1, and
JFreeCharts. We want to give special thanks to our advisor Prof. Roberto Limão, who
introduced us to the subject of neural networks and coauthored many papers with
us in this field. We also acknowledge the work performed by several authors cited
in the references, which gave us a broader vision on neural networks and insights on
how to adapt them to the Java language in a didactic way.

[ vii ]
Preface

We welcome you to have a very pleasurable reading experience and you are
encouraged to download the source code and follow the examples presented
in this book.

What this book covers

Chapter 1, Getting Started with Neural Networks, is an introductory foundation on
the neural networks and what they are designed for. You will be presented with
the basic concepts involved in this book. A brief review of the Java programming
language is provided. As in all subsequent chapters, an implementation of a neural
network in Java code is also provided.

Chapter 2, How Neural Networks Learn, covers the learning process of neural networks
and shows how data is used to that end. The complete structure and design of a
learning algorithm is presented here.

Chapter 3, Handling Perceptrons, covers the use of perceptrons, which are one of the
most commonly used neural network architectures. We present a neural network
structure containing layers of neurons and show how they can learn by data in
basic problems.

Chapter 4, Self-Organizing Maps, shows an unsupervised neural network architecture

(the Self-Organising Map), which is applied to finding patterns or clusters in records.

Chapter 5, Forecasting Weather, is the first practical chapter showing an interesting

application of neural networks in forecasting values, namely weather data.

Chapter 6, Classifying Disease Diagnostics, covers another useful task neural networks
are very good at—classification. In this chapter, you will be presented with a very
didactic but interesting application for disease diagnosis.

Chapter 7, Clustering Customer Profiles, talks about how neural networks are able to
find patterns in data, and a common application is to group customers that share the
same properties of buying.

Chapter 8, Pattern Recognition (OCR Case), talks about a very interesting and amazing
capability of recognizing patterns, including optical character recognition, and this
chapter explores how this can be done with neural networks in the Java language.

Chapter 9, Neural Network Optimization and Adaptation, shows advancements

regarding how to optimize and add adaptability to neural networks, thereby
strengthening their power.

[ viii ]
 Preface

What you need for this book

You'll need Netbeans (www.netbeans.org) or Eclipse (www.eclipse.org). Both are free
and available for download at the previously mentioned websites.

Who this book is for

This book is targeted at both developers and enthusiasts who have a basic or even no
Java programming knowledge. No previous knowledge of neural networks is required,
this book will teach from scratch. Even if you are familiar with neural networks and/or
other machine learning techniques but have little or no experience with Java, this book
will take you to the level at which you will be able to develop useful applications. Of
course, if you know basic programming concepts, you will benefit most from this book,
but no prior experience is required.

Conventions
In this book, you will find a number of text styles that distinguish between different
kinds of information. Here are some examples of these styles and an explanation of
their meaning.

Code words in text, database table names, folder names, filenames, file
extensions, pathnames, dummy URLs, user input, and Twitter handles are shown
as follows: "In Java projects, the calculation of these values is done through the
Classification class."

A block of code is set as follows:

Data cardDataInput = new Data("data", "card_inputs_training.csv");
Data cardDataInputTestRNA = new Data("data", "card_inputs_test.
csv");
Data cardDataOutputTestRNA = new Data("data", "card_output_test.
csv");

New terms and important words are shown in bold.

Warnings or important notes appear in a box like this.

Tips and tricks appear like this.

[ ix ]
Preface

Reader feedback
Feedback from our readers is always welcome. Let us know what you think about
this book—what you liked or disliked. Reader feedback is important for us as it helps
us develop titles that you will really get the most out of.

To send us general feedback, simply e-mail feedback@packtpub.com, and mention

the book's title in the subject of your message.

If there is a topic that you have expertise in and you are interested in either writing
or contributing to a book, see our author guide at www.packtpub.com/authors.

Customer support
Now that you are the proud owner of a Packt book, we have a number of things to
help you to get the most from your purchase.

Downloading the example code

You can download the example code files from your account at http://www.
packtpub.com for all the Packt Publishing books you have purchased. If you
purchased this book elsewhere, you can visit http://www.packtpub.com/support
and register to have the files e-mailed directly to you.

Errata
Although we have taken every care to ensure the accuracy of our content, mistakes
do happen. If you find a mistake in one of our books—maybe a mistake in the text or
the code—we would be grateful if you could report this to us. By doing so, you can
save other readers from frustration and help us improve subsequent versions of this
book. If you find any errata, please report them by visiting http://www.packtpub.
com/submit-errata, selecting your book, clicking on the Errata Submission Form
link, and entering the details of your errata. Once your errata are verified, your
submission will be accepted and the errata will be uploaded to our website or added
to any list of existing errata under the Errata section of that title.

To view the previously submitted errata, go to https://www.packtpub.com/books/

content/support and enter the name of the book in the search field. The required
information will appear under the Errata section.

[x]
 Preface

Piracy
Piracy of copyrighted material on the Internet is an ongoing problem across all
media. At Packt, we take the protection of our copyright and licenses very seriously.
If you come across any illegal copies of our works in any form on the Internet, please
provide us with the location address or website name immediately so that we can
pursue a remedy.

Please contact us at copyright@packtpub.com with a link to the suspected pirated

material.

We appreciate your help in protecting our authors and our ability to bring you
valuable content.

Questions
If you have a problem with any aspect of this book, you can contact us at
questions@packtpub.com, and we will do our best to address the problem.

[ xi ]
 Getting Started with
Neural Networks
In this chapter, we will introduce neural networks and what they are designed for.
This chapter serves as a foundation layer for the subsequent chapters, while
it presents the basic concepts for neural networks. In this chapter, we will cover
the following:

• Artificial Neurons
• Weights and Biases
• Activation Functions
• Layers of Neurons
• Neural Network Implementation in Java

[1]
Getting Started with Neural Networks

Discovering neural networks

First, the term "neural networks" may create a snapshot of a brain in our minds,
particularly for those who have just been introduced to it. In fact, that's right, we
consider the brain to be a big and natural neural network. However, what if we talk
about artificial neural networks (ANNs)? Well, here comes an opposite word to
natural, and the first thing now that comes into our head is an image of an artificial
brain or a robot, given the term "artificial." In this case, we also deal with creating
a structure similar to and inspired by the human brain; therefore, this can be called
artificial intelligence. So, the reader who doesn't have any previous experience with
ANN now may be thinking that this book teaches how to build intelligent systems,
including an artificial brain, capable of emulating the human mind using Java codes,
isn't it? Of course, we will not cover the creation of artificial thinking machines such
as those from the Matrix trilogy movies; however, this book will discuss several
incredible capabilities that these structures can do. We will provide the reader
with Java codes for defining and creating basic neural network structures, taking
advantage of the entire Java programming language framework.

Why artificial neural network?

We cannot begin talking about neural networks without understanding their origins,
including the term as well. We use the terms neural networks (NN) and ANN
interchangeably in this book, although NNs are more general, covering the natural
neural networks as well. So, what actually is an ANN? Let's explore a little of the
history of this term.

In the 1940s, the neurophysiologist Warren McCulloch and the mathematician

Walter Pits designed the first mathematical implementation of an artificial neuron
combining the neuroscience foundations with mathematical operations. At that
time, many studies were being carried out on understanding the human brain and
how and if it could be simulated, but within the field of neuroscience. The idea
of McCulloch and Pits was a real novelty because it added the math component.
Further, considering that the brain is composed of billions of neurons, each one
interconnected with another million, resulting in some trillions of connections, we
are talking about a giant network structure. However, each neuron unit is very
simple, acting as a mere processor capable to sum and propagate signals.

[2]
 Chapter 1

On the basis of this fact, McCulloch and Pits designed a simple model for a
single neuron, initially to simulate the human vision. The available calculators or
computers at that time were very rare but capable of dealing with mathematical
operations quite well; on the other hand, even today tasks such as vision and sound
recognition are not easily programmed without the use of special frameworks, as
opposed to the mathematical operations and functions. Nevertheless, the human
brain can perform these latter tasks more efficiently than the first ones, and this fact
really instigates scientists and researchers.

So, an ANN is supposed to be a structure to perform tasks such as pattern

recognition, learning from data, and forecasting trends, just like an expert can do on
the basis of knowledge, as opposed to the conventional algorithmic approach that
requires a set of steps to be performed to achieve a defined goal. An ANN instead
has the capability to learn how to solve some task by itself, because of its highly
interconnected network structure.

Tasks Quickly Solvable by Humans Tasks Quickly Solvable by Computers

Classification of images Complex calculation
Voice recognition Grammatical error correction
Face identification Signal processing
Forecast events on the basis of Operating system management
experience

[3]
Getting Started with Neural Networks

How neural networks are arranged

It can be said that the ANN is a nature-inspired structure, so it does have similarities
with the human brain. As shown in the following figure, a natural neuron is
composed of a nucleus, dendrites, and axon. The axon extends itself into several
branches to form synapses with other neurons' dendrites.

So, the artificial neuron has a similar structure. It contains a nucleus (processing
unit), several dendrites (analogous to inputs), and one axon (analogous to output), as
shown in the following figure:

The links between neurons form the so-called neural network, analogous to the
synapses in the natural structure.

[4]
 Chapter 1

The very basic element – artificial neuron

Natural neurons have proven to be signal processors since they receive micro signals
in the dendrites that can trigger a signal in the axon depending on their strength or
magnitude. We can then think of a neuron as having a signal collector in the inputs
and an activation unit in the output that can trigger a signal that will be forwarded
to other neurons. So, we can define the artificial neuron structure as shown in the
following figure:

In natural neurons, there is a threshold potential that

when reached, fires the axon and propagates the signal to
the other neurons. This firing behavior is emulated with
activation functions, which have proven to be useful in
representing nonlinear behaviors in the neurons.

Giving life to neurons – activation function

The neuron's output is given by an activation function. This component adds
nonlinearity to neural network processing, which is needed because the natural
neuron has nonlinear behaviors. An activation function is usually bounded between
two values at the output, therefore being a nonlinear function, but in some special
cases, it can be a linear function.

The four most used activation functions are as follows:

• Sigmoid
• Hyperbolic tangent
• Hard limiting threshold
• Purely linear

[5]
Getting Started with Neural Networks

The equations and charts associated with these functions are shown in the
following table:

Function Equation Chart

Sigmoid

Hyperbolic
tangent

Hard
limiting
threshold

Linear

The fundamental values – weights

In neural networks, weights represent the connections between neurons and have the
capability to amplify or attenuate neuron signals, for example, multiply the signals,
thus modifying them. So, by modifying the neural network signals, neural weights
have the power to influence a neuron's output, therefore a neuron's activation will
be dependent on the inputs and on the weights. Provided that the inputs come from
other neurons or from the external world, the weights are considered to be a neural
network's established connections between its neurons. Thus, since the weights are
internal to the neural network and influence its outputs, we can consider them as
neural network knowledge, provided that changing the weights will change the
neural network's capabilities and therefore actions.

[6]
 Chapter 1

An important parameter – bias

The artificial neuron can have an independent component that adds an extra signal
to the activation function. This component is called bias.

Just like the inputs, biases also have an associated weight. This feature helps in the
neural network knowledge representation as a more purely nonlinear system.

The parts forming the whole – layers

Natural neurons are organized in layers, each one providing a specific level of
processing; for example, the input layer receives direct stimuli from the outside
world, and the output layers fire actions that will have a direct influence on the
outside world. Between these layers, there are a number of hidden layers, in the
sense that they do not interact directly with the outside world. In the artificial neural
networks, all neurons in a layer share the same inputs and activation function, as
shown in the following figure:

Neural networks can be composed of several linked layers, forming the so-called
multilayer networks. The neural layers can be basically divided into three classes:

• Input layer
• Hidden layer
• Output layer

In practice, an additional neural layer adds another level of abstraction of the

outside stimuli, thereby enhancing the neural network's capacity to represent more
complex knowledge.

[7]
Getting Started with Neural Networks

Every neural network has at least an input/output layer

irrespective of the number of layers. In the case of a
multilayer network, the layers between the input and the
output are called hidden.

Learning about neural network

architectures
Basically, a neural network can have different layouts, depending on how the
neurons or neuron layers are connected to each other. Every neural network
architecture is designed for a specific end. Neural networks can be applied to a
number of problems, and depending on the nature of the problem, the neural
network should be designed in order to address this problem more efficiently.

Basically, there are two modalities of architectures for neural networks:

• Neuron connections
°° Monolayer networks
°° Multilayer networks

• Signal flow
°° Feedforward networks
°° Feedback networks

Monolayer networks
In this architecture, all neurons are laid out in the same level, forming one single
layer, as shown in the following figure:

[8]
Random documents with unrelated
content Scribd suggests to you:
96 The recital of the first chapter of the Koran.
Printed by The Superintendent, Government Press, Madras.
 Colophon
Availability

This eBook is for the use of anyone anywhere at no cost and with
almost no restrictions whatsoever. You may copy it, give it away or
re-use it under the terms of the Project Gutenberg License included
with this eBook or online at www.gutenberg.org .

This eBook is produced by the Online Distributed Proofreading Team

at www.pgdp.net .

Volume Contents First Article

I A and B Abhishēka
II C to J Canji
III K Kabbēra
VI K to M Kōri
V M to P Marakkāyar
VI P to S Palli
VII T to Z Tābēlu

Scans of this book are available from the Internet Archive (copy 1 ,
2 ).

Project Gutenberg catalog page: 42994 .

Related Library of Congress catalog page: 10014128 .

Related Open Library catalog page (for source): OL7024564M .

Related Open Library catalog page (for work): OL1106958W .

Related WorldCat catalog page: 1967849 .

Encoding

Revision History

2011-11-12 Started.

External References

This Project Gutenberg eBook contains external references. These

links may not work for you.

Corrections

The following corrections have been applied to the text:

Page Source Correction

31 8 12.8
32 Kotippatans Kotippattans
32 Kotipattans Kotippattans
107 Marathi Marāthi
126 marumakkattāyam marumakkathāyam
127 Dikenson Dickenson
176 puja pūja
244 gotra gōtra
250, 250,
250, 250 tirtha tīrtha
286 metempscychosis metempsychosis
308 Madigas Mādigas
319 Madiga Mādiga
353 negociations negotiations
389 [Not in source] believed
393 chosing choosing
393, 491,
492, 492,
499 marumakkatāyam marumakkathāyam
453 Marumakkathayam Marumakkathāyam
457 Kodungallur Kodungallūr
468 [Not in source] .
474 eat ate
497 Kodungalur Kodungallūr
*** END OF THE PROJECT GUTENBERG EBOOK CASTES AND
TRIBES OF SOUTHERN INDIA. VOL. 4 OF 7 ***

Updated editions will replace the previous one—the old editions

will be renamed.

Creating the works from print editions not protected by U.S.

copyright law means that no one owns a United States
copyright in these works, so the Foundation (and you!) can copy
and distribute it in the United States without permission and
without paying copyright royalties. Special rules, set forth in the
General Terms of Use part of this license, apply to copying and
distributing Project Gutenberg™ electronic works to protect the
PROJECT GUTENBERG™ concept and trademark. Project
Gutenberg is a registered trademark, and may not be used if
you charge for an eBook, except by following the terms of the
trademark license, including paying royalties for use of the
Project Gutenberg trademark. If you do not charge anything for
copies of this eBook, complying with the trademark license is
very easy. You may use this eBook for nearly any purpose such
as creation of derivative works, reports, performances and
research. Project Gutenberg eBooks may be modified and
printed and given away—you may do practically ANYTHING in
the United States with eBooks not protected by U.S. copyright
law. Redistribution is subject to the trademark license, especially
commercial redistribution.

START: FULL LICENSE

THE FULL PROJECT GUTENBERG LICENSE
 PLEASE READ THIS BEFORE YOU DISTRIBUTE OR USE THIS WORK

To protect the Project Gutenberg™ mission of promoting the

free distribution of electronic works, by using or distributing this
work (or any other work associated in any way with the phrase
“Project Gutenberg”), you agree to comply with all the terms of
the Full Project Gutenberg™ License available with this file or
online at www.gutenberg.org/license.

Section 1. General Terms of Use and

Redistributing Project Gutenberg™
electronic works
1.A. By reading or using any part of this Project Gutenberg™
electronic work, you indicate that you have read, understand,
agree to and accept all the terms of this license and intellectual
property (trademark/copyright) agreement. If you do not agree
to abide by all the terms of this agreement, you must cease
using and return or destroy all copies of Project Gutenberg™
electronic works in your possession. If you paid a fee for
obtaining a copy of or access to a Project Gutenberg™
electronic work and you do not agree to be bound by the terms
of this agreement, you may obtain a refund from the person or
entity to whom you paid the fee as set forth in paragraph 1.E.8.

1.B. “Project Gutenberg” is a registered trademark. It may only

be used on or associated in any way with an electronic work by
people who agree to be bound by the terms of this agreement.
There are a few things that you can do with most Project
Gutenberg™ electronic works even without complying with the
full terms of this agreement. See paragraph 1.C below. There
are a lot of things you can do with Project Gutenberg™
electronic works if you follow the terms of this agreement and
help preserve free future access to Project Gutenberg™
electronic works. See paragraph 1.E below.
1.C. The Project Gutenberg Literary Archive Foundation (“the
Foundation” or PGLAF), owns a compilation copyright in the
collection of Project Gutenberg™ electronic works. Nearly all the
individual works in the collection are in the public domain in the
United States. If an individual work is unprotected by copyright
law in the United States and you are located in the United
States, we do not claim a right to prevent you from copying,
distributing, performing, displaying or creating derivative works
based on the work as long as all references to Project
Gutenberg are removed. Of course, we hope that you will
support the Project Gutenberg™ mission of promoting free
access to electronic works by freely sharing Project Gutenberg™
works in compliance with the terms of this agreement for
keeping the Project Gutenberg™ name associated with the
work. You can easily comply with the terms of this agreement
by keeping this work in the same format with its attached full
Project Gutenberg™ License when you share it without charge
with others.

1.D. The copyright laws of the place where you are located also
govern what you can do with this work. Copyright laws in most
countries are in a constant state of change. If you are outside
the United States, check the laws of your country in addition to
the terms of this agreement before downloading, copying,
displaying, performing, distributing or creating derivative works
based on this work or any other Project Gutenberg™ work. The
Foundation makes no representations concerning the copyright
status of any work in any country other than the United States.

1.E. Unless you have removed all references to Project

Gutenberg:

1.E.1. The following sentence, with active links to, or other

immediate access to, the full Project Gutenberg™ License must
appear prominently whenever any copy of a Project
Gutenberg™ work (any work on which the phrase “Project
Gutenberg” appears, or with which the phrase “Project
Gutenberg” is associated) is accessed, displayed, performed,
viewed, copied or distributed:

This eBook is for the use of anyone anywhere in the United

States and most other parts of the world at no cost and
with almost no restrictions whatsoever. You may copy it,
give it away or re-use it under the terms of the Project
Gutenberg License included with this eBook or online at
www.gutenberg.org. If you are not located in the United
States, you will have to check the laws of the country
where you are located before using this eBook.

1.E.2. If an individual Project Gutenberg™ electronic work is

derived from texts not protected by U.S. copyright law (does not
contain a notice indicating that it is posted with permission of
the copyright holder), the work can be copied and distributed to
anyone in the United States without paying any fees or charges.
If you are redistributing or providing access to a work with the
phrase “Project Gutenberg” associated with or appearing on the
work, you must comply either with the requirements of
paragraphs 1.E.1 through 1.E.7 or obtain permission for the use
of the work and the Project Gutenberg™ trademark as set forth
in paragraphs 1.E.8 or 1.E.9.

1.E.3. If an individual Project Gutenberg™ electronic work is

posted with the permission of the copyright holder, your use and
distribution must comply with both paragraphs 1.E.1 through
1.E.7 and any additional terms imposed by the copyright holder.
Additional terms will be linked to the Project Gutenberg™
License for all works posted with the permission of the copyright
holder found at the beginning of this work.

1.E.4. Do not unlink or detach or remove the full Project

Gutenberg™ License terms from this work, or any files
 containing a part of this work or any other work associated with
Project Gutenberg™.

1.E.5. Do not copy, display, perform, distribute or redistribute

this electronic work, or any part of this electronic work, without
prominently displaying the sentence set forth in paragraph 1.E.1
with active links or immediate access to the full terms of the
Project Gutenberg™ License.

1.E.6. You may convert to and distribute this work in any binary,
compressed, marked up, nonproprietary or proprietary form,
including any word processing or hypertext form. However, if
you provide access to or distribute copies of a Project
Gutenberg™ work in a format other than “Plain Vanilla ASCII” or
other format used in the official version posted on the official
Project Gutenberg™ website (www.gutenberg.org), you must,
at no additional cost, fee or expense to the user, provide a copy,
a means of exporting a copy, or a means of obtaining a copy
upon request, of the work in its original “Plain Vanilla ASCII” or
other form. Any alternate format must include the full Project
Gutenberg™ License as specified in paragraph 1.E.1.

1.E.7. Do not charge a fee for access to, viewing, displaying,

performing, copying or distributing any Project Gutenberg™
works unless you comply with paragraph 1.E.8 or 1.E.9.

1.E.8. You may charge a reasonable fee for copies of or

providing access to or distributing Project Gutenberg™
electronic works provided that:

• You pay a royalty fee of 20% of the gross profits you derive
from the use of Project Gutenberg™ works calculated using the
method you already use to calculate your applicable taxes. The
fee is owed to the owner of the Project Gutenberg™ trademark,
but he has agreed to donate royalties under this paragraph to
the Project Gutenberg Literary Archive Foundation. Royalty
 payments must be paid within 60 days following each date on
which you prepare (or are legally required to prepare) your
periodic tax returns. Royalty payments should be clearly marked
as such and sent to the Project Gutenberg Literary Archive
Foundation at the address specified in Section 4, “Information
about donations to the Project Gutenberg Literary Archive
Foundation.”

• You provide a full refund of any money paid by a user who

notifies you in writing (or by e-mail) within 30 days of receipt
that s/he does not agree to the terms of the full Project
Gutenberg™ License. You must require such a user to return or
destroy all copies of the works possessed in a physical medium
and discontinue all use of and all access to other copies of
Project Gutenberg™ works.

• You provide, in accordance with paragraph 1.F.3, a full refund of

any money paid for a work or a replacement copy, if a defect in
the electronic work is discovered and reported to you within 90
days of receipt of the work.

• You comply with all other terms of this agreement for free
distribution of Project Gutenberg™ works.

1.E.9. If you wish to charge a fee or distribute a Project

Gutenberg™ electronic work or group of works on different
terms than are set forth in this agreement, you must obtain
permission in writing from the Project Gutenberg Literary
Archive Foundation, the manager of the Project Gutenberg™
trademark. Contact the Foundation as set forth in Section 3
below.

1.F.

1.F.1. Project Gutenberg volunteers and employees expend

considerable effort to identify, do copyright research on,
transcribe and proofread works not protected by U.S. copyright
law in creating the Project Gutenberg™ collection. Despite these
efforts, Project Gutenberg™ electronic works, and the medium
on which they may be stored, may contain “Defects,” such as,
but not limited to, incomplete, inaccurate or corrupt data,
transcription errors, a copyright or other intellectual property
infringement, a defective or damaged disk or other medium, a
computer virus, or computer codes that damage or cannot be
read by your equipment.

1.F.2. LIMITED WARRANTY, DISCLAIMER OF DAMAGES - Except

for the “Right of Replacement or Refund” described in
paragraph 1.F.3, the Project Gutenberg Literary Archive
Foundation, the owner of the Project Gutenberg™ trademark,
and any other party distributing a Project Gutenberg™ electronic
work under this agreement, disclaim all liability to you for
damages, costs and expenses, including legal fees. YOU AGREE
THAT YOU HAVE NO REMEDIES FOR NEGLIGENCE, STRICT
LIABILITY, BREACH OF WARRANTY OR BREACH OF CONTRACT
EXCEPT THOSE PROVIDED IN PARAGRAPH 1.F.3. YOU AGREE
THAT THE FOUNDATION, THE TRADEMARK OWNER, AND ANY
DISTRIBUTOR UNDER THIS AGREEMENT WILL NOT BE LIABLE
TO YOU FOR ACTUAL, DIRECT, INDIRECT, CONSEQUENTIAL,
PUNITIVE OR INCIDENTAL DAMAGES EVEN IF YOU GIVE
NOTICE OF THE POSSIBILITY OF SUCH DAMAGE.

1.F.3. LIMITED RIGHT OF REPLACEMENT OR REFUND - If you

discover a defect in this electronic work within 90 days of
receiving it, you can receive a refund of the money (if any) you
paid for it by sending a written explanation to the person you
received the work from. If you received the work on a physical
medium, you must return the medium with your written
explanation. The person or entity that provided you with the
defective work may elect to provide a replacement copy in lieu
of a refund. If you received the work electronically, the person
or entity providing it to you may choose to give you a second
opportunity to receive the work electronically in lieu of a refund.
If the second copy is also defective, you may demand a refund
in writing without further opportunities to fix the problem.

1.F.4. Except for the limited right of replacement or refund set

forth in paragraph 1.F.3, this work is provided to you ‘AS-IS’,
WITH NO OTHER WARRANTIES OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO WARRANTIES OF
MERCHANTABILITY OR FITNESS FOR ANY PURPOSE.

1.F.5. Some states do not allow disclaimers of certain implied

warranties or the exclusion or limitation of certain types of
damages. If any disclaimer or limitation set forth in this
agreement violates the law of the state applicable to this
agreement, the agreement shall be interpreted to make the
maximum disclaimer or limitation permitted by the applicable
state law. The invalidity or unenforceability of any provision of
this agreement shall not void the remaining provisions.

1.F.6. INDEMNITY - You agree to indemnify and hold the

Foundation, the trademark owner, any agent or employee of the
Foundation, anyone providing copies of Project Gutenberg™
electronic works in accordance with this agreement, and any
volunteers associated with the production, promotion and
distribution of Project Gutenberg™ electronic works, harmless
from all liability, costs and expenses, including legal fees, that
arise directly or indirectly from any of the following which you
do or cause to occur: (a) distribution of this or any Project
Gutenberg™ work, (b) alteration, modification, or additions or
deletions to any Project Gutenberg™ work, and (c) any Defect
you cause.

Section 2. Information about the Mission

of Project Gutenberg™
Project Gutenberg™ is synonymous with the free distribution of
electronic works in formats readable by the widest variety of
computers including obsolete, old, middle-aged and new
computers. It exists because of the efforts of hundreds of
volunteers and donations from people in all walks of life.

Volunteers and financial support to provide volunteers with the

assistance they need are critical to reaching Project
Gutenberg™’s goals and ensuring that the Project Gutenberg™
collection will remain freely available for generations to come. In
2001, the Project Gutenberg Literary Archive Foundation was
created to provide a secure and permanent future for Project
Gutenberg™ and future generations. To learn more about the
Project Gutenberg Literary Archive Foundation and how your
efforts and donations can help, see Sections 3 and 4 and the
Foundation information page at www.gutenberg.org.

Section 3. Information about the Project

Gutenberg Literary Archive Foundation
The Project Gutenberg Literary Archive Foundation is a non-
profit 501(c)(3) educational corporation organized under the
laws of the state of Mississippi and granted tax exempt status
by the Internal Revenue Service. The Foundation’s EIN or
federal tax identification number is 64-6221541. Contributions
to the Project Gutenberg Literary Archive Foundation are tax
deductible to the full extent permitted by U.S. federal laws and
your state’s laws.

The Foundation’s business office is located at 809 North 1500

West, Salt Lake City, UT 84116, (801) 596-1887. Email contact
links and up to date contact information can be found at the
Foundation’s website and official page at
www.gutenberg.org/contact
 Section 4. Information about Donations to
the Project Gutenberg Literary Archive
Foundation
Project Gutenberg™ depends upon and cannot survive without
widespread public support and donations to carry out its mission
of increasing the number of public domain and licensed works
that can be freely distributed in machine-readable form
accessible by the widest array of equipment including outdated
equipment. Many small donations ($1 to $5,000) are particularly
important to maintaining tax exempt status with the IRS.

The Foundation is committed to complying with the laws

regulating charities and charitable donations in all 50 states of
the United States. Compliance requirements are not uniform
and it takes a considerable effort, much paperwork and many
fees to meet and keep up with these requirements. We do not
solicit donations in locations where we have not received written
confirmation of compliance. To SEND DONATIONS or determine
the status of compliance for any particular state visit
www.gutenberg.org/donate.

While we cannot and do not solicit contributions from states

where we have not met the solicitation requirements, we know
of no prohibition against accepting unsolicited donations from
donors in such states who approach us with offers to donate.

International donations are gratefully accepted, but we cannot

make any statements concerning tax treatment of donations
received from outside the United States. U.S. laws alone swamp
our small staff.

Please check the Project Gutenberg web pages for current

donation methods and addresses. Donations are accepted in a
number of other ways including checks, online payments and
credit card donations. To donate, please visit:
www.gutenberg.org/donate.

Section 5. General Information About

Project Gutenberg™ electronic works
Professor Michael S. Hart was the originator of the Project
Gutenberg™ concept of a library of electronic works that could
be freely shared with anyone. For forty years, he produced and
distributed Project Gutenberg™ eBooks with only a loose
network of volunteer support.

Project Gutenberg™ eBooks are often created from several

printed editions, all of which are confirmed as not protected by
copyright in the U.S. unless a copyright notice is included. Thus,
we do not necessarily keep eBooks in compliance with any
particular paper edition.

Most people start at our website which has the main PG search
facility: www.gutenberg.org.

This website includes information about Project Gutenberg™,

including how to make donations to the Project Gutenberg
Literary Archive Foundation, how to help produce our new
eBooks, and how to subscribe to our email newsletter to hear
about new eBooks.
Welcome to our website – the ideal destination for book lovers and
knowledge seekers. With a mission to inspire endlessly, we offer a
vast collection of books, ranging from classic literary works to
specialized publications, self-development books, and children's
literature. Each book is a new journey of discovery, expanding
knowledge and enriching the soul of the reade

Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.

Let us accompany you on the journey of exploring knowledge and

personal growth!

textbookfull.com