Experience Seamless Full Ebook Downloads for Every Genre at textbookfull.

com

Data Analysis from Scratch with Python Peters

Morgan

https://textbookfull.com/product/data-analysis-from-scratch-
with-python-peters-morgan/

OR CLICK BUTTON

DOWNLOAD NOW

Explore and download more ebook at https://textbookfull.com

 Recommended digital products (PDF, EPUB, MOBI) that
you can download immediately if you are interested.

Data Science from Scratch First Principles with Python 2nd

Edition Joel Grus

https://textbookfull.com/product/data-science-from-scratch-first-
principles-with-python-2nd-edition-joel-grus/

textboxfull.com

Data Science from Scratch First Principles with Python 2nd

Edition Grus Joel

https://textbookfull.com/product/data-science-from-scratch-first-
principles-with-python-2nd-edition-grus-joel/

textboxfull.com

Data Analysis with Python and PySpark (MEAP V07) Jonathan

Rioux

https://textbookfull.com/product/data-analysis-with-python-and-
pyspark-meap-v07-jonathan-rioux/

textboxfull.com

A Python Data Analyst’s Toolkit: Learn Python and Python-

based Libraries with Applications in Data Analysis and
Statistics Gayathri Rajagopalan
https://textbookfull.com/product/a-python-data-analysts-toolkit-learn-
python-and-python-based-libraries-with-applications-in-data-analysis-
and-statistics-gayathri-rajagopalan/
textboxfull.com
Python for Data Analysis Data Wrangling with Pandas NumPy
and IPython Wes Mckinney

https://textbookfull.com/product/python-for-data-analysis-data-
wrangling-with-pandas-numpy-and-ipython-wes-mckinney/

textboxfull.com

Python for Data Analysis Data Wrangling with pandas NumPy

and Jupyter 3rd Edition Wes Mckinney

https://textbookfull.com/product/python-for-data-analysis-data-
wrangling-with-pandas-numpy-and-jupyter-3rd-edition-wes-mckinney/

textboxfull.com

Learning Data Mining with Python Layton

https://textbookfull.com/product/learning-data-mining-with-python-
layton/

textboxfull.com

Applied Text Analysis with Python Enabling Language Aware

Data Products with Machine Learning 1st Edition Benjamin
Bengfort
https://textbookfull.com/product/applied-text-analysis-with-python-
enabling-language-aware-data-products-with-machine-learning-1st-
edition-benjamin-bengfort/
textboxfull.com

Web Scraping with Python: Data Extraction from the Modern

Web 3rd Edition Mitchell

https://textbookfull.com/product/web-scraping-with-python-data-
extraction-from-the-modern-web-3rd-edition-mitchell/

textboxfull.com
D ATA A N A LY S I S F R O M S C R AT C H W I T H P Y T H O N
Step By Step Guide

Peters Morgan
 How to contact us
If you find any damage, editing issues or any other issues in this book contain
please immediately notify our customer service by email at:
contact@aiscicences.com

Our goal is to provide high-quality books for your technical learning in

computer science subjects.
Thank you so much for buying this book.
 Preface
“Humanity is on the verge of digital slavery at the hands of AI and biometric technologies. One way to
prevent that is to develop inbuilt modules of deep feelings of love and compassion in the learning
algorithms.”
― Amit Ray, Compassionate Artificial Superintelligence AI 5.0 - AI with Blockchain, BMI, Drone, IOT,
and Biometric Technologies
If you are looking for a complete guide to the Python language and its library
that will help you to become an effective data analyst, this book is for you.
This book contains the Python programming you need for Data Analysis.
Why the AI Sciences Books are different?
The AI Sciences Books explore every aspect of Artificial Intelligence and Data
Science using computer Science programming language such as Python and R.
Our books may be the best one for beginners; it's a step-by-step guide for any
person who wants to start learning Artificial Intelligence and Data Science from
scratch. It will help you in preparing a solid foundation and learn any other high-
level courses will be easy to you.
Step By Step Guide and Visual Illustrations and Examples

The Book give complete instructions for manipulating, processing, cleaning,

modeling and crunching datasets in Python. This is a hands-on guide with
practical case studies of data analysis problems effectively. You will learn
pandas, NumPy, IPython, and Jupiter in the Process.
Who Should Read This?

This book is a practical introduction to data science tools in Python. It is ideal

for analyst’s beginners to Python and for Python programmers new to data
science and computer science. Instead of tough math formulas, this book
contains several graphs and images.
© Copyright 2016 by AI Sciences LLC
All rights reserved.
First Printing, 2016
Edited by Davies Company
Ebook Converted and Cover by Pixels Studio Publised by AI Sciences LLC

ISBN-13: 978-1721942817
ISBN-10: 1721942815

The contents of this book may not be reproduced, duplicated or transmitted without the direct written
permission of the author.

Under no circumstances will any legal responsibility or blame be held against the publisher for any
reparation, damages, or monetary loss due to the information herein, either directly or indirectly.
Legal Notice:

You cannot amend, distribute, sell, use, quote or paraphrase any part or the content within this book without
the consent of the author.

Disclaimer Notice:
Please note the information contained within this document is for educational and entertainment purposes
only. No warranties of any kind are expressed or implied. Readers acknowledge that the author is not
engaging in the rendering of legal, financial, medical or professional advice. Please consult a licensed
professional before attempting any techniques outlined in this book.

By reading this document, the reader agrees that under no circumstances is the author responsible for any
losses, direct or indirect, which are incurred as a result of the use of information contained within this
document, including, but not limited to, errors, omissions, or inaccuracies.

From AI Sciences Publisher

 To my wife Melania
and my children Tanner and Daniel
without whom this book would have
been completed.
 Author Biography
Peters Morgan is a long-time user and developer of the Python. He is one of the
core developers of some data science libraries in Python. Currently, Peter works
as Machine Learning Scientist at Google.
 Table of Contents
Preface
Why the AI Sciences Books are different?
Step By Step Guide and Visual Illustrations and Examples
Who Should Read This?

From AI Sciences Publisher

Author Biography
Table of Contents
Introduction
2. Why Choose Python for Data Science & Machine Learning
Python vs R
Widespread Use of Python in Data Analysis
Clarity
3. Prerequisites & Reminders
Python & Programming Knowledge
Installation & Setup
Is Mathematical Expertise Necessary?
4. Python Quick Review
Tips for Faster Learning
5. Overview & Objectives
Data Analysis vs Data Science vs Machine Learning
Possibilities
Limitations of Data Analysis & Machine Learning
Accuracy & Performance
6. A Quick Example
Iris Dataset
Potential & Implications
7. Getting & Processing Data
CSV Files
Feature Selection
Online Data Sources
Internal Data Source
8. Data Visualization
Goal of Visualization
Importing & Using Matplotlib
9. Supervised & Unsupervised Learning
What is Supervised Learning?
What is Unsupervised Learning?
How to Approach a Problem
10. Regression
Simple Linear Regression
Multiple Linear Regression
Decision Tree
Random Forest
11. Classification
Logistic Regression
K-Nearest Neighbors
Decision Tree Classification
Random Forest Classification
12. Clustering
Goals & Uses of Clustering
K-Means Clustering
Anomaly Detection
13. Association Rule Learning
Explanation
Apriori
14. Reinforcement Learning
What is Reinforcement Learning?
Comparison with Supervised & Unsupervised Learning
Applying Reinforcement Learning
15. Artificial Neural Networks
An Idea of How the Brain Works
Potential & Constraints
Here’s an Example
16. Natural Language Processing
Analyzing Words & Sentiments
Using NLTK
Thank you !
Sources & References
Software, libraries, & programming language
Datasets
Online books, tutorials, & other references

Thank you !
Introduction
Why read on? First, you’ll learn how to use Python in data analysis (which is a
bit cooler and a bit more advanced than using Microsoft Excel). Second, you’ll
also learn how to gain the mindset of a real data analyst (computational
thinking).
More importantly, you’ll learn how Python and machine learning applies to real
world problems (business, science, market research, technology, manufacturing,
retail, financial). We’ll provide several examples on how modern methods of
data analysis fit in with approaching and solving modern problems.
This is important because the massive influx of data provides us with more
opportunities to gain insights and make an impact in almost any field. This
recent phenomenon also provides new challenges that require new technologies
and approaches. In addition, this also requires new skills and mindsets to
successfully navigate through the challenges and successfully tap the fullest
potential of the opportunities being presented to us.
For now, forget about getting the “sexiest job of the 21st century” (data scientist,
machine learning engineer, etc.). Forget about the fears about artificial
intelligence eradicating jobs and the entire human race. This is all about learning
(in the truest sense of the word) and solving real world problems.
We are here to create solutions and take advantage of new technologies to make
better decisions and hopefully make our lives easier. And this starts at building a
strong foundation so we can better face the challenges and master advanced
concepts.
2. Why Choose Python for Data Science & Machine Learning
Python is said to be a simple, clear and intuitive programming language. That’s
why many engineers and scientists choose Python for many scientific and
numeric applications. Perhaps they prefer getting into the core task quickly (e.g.
finding out the effect or correlation of a variable with an output) instead of
spending hundreds of hours learning the nuances of a “complex” programming
language.
This allows scientists, engineers, researchers and analysts to get into the project
more quickly, thereby gaining valuable insights in the least amount of time and
resources. It doesn’t mean though that Python is perfect and the ideal
programming language on where to do data analysis and machine learning.
Other languages such as R may have advantages and features Python has not.
But still, Python is a good starting point and you may get a better understanding
of data analysis if you use it for your study and future projects.
Python vs R
You might have already encountered this in Stack Overflow, Reddit, Quora, and
other forums and websites. You might have also searched for other programming
languages because after all, learning Python or R (or any other programming
language) requires several weeks and months. It’s a huge time investment and
you don’t want to make a mistake.
To get this out of the way, just start with Python because the general skills and
concepts are easily transferable to other languages. Well, in some cases you
might have to adopt an entirely new way of thinking. But in general, knowing
how to use Python in data analysis will bring you a long way towards solving
many interesting problems.
Many say that R is specifically designed for statisticians (especially when it
comes to easy and strong data visualization capabilities). It’s also relatively easy
to learn especially if you’ll be using it mainly for data analysis. On the other
hand, Python is somewhat flexible because it goes beyond data analysis. Many
data scientists and machine learning practitioners may have chosen Python
because the code they wrote can be integrated into a live and dynamic web
application.
Although it’s all debatable, Python is still a popular choice especially among
beginners or anyone who wants to get their feet wet fast with data analysis and
machine learning. It’s relatively easy to learn and you can dive into full time
programming later on if you decide this suits you more.
Widespread Use of Python in Data Analysis
There are now many packages and tools that make the use of Python in data
analysis and machine learning much easier. TensorFlow (from Google), Theano,
scikit-learn, numpy, and pandas are just some of the things that make data
science faster and easier.
Also, university graduates can quickly get into data science because many
universities now teach introductory computer science using Python as the main
programming language. The shift from computer programming and software
development can occur quickly because many people already have the right
foundations to start learning and applying programming to real world data
challenges.
Another reason for Python’s widespread use is there are countless resources that
will tell you how to do almost anything. If you have any question, it’s very likely
that someone else has already asked that and another that solved it for you
(Google and Stack Overflow are your friends). This makes Python even more
popular because of the availability of resources online.
Clarity
Due to the ease of learning and using Python (partly due to the clarity of its
syntax), professionals are able to focus on the more important aspects of their
projects and problems. For example, they could just use numpy, scikit-learn, and
TensorFlow to quickly gain insights instead of building everything from scratch.
This provides another level of clarity because professionals can focus more on
the nature of the problem and its implications. They could also come up with
more efficient ways of dealing with the problem instead of getting buried with
the ton of info a certain programming language presents.
The focus should always be on the problem and the opportunities it might
introduce. It only takes one breakthrough to change our entire way of thinking
about a certain challenge and Python might be able to help accomplish that
because of its clarity and ease.
3. Prerequisites & Reminders
Python & Programming Knowledge
By now you should understand the Python syntax including things about
variables, comparison operators, Boolean operators, functions, loops, and lists.
You don’t have to be an expert but it really helps to have the essential knowledge
so the rest becomes smoother.
You don’t have to make it complicated because programming is only about
telling the computer what needs to be done. The computer should then be able to
understand and successfully execute your instructions. You might just need to
write few lines of code (or modify existing ones a bit) to suit your application.
Also, many of the things that you’ll do in Python for data analysis are already
routine or pre-built for you. In many cases you might just have to copy and
execute the code (with a few modifications). But don’t get lazy because
understanding Python and programming is still essential. This way, you can spot
and troubleshoot problems in case an error message appears. This will also give
you confidence because you know how something works.
Installation & Setup
If you want to follow along with our code and execution, you should have
Anaconda downloaded and installed in your computer. It’s free and available for
Windows, macOS, and Linux. To download and install, go to
https://www.anaconda.com/download/ and follow the succeeding instructions
from there.
The tool we’ll be mostly using is Jupyter Notebook (already comes with
Anaconda installation). It’s literally a notebook wherein you can type and
execute your code as well as add text and notes (which is why many online
instructors use it).
If you’ve successfully installed Anaconda, you should be able to launch
Anaconda Prompt and type jupyter notebook on the blinking underscore. This
will then launch Jupyter Notebook using your default browser. You can then
create a new notebook (or edit it later) and run the code for outputs and
visualizations (graphs, histograms, etc.).
These are convenient tools you can use to make studying and analyzing easier
and faster. This also makes it easier to know which went wrong and how to fix
them (there are easy to understand error messages in case you mess up).
Is Mathematical Expertise Necessary?
Data analysis often means working with numbers and extracting valuable
insights from them. But do you really have to be expert on numbers and
mathematics?
Successful data analysis using Python often requires having decent skills and
knowledge in math, programming, and the domain you’re working on. This
means you don’t have to be an expert in any of them (unless you’re planning to
present a paper at international scientific conferences).
Don’t let many “experts” fool you because many of them are fakes or just plain
inexperienced. What you need to know is what’s the next thing to do so you can
successfully finish your projects. You won’t be an expert in anything after you
read all the chapters here. But this is enough to give you a better understanding
about Python and data analysis.
Back to mathematical expertise. It’s very likely you’re already familiar with
mean, standard deviation, and other common terms in statistics. While going
deeper into data analysis you might encounter calculus and linear algebra. If you
have the time and interest to study them, you can always do anytime or later.
This may or may not give you an edge on the particular data analysis project
you’re working on.
Again, it’s about solving problems. The focus should be on how to take a
challenge and successfully overcome it. This applies to all fields especially in
business and science. Don’t let the hype or myths to distract you. Focus on the
core concepts and you’ll do fine.
4. Python Quick Review
Here’s a quick Python review you can use as reference. If you’re stuck or need
help with something, you can always use Google or Stack Overflow.
To have Python (and other data analysis tools and packages) in your computer,
download and install Anaconda.
Python Data Types are strings (“You are awesome.”), integers (-3, 0, 1), and
floats (3.0, 12.5, 7.77).
You can do mathematical operations in Python such as: 3 + 3
print(3+3) 7 -1

5*2

20 / 5

9 % 2 #modulo operation, returns the remainder of the division 2 ** 3 #exponentiation, 2 to the 3rd
power Assigning values to variables: myName = “Thor”

print(myName) #output is “Thor”

x=5

y=6

print(x + y) #result is 11

print(x*3) #result is 15

Working on strings and variables: myName = “Thor”

age = 25

hobby = “programming”

print('Hi, my name is ' + myname + ' and my age is ' + str(age) + '. Anyway, my hobby is ' + hobby +
'.') Result is Hi, my name is Thon and my age is 25. Anyway, my hobby is programming.

Comments # Everything after the hashtag in this line is a comment.

# This is to keep your sanity.

# Make it understandable to you, learners, and other programmers.

Comparison Operators >>>8 == 8

True
>>>8 > 4
True
>>>8 < 4
False
>>>8 != 4
True
>>>8 != 8
False
>>>8 >= 2
True
>>>8 <= 2
False
>>>’hello’ == ‘hello’
True
>>>’cat’ != ‘dog’
True

Boolean Operators (and, or, not) >>>8 > 3 and 8 > 4

True
>>>8 > 3 and 8 > 9
False
>>>8 > 9 and 8 > 10
False
>>>8 > 3 or 8 > 800
True
>>>’hello’ == ‘hello’ or ‘cat’ == ‘dog’
True

If, Elif, and Else Statements (for Flow Control) print(“What’s your email?”)
myEmail = input()
print(“Type in your password.”)
typedPassword = input()
if typedPassword == savedPassword:
print(“Congratulations! You’re now logged in.”)
else:
print(“Your password is incorrect. Please try again.”)

While loop inbox = 0

while inbox < 10:
print(“You have a message.”)
inbox = inbox + 1
Result is this: You have a message.
You have a message.
You have a message.
You have a message.
You have a message.
You have a message.
You have a message.
You have a message.
You have a message.
You have a message.

Loop doesn’t exit until you typed ‘Casanova’

name = ''
while name != 'Casanova':
print('Please type your name.')
name = input()
print('Congratulations!')

For loop for i in range(10):

print(i ** 2)
Here’s the output: 0
1
4
9
16
25
36
49
64
81
#Adding numbers from 0 to 100

total = 0
for num in range(101):
total = total + num
print(total)

When you run this, the sum will be 5050.

#Another example. Positive and negative reviews.

all_reviews = [5, 5, 4, 4, 5, 3, 2, 5, 3, 2, 5, 4, 3, 1, 1, 2, 3, 5, 5]
positive_reviews = []
for i in all_reviews:
if i > 3:
print('Pass')
positive_reviews.append(i)
else:
print('Fail')

print(positive_reviews)
print(len(positive_reviews))
ratio_positive = len(positive_reviews) / len(all_reviews)
print('Percentage of positive reviews: ')
print(ratio_positive * 100)

When you run this, you should see: Pass

Pass
Pass
Pass
Pass
Fail
Fail
Pass
Fail
Fail
Pass
Pass
Fail
Fail
Fail
Fail
Fail
Pass
Pass
[5, 5, 4, 4, 5, 5, 5, 4, 5, 5]
10
Percentage of positive reviews:
52.63157894736842
Functions def hello():
print('Hello world!')
hello()
Define the function, tell what it should do, and then use or call it later.
def add_numbers(a,b):
print(a + b)

add_numbers(5,10)
add_numbers(35,55)

#Check if a number is odd or even.

def even_check(num):
if num % 2 == 0:
print('Number is even.')
else:
print('Hmm, it is odd.')

even_check(50)
even_check(51)

Lists my_list = [‘eggs’, ‘ham’, ‘bacon’] #list with strings colours = [‘red’,
‘green’, ‘blue’]
cousin_ages = [33, 35, 42] #list with integers mixed_list = [3.14, ‘circle’, ‘eggs’, 500] #list with integers
and strings #Working with lists colours = [‘red’, ‘blue’, ‘green’]

colours[0] #indexing starts at 0, so it returns first item in the list which is ‘red’

colours[1] #returns second item, which is ‘green’

#Slicing the list my_list = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

print(my_list[0:2]) #returns [0, 1]
print(my_list[1:]) #returns [1, 2, 3, 4, 5, 6, 7, 8, 9]
print(my_list[3:6]) #returns [3, 4, 5]

#Length of list my_list = [0,1,2,3,4,5,6,7,8,9]

print(len(my_list)) #returns 10

#Assigning new values to list items colours = ['red', 'green', 'blue']

colours[0] = 'yellow'
print(colours) #result should be ['yellow', 'green', 'blue']

#Concatenation and appending colours = ['red', 'green', 'blue']

colours.append('pink')
print(colours)
The result will be:
['red', 'green', 'blue', 'pink']

fave_series = ['GOT', 'TWD', 'WW']

fave_movies = ['HP', 'LOTR', 'SW']
fave_all = fave_series + fave_movies
print(fave_all)

This prints ['GOT', 'TWD', 'WW', 'HP', 'LOTR', 'SW']

Those are just the basics. You might still need to refer to this whenever you’re
doing anything related to Python. You can also refer to Python 3 Documentation
for more extensive information. It’s recommended that you bookmark that for
future reference. For quick review, you can also refer to Learn python3 in Y
Minutes.
Tips for Faster Learning
If you want to learn faster, you just have to devote more hours each day in
learning Python. Take note that programming and learning how to think like a
programmer takes time.
There are also various cheat sheets online you can always use. Even experienced
programmers don’t know everything. Also, you actually don’t have to learn
everything if you’re just starting out. You can always go deeper anytime if
something interests you or you want to stand out in job applications or startup
funding.
5. Overview & Objectives
Let’s set some expectations here so you know where you’re going. This is also to
introduce about the limitations of Python, data analysis, data science, and
machine learning (and also the key differences). Let’s start.
Data Analysis vs Data Science vs Machine Learning
Data Analysis and Data Science are almost the same because they share the
same goal, which is to derive insights from data and use it for better decision
making.
Often, data analysis is associated with using Microsoft Excel and other tools for
summarizing data and finding patterns. On the other hand, data science is often
associated with using programming to deal with massive data sets. In fact, data
science became popular as a result of the generation of gigabytes of data coming
from online sources and activities (search engines, social media).
Being a data scientist sounds way cooler than being a data analyst. Although the
job functions might be similar and overlapping, it all deals with discovering
patterns and generating insights from data. It’s also about asking intelligent
questions about the nature of the data (e.g. Are data points form organic clusters?
Is there really a connection between age and cancer?).
What about machine learning? Often, the terms data science and machine
learning are used interchangeably. That’s because the latter is about “learning
from data.” When applying machine learning algorithms, the computer detects
patterns and uses “what it learned” on new data.
For instance, we want to know if a person will pay his debts. Luckily we have a
sizable dataset about different people who either paid his debt or not. We also
have collected other data (creating customer profiles) such as age, income range,
location, and occupation. When we apply the appropriate machine learning
algorithm, the computer will learn from the data. We can then input new data
(new info from a new applicant) and what the computer learned will be applied
to that new data.
We might then create a simple program that immediately evaluates whether a
person will pay his debts or not based on his information (age, income range,
location, and occupation). This is an example of using data to predict someone’s
likely behavior.
Possibilities
Learning from data opens a lot of possibilities especially in predictions and
optimizations. This has become a reality thanks to availability of massive
datasets and superior computer processing power. We can now process data in
gigabytes within a day using computers or cloud capabilities.
Although data science and machine learning algorithms are still far from perfect,
these are already useful in many applications such as image recognition, product
recommendations, search engine rankings, and medical diagnosis. And to this
moment, scientists and engineers around the globe continue to improve the
accuracy and performance of their tools, models, and analysis.
Limitations of Data Analysis & Machine Learning
You might have read from news and online articles that machine learning and
advanced data analysis can change the fabric of society (automation, loss of jobs,
universal basic income, artificial intelligence takeover).
In fact, the society is being changed right now. Behind the scenes machine
learning and continuous data analysis are at work especially in search engines,
social media, and e-commerce. Machine learning now makes it easier and faster
to do the following:
● Are there human faces in the picture?
● Will a user click an ad? (is it personalized and appealing to him/her?)
● How to create accurate captions on YouTube videos? (recognise speech
and translate into text)
● Will an engine or component fail? (preventive maintenance in
manufacturing)
● Is a transaction fraudulent?
● Is an email spam or not?
These are made possible by availability of massive datasets and great processing
power. However, advanced data analysis using Python (and machine learning) is
not magic. It’s not the solution to all problem. That’s because the accuracy and
performance of our tools and models heavily depend on the integrity of data and
our own skill and judgment.
Yes, computers and algorithms are great at providing answers. But it’s also about
asking the right questions. Those intelligent questions will come from us
humans. It also depends on us if we’ll use the answers being provided by our
computers.
Accuracy & Performance
The most common use of data analysis is in successful predictions (forecasting)
and optimization. Will the demand for our product increase in the next five
years? What are the optimal routes for deliveries that lead to the lowest
operational costs?
That’s why an accuracy improvement of even just 1% can translate into millions
of dollars of additional revenues. For instance, big stores can stock up certain
products in advance if the results of the analysis predicts an increasing demand.
Shipping and logistics can also better plan the routes and schedules for lower
fuel usage and faster deliveries.
Aside from improving accuracy, another priority is on ensuring reliable
performance. How can our analysis perform on new data sets? Should we
consider other factors when analyzing the data and making predictions? Our
work should always produce consistently accurate results. Otherwise, it’s not
scientific at all because the results are not reproducible. We might as well shoot
in the dark instead of making ourselves exhausted in sophisticated data analysis.
Apart from successful forecasting and optimization, proper data analysis can
also help us uncover opportunities. Later we can realize that what we did is also
applicable to other projects and fields. We can also detect outliers and interesting
patterns if we dig deep enough. For example, perhaps customers congregate in
clusters that are big enough for us to explore and tap into. Maybe there are
unusually higher concentrations of customers that fall into a certain income
range or spending level.
Those are just typical examples of the applications of proper data analysis. In the
next chapter, let’s discuss one of the most used examples in illustrating the
promising potential of data analysis and machine learning. We’ll also discuss its
implications and the opportunities it presents.
6. A Quick Example
Iris Dataset
Let’s quickly see how data analysis and machine learning work in real world
data sets. The goal here is to quickly illustrate the potential of Python and
machine learning on some interesting problems.
In this particular example, the goal is to predict the species of an Iris flower
based on the length and width of its sepals and petals. First, we have to create a
model based on a dataset with the flowers’ measurements and their
corresponding species. Based on our code, our computer will “learn from the
data” and extract patterns from it. It will then apply what it learned to a new
dataset. Let’s look at the code.
#importing the necessary libraries from sklearn.datasets import load_iris
from sklearn import tree
from sklearn.metrics import accuracy_score
import numpy as np

#loading the iris dataset

iris = load_iris()

x = iris.data #array of the data

y = iris.target #array of labels (i.e answers) of each data entry

#getting label names i.e the three flower species

y_names = iris.target_names

#taking random indices to split the dataset into train and test
test_ids = np.random.permutation(len(x))

#splitting data and labels into train and test

#keeping last 10 entries for testing, rest for training

x_train = x[test_ids[:-10]]
x_test = x[test_ids[-10:]]

y_train = y[test_ids[:-10]]
y_test = y[test_ids[-10:]]

#classifying using decision tree

clf = tree.DecisionTreeClassifier()

#training (fitting) the classifier with the training set

clf.fit(x_train, y_train)
#predictions on the test dataset
pred = clf.predict(x_test)

print(pred) #predicted labels i.e flower species

print(y_test) #actual labels
print((accuracy_score(pred, y_test)))*100 #prediction accuracy #Reference: http://docs.python-
guide.org/en/latest/scenarios/ml/

If we run the code, we’ll get something like this: [0 1 1 1 0 2 0 2 2 2]

[0 1 1 1 0 2 0 2 2 2]
100.0
The first line contains the predictions (0 is Iris setosa, 1 is Iris versicolor, 2 is Iris
virginica). The second line contains the actual flower species as indicated in the
dataset. Notice the prediction accuracy is 100%, which means we correctly
predicted each flower’s species.
These might all seem confusing at first. What you need to understand is that the
goal here is to create a model that predicts a flower’s species. To do that, we split
the data into training and test sets. We run the algorithm on the training set and
use it against the test set to know the accuracy. The result is we’re able to predict
the flower’s species on the test set based on what the computer learned from the
training set.
Potential & Implications
It’s a quick and simple example. But its potential and implications can be
enormous. With just a few modifications, you can apply the workflow to a wide
variety of tasks and problems.
For instance, we might be able to apply the same methodology on other flower
species, plants, and animals. We can also apply this in other Classification
problems (more on this later) such as determining if a cancer is benign or
malignant, if a person is a very likely customer, or if there’s a human face in the
photo.
The challenge here is to get enough quality data so our computer can properly
get “good training.” It’s a common methodology to first learn from the training
set and then apply the learning into the test set and possibly new data in the
future (this is the essence of machine learning).
It’s obvious now why many people are hyped about the true potential of data
analysis and machine learning. With enough data, we can create automated
Exploring the Variety of Random
Documents with Different Content
of them possess organs of stridulation, and the structure of their
spiracles is very peculiar, each one being more or less completely
surrounded by a chitinous plate. The spiracles usually form a system
entirely closed, except at the moment when the skin is shed and the
tracheal exuviae are detached. Meinert[79] considers these spiracles
to be organs of hearing. The life of the larvae is passed underground
or in the decaying wood on which the Insect feeds.

Fig. 85.—Antennae of Lamellicorns. 1, Neleus interruptus; 2, Lucanus

cervus ♂; 3, Phanaeus splendidulus ♀; 4, Phileurus didymus ♀; 5,
Polyphylla fullo ♂.

Most of the members of this series are remarkable on account of the

great concentration of the nerve-centres. This is extreme in
Rhizotrogus, where there are only two great ganglia, viz. the supra-
oesophageal and a great ganglion situate in the thorax, and
consisting of the conjoined infra-oesophageal, thoracic, and
abdominal ganglia. According to Brandt[80] there are several distinct
forms of concentration in the series; the Lucanidae only participate in
it to the extent that the perfect Insects exhibit fewer ganglia than the
larvae; the latter possess two cephalic, three thoracic, and eight
abdominal ganglia, while the perfect Insect has the abdominal
ganglia reduced in number to six, and they are placed partially in the
thorax. The diminution in number takes place in this case by the
amalgamation of the first two abdominal with the last thoracic
ganglia.
 Fig. 86—View of one side of meso- and metathorax of a Passalid larva
from Borneo showing the stridulating organs. a, b, Portions of the
metathorax; c, coxa of 2nd leg; d, striate or stridulating area
thereon; e, basal part of femur of middle leg; f, hairs with chitinous
process at base of each; g, the diminutive 3rd leg modified for
scratching the striated area. × 20.

Fam. 1. Passalidae.—Labrum large, mobile; mentum deeply cut out

in the middle for the accommodation of the ligula; the lamellae of the
antenna brought together by the curling up of the antenna. The elytra
entirely cover the dorsal surface of the abdomen. There are four or
five hundred species of this family known; they are usually shining-
black, unattractive beetles, of large size, and are abundant in the
decaying wood of tropical forests. They are quite absent from
Europe, and there is only one species in the United States of North
America. The larvae are very interesting, from the fact that they
appear to have only four legs. This arises from the posterior pair
being present only as very short processes, the function of which is
to scrape striated areas on the preceding pair of legs and so produce
sound. In the species figured (Fig. 86) this short leg is a paw-like
structure, bearing several hard digits; but in other species it is more
simple, and without the digits. The perfect Insect has no sound-
producing organs, and it is very remarkable therefore to find the
larvae provided with highly-developed stridulatory structures. No
auditory organ is known, unless the peculiar spiracles be such.
 Fig. 87—Head and prothorax of forms of the male of a stag-beetle;
Homocoderus mellyi (Africa). A, Large, B, intermediate, C, small
forms. (From a photograph by R. Oberthür.)

Fam. 2. Lucanidae (Stag-beetles).—Labrum indistinct, fixed;

mentum not excised; antennae not curled in repose, with but little
coadaptation of the terminal joints; the elytra entirely cover the dorsal
surface of the abdomen. The Stag-beetles are well known on
account of the extraordinary development of the mandibles in the
male sex, these organs being in some cases nearly as long as the
whole of the rest of the Insect, and armed with projections or teeth
that give the Insects a most formidable appearance. So far as we
have been able to discover, these structures are put to very little use,
and in many cases are not capable of being of service even as
weapons of offence. The males are usually very much larger than
the females, and are remarkable on account of the great variation in
the stature of different individuals of the same species; correlative
with these distinctions of individual size we find extreme differences
in the development of the head and mandibles. Moreover, the small
male specimens exhibit not merely reductions in the size of the
mandibles, but also show considerable differences in the form of
these parts, due, in some cases, apparently to the fact that only
when a certain length of the mandible is attained is there any
development of certain of the minor projections: in other cases it is
not possible to adopt this view, as the small mandibles bear as many
projections as the large forms do, or even more. In each species
these variations fall, in the majority of cases, into distinct states, so
that entomologists describe them as "forms," the largest
developments being called teleodont, the smallest priodont; the
terms mesodont and amphiodont being applied to intermediate
states. Leuthner, who has examined many specimens, states that in
Odontolabis sinensis, no intermediates between the teleodont and
mesodont forms occur, and as the two forms are very different they
are liable to be mistaken for distinct species.

There are at present between 500 and 600 species of stag-beetles

known; the Indo-Malayan and Austro-Malayan regions being richest
in them. Australia possesses many remarkable and aberrant forms.
In the Ceratognathini—a group well represented in New Zealand as
well as in Australia—the structure of the antennae is like that of the
Scarabaeidae, rather than of the Lucanidae. The most aberrant form
known is, however, our common Sinodendron cylindricum; this
departs in numerous features from other Lucanidae, and instead of
the mandibles of the male being more largely developed, there is a
horn on the head; it is very doubtful whether this Insect should be
allowed to remain in the family. Little is known of the habits and
development of Lucanidæ, except in the case of three or four
species that are common in Europe.

Fig. 88—Sinodendron cylindricum. A, Larva; B, pupa. New Forest.

The common stag-beetle, Lucanus cervus, is our largest British

beetle. The larva much resembles that of Melolontha vulgaris, but
attains a larger size, and the anal aperture is placed longitudinally
instead of transversely; it lives in decaying wood, or eats the roots of
trees without being injurious; its life in this state lasts about four
years; the pupal period is passed through rapidly, and the perfect
Insect may remain for months underground before it becomes active;
this occurs in June and July. This larva stridulates by scraping
certain hard tubercular ridges on the third pair of legs, over a
specially adapted rough area at the base of the second pair.
The Passalidae and Lucanidae are united by some authorities as a
group called Pectinicornia; the term Lamellicornia being then
confined to the Scarabaeidae. The Passalidae appear, however, to
be really more nearly allied to the Scarabaeidae than to the
Lucanidae.

Fam. 3. Scarabaeidae (Chafers).—The leaflets of the antennae are

well coadapted, and are susceptible of separation. The elytra usually
leave the pygidium uncovered. The number of visible ventral
segments is usually six, or at the sides seven, not five, as in
Lucanidae and Passalidae. This is one of the most important families
of Insects. About 13,000 species are already known; as some of
them are highly remarkable creatures on account of the males being
armed with horns, they are figured in many works on natural history.
There is great variety of form, and the following five sub-families may
be adopted, though authorities are by no means agreed as to the
classification of this extensive family, which, moreover, be it
remarked, is increasing by the discovery of about 300 new species
every year.

Abdominal spiracles placed in a line on the connecting membranes, and

entirely covered by the wing-cases (Laparosticti). .......... Sub-fam. 1.
Coprides.[81]
Abdominal spiracles placed almost in a line, but only the basal three on the
connecting membranes; the terminal one usually not covered by the
wing-cases. .......... Sub-fam. 2. Melolonthides.
Abdominal spiracles placed in two lines, the basal three on the connecting
membranes, the others on the ventral segments (Pleurosticti).
The claws of the tarsi unequal. .......... Sub-fam. 3. Rutelides.
The claws of the tarsi equal; the front coxae transverse, but little prominent
in the descending axis. .......... Sub-fam. 4. Dynastides.
The claws of the tarsi equal; the front coxae more prominent, shorter
transversely. .......... Sub-fam. 5. Cetoniides.
i. The Coprides form an immense group of about 5000 species; they
differ somewhat in habits from other Lamellicorns, inasmuch as most
of them live on dung, or decaying animal matter; the sub-family
connects with the Lucanidae, so far as superficial characters go, by
means of two of its groups, Trogini and Nicagini, the latter being very
near to the Ceratognathini in Lucanidae. So little is known as to the
morphology and development of these groups that it is not possible
to pronounce an opinion as to the validity of this apparent alliance.
Trox stridulates by rubbing two raised lines on the penultimate dorsal
segment across two striate ribs on the inner face of the elytra;
Geotrupes, on the other hand, produces an audible sound by
rubbing together a file on the posterior coxa and a fine ridge on the
contiguous ventral segment. The larva in this genus has a different
organ for stridulation from the imago; it is placed on the second and
third pairs of legs, the latter pair being much reduced in size.

The most interesting division of the Coprides is the group

Scarabaeini. No member of this group inhabits the British islands,
but in Southern Europe, and in still warmer lands, these Insects are
well known from the curious habit many of the species have of rolling
about balls of dung and earth. The long hind legs are chiefly used for
this purpose, and it is on the peculiar structure of these limbs that the
group has been established. Many of the stone Scarabaei found in
Egyptian tombs represent some kind of Scarabaeini, and it has been
said that the ancient Egyptians looked on these Insects as sacred
because of their movements. These must certainly appear very
strange to those who see them and are unacquainted with their
object. It is stated that the dwellers in the valley of the Nile thought
the actions of these Insects, when rolling their balls, were typical of
the planetary and lunar revolutions; and that the sudden appearance
of the beetles after a period of complete absence was emblematic of
a future life. Many accounts have been given of the habits of
members of this group, but according to Fabre all are more or less
erroneous; and he has described the habits and life-history of
Scarabaeus sacer (Fig. 89), as observed by him in Southern France.
These Insects act the part of scavengers by breaking up and burying
the droppings of cattle and other animals. The female Scarabaeus
detaches a portion of the dung and forms it into a ball, sometimes as
large as the fist; this it rolls along by means of its hind legs, by
pushing when necessary with its broad head, or by walking
backwards and dragging the ball with its front legs. The strength and
patience displayed by the creature in the execution of this task are
admirable. Frequently the owner of this small spherical property is
joined, so Fabre informs us, by a friend, who is usually of the same
sex and assists her in pushing along the ball till a suitable place is
reached. When this is attained, the owner commences to excavate a
chamber for the reception of the ball; sometimes the false friend
takes advantage of the opportunity thus offered and carries off the
ball for her own use. Should no disappointment of this sort occur, the
Scarabaeus accomplishes the burying of the ball in its subterranean
chamber, and accompanies it for the purpose of devouring it; the
feast is continued without intermission till the food is entirely
exhausted, when the Scarabaeus seeks a fresh store of provender
to be treated in a similar manner. According to M. Fabre's account
these events occur in the spring of the year, and when the hot
weather sets in the Scarabaeus passes through a period of
quiescence, emerging again in the autumn to recommence its
labours, which are now, however, directed immediately to the
continuance of the species; a larger subterranean chamber is
formed, and to this retreat the beetle carries dung till it has
accumulated a mass of the size of a moderate apple; this material is
carefully arranged, previous to the laying of the egg, in such a
manner that the grub to be hatched from the egg shall find the
softest and most nutritive portions close to it, while the coarser and
more innutritious parts are arranged so as to be reached by the grub
only after it has acquired some strength; lastly, a still more delicate
and nutritive paste is prepared by the mother beetle for a first meal
for the newly-hatched grub, by some of the food being submitted to a
partial digestion in her organs; finally, the egg is deposited in the
selected spot, and the chamber closed. Certain of the Coprides
exhibit, according to Fabre, some extremely exceptional features in
their life-histories. The mother, instead of dying after oviposition,
survives, and sees the growth of her young to the perfect state, and
then produces another generation. No similar case can be pointed
out in Insects, except in the Social kinds; but from these the
Coprides observed by Fabre differ profoundly, inasmuch as the
number of eggs produced by the mother is extremely small; Copris
hispanus, for instance, producing in each of its acts of oviposition
only one, two, or three eggs.

Fig. 89—Scarabaeus sacer. Portugal.

ii. The Melolonthides are probably almost as numerous as the

Coprides, some 4000 species being already known. The larvae are
believed to feed chiefly on roots. Melolontha vulgaris, the common
cockchafer, is very abundant in some parts of Europe, and owing to
this and to the great damage it causes, has attracted much attention.
The memoir by Straus-Durckheim[82] on its anatomy is one of the
classical works of Entomology. This Insect is so injurious in some
parts of France that money is paid by the local authorities for its
destruction. M. Reiset informs us that under this arrangement
867,173,000 perfect cockchafers, and 647,000,000 larvae were
destroyed in the Seine-inférieure in the four years from the middle of
1866 to 1870. Unlike the Coprides, the larval life in Melolonthides is
prolonged, and that of the imago is of brief duration. In Central
Europe the life-cycle of the individual in M. vulgaris occupies three
years, though in dry periods it may be extended to four years. In
Scandinavia the time occupied by the development appears to be
usually five years. The fertile female enters the ground and deposits
its eggs in two or three successive batches of about fifteen each.
The eggs swell as the development of the embryo progresses; the
larva emerges about five weeks after the eggs have been deposited,
and is of relatively large size. When young the larvae can straighten
themselves out and crawl, but when older they lose this power, and
when above ground rest helplessly on their sides. In the winter they
descend deeply into the earth to protect themselves from frost. The
pupa state lasts only a few days, but after the final transformation the
perfect Insect may remain motionless for as much as eight months
underground before commencing its active life in the air.[83] In the
perfect state the Insect is sometimes injurious from the large quantity
of foliage it destroys. Schiödte[84] considered that these larvae (and
those of numerous other Scarabaeidae) stridulate by rubbing certain
projections on the stipes of the maxilla against the under-surface of
the mandible. These surfaces appear, however, but little adapted for
the purpose of producing sound.

iii. The Rutelides number about 1500 species; there are many
Insects of brilliant metallic colours amongst them, but very little is
known as to their life-histories. The larvae are very much like those
of Melolonthides.

iv. The Dynastides are the smallest division in number of species,

there being scarcely 1000 known; but amongst them we find in the
genera Dynastes and Megasoma some of the largest of existing
Insects. The horns and projections on the heads and prothoraces of
some of the males of these Insects are truly extraordinary, and it
does not appear possible to explain their existence by any use they
are to their possessors. These structures are but little used for
fighting. Baron von Hügel informs the writer that in Java he has
observed large numbers of Xylotrupes gideon; he noticed that the
males sometimes carry the females by the aid of their horns; but this
must be an exceptional case, for the shape of these instruments, in
the majority of Dynastides, would not allow of their being put to this
use. The development of these horns varies greatly in most of the
species, but he did not find that the females exhibited any preference
for the highly armed males. The fact that the males are very much
larger than the females, and that the armature is usually confined to
them, suggests, however, that some sexual reason exists for these
remarkable projections. Many Dynastides possess organs of
stridulation, consisting of lines of sculpture placed so as to form one
or two bands on the middle of the propygidium, and brought into play
by being rubbed by the extremities of the wing-cases. This apparatus
is of a less perfect nature than the structures for the same purpose
found in numerous other beetles. We have no member of this sub-
family in Britain, and there are scarcely a dozen in all Europe.
Decaying vegetable matter is believed to be the nutriment of
Dynastides. The European Oryctes nasicornis is sometimes found in
numbers in spent tan. The growth and development of the individual
is believed to be but slow.

v. The Cetoniides are renowned for the beauty of their colours and
the elegance of their forms; hence they are a favourite group, and
about 1600 species have been catalogued. They are specially fond
of warm regions, but it is a peculiarity of the sub-family that a large
majority of the species are found in the Old World; South America is
inexplicably poor in these Insects, notwithstanding its extensive
forests. In this sub-family the mode of flight is peculiar; the elytra do
not extend down the sides of the body, so that, if they are elevated a
little, the wings can be protruded. This is the mode of flight adopted
by most Cetoniides, but the members of the group Trichiini fly in the
usual manner. In Britain we have only four kinds of Cetoniides; they
are called Rose-chafers. The larvae of C. floricola and some other
species live in ants' nests made of vegetable refuse, and it is said
that they eat the ants' progeny. Two North American species of
Euphoria have similar habits. The group Cremastochilini includes
numerous peculiar Insects that apparently have still closer relations
with ants. Most of them are very aberrant as well as rare forms, and
it has been several times observed in North America that species of
Cremastochilus not only live in the nests of the ants, but are forcibly
detained therein by the owners, who clearly derive some kind of
satisfaction from the companionship of the beetles. The species of
the genus Lomaptera stridulate in a peculiar manner, by rubbing the
edges of the hind femora over a striate area on the ventral
segments.
 Series II. Adephaga or Caraboidea.

All the tarsi five-jointed; antennae filiform, or nearly so; mouth-

parts highly developed, the outer lobe of the maxilla nearly
always divided into a two-jointed palpus; supports of the labial
palpi developed as joints of the palpi, and in some cases
approximate at their bases. Abdomen with the exposed
segments one more in number at the sides than along the
middle, the number being usually five along the middle, six at
each side.

This extensive series includes the tiger-beetles, ground-beetles, and

true water-beetles; it consists of six families, and forms a natural
assemblage. It is sometimes called Carnivora or Filicornia. The
exceptions to the characters we have mentioned are but few. The
supports of the labial palpi are frequently covered by the mentum,
and then the palpi appear three-jointed; but when the joint-like
palpiger is not covered these palps appear four-jointed. As a rule,
approximation of these supports is indicative of high development. In
some of the lower forms the trophi remain at a lower stage of
development than is usual. This is especially the case with the genus
Amphizoa, which forms of itself the family Amphizoidae. The
Bombardier-beetles make an exception as regards the abdominal
structure, for in some of them no less than eight segments are
visible, either along the middle line or at the sides of the venter. In
Hydroporides (one of the divisions of Dytiscidae) the front and
middle feet have each only four joints. Many naturalists unite the
Gyrinidae with the Adephaga, and a few also associate with them
the Paussidae and Rhysodidae; but we think it better at present to
exclude all these, though we believe that both Paussidae and
Rhysodidae will ultimately be assigned to the series. The larvae are
usually very active, and have a higher development of the legs than
is usual in this Order. Their tarsi possess two claws.

Fam. 4. Cicindelidae (Tiger-beetles).—Clypeus extending laterally

in front of the insertion of the antennae. Lower lip with the palpi
usually greatly developed, but with the ligula and paraglossae very
much reduced, often scarcely to be detected. Maxillae with the outer
lobe forming a two-jointed palp,[85] the inner lobe elongate, furnished
at the tip with a hook-like process, which is usually articulated by a
joint with the lobe itself. The tiger-beetles are very active Insects,
running with extreme speed, and sometimes flying in a similar
manner; they are all predaceous, and amongst the most voracious
and fierce of the carnivorous beetles, so that they well deserve their
name. Bates, speaking of the Amazonian Megacephala, says "their
powers of running exceed anything I have ever observed in this style
of Insect locomotion; they run in a serpentine course over the
smooth sand, and when closely pursued by the hand they are apt to
turn suddenly back and thus baffle the most practised hand and
eye." He further says that the species he observed (being of diverse
colours) agreed in colour with the general colours of the "locale they
inhabit." The larvae of Cicindelidae live in deep burrows, sinking
more or less vertically into the ground, and in these they take up a
peculiar position, for which their shape is specially adapted; the head
and prothorax are broad, the rest of the body slender, the fifth
segment of the abdomen is furnished on the back with a pair of
strong hooks; the ocelli on the sides of the head are very perfect.
Supporting itself at the top of the burrow by means of these hooks
and of its terminal tube, the larva blocks the mouth of the burrow with
its large head and prothorax, and in this position waits for its prey.
This consists of Insects that may alight on the spot or run over it.
When an Insect ventures within reach, the head of the larva is
thrown back with a rapid jerk, the prey is seized by the long sharp
mandibles, dragged to the bottom of the burrow and devoured. The
burrows are often more than a foot deep, and are said to be
excavated by the larva itself, which carries up the earth on the
shovel-like upper surface of its head. The female tiger-beetle is
endowed with powerful and elongate excavating instruments at the
termination of the body, and it is probable that when placing the egg
in the earth she facilitates the future operations of the larva by
forming the outlines of the burrow. Extremely few larvae of
Cicindelidae are known, but they all exhibit the type of structure
mentioned above, and apparently have similar habits. Our little
British Cicindela, most of which are so active on the wing, agree in
these respects with the African species of Manticora, which are
entirely apterous, and are the largest of the Cicindelidae. Péringuey
found a breeding-ground of M. tuberculata near Kimberley; the
larvae were living in the usual Cicindelid manner; but the ground was
so hard that he was not able to investigate the burrows, and there
were but few Insects that could serve as food in the neighbourhood.

Fig. 90—Cicindela hybrida. Britain. A, larva (after Schiödte); B, imago,

male.

The Cicindelidae, although one of the smaller families of Coleoptera,

now number about 1400 species; of these about one-half belong to
the great genus Cicindela, to which our four British representatives
of the Cicindelidae are all assigned. There is no general work of
much consequence on this important family, and its classification is
not thoroughly established.[86]

Fig. 91—Mouth-parts of tiger-beetles. A, Profile of Pogonostoma sp.

(Madagascar): a, antenna; b, labial palp; c, maxillary palp; d,
palpiform lobe of maxilla; e, mandible; f, labrum. B, Section of
head of Manticora maxillosa (South Africa): a, front of upper part
of head-capsule; b, gula; c, tentorium; d, eye; e, labrum; f, left
mandible; g, maxilla; h, maxillary palp; i, labial palp; k, support of
this palp; l, labium.
Tiger-beetles display considerable variety of structure, especially as
regards the mouth, which exhibits very remarkable developments of
the palpi and labrum (Fig. 91). The tiger-beetles, like most other
Insects that capture living prey, do not consume their victims entire,
but subsist chiefly on the juices they squeeze out of them; the hard
and innutritious parts are rejected after the victim has been
thoroughly lacerated and squeezed; the mouth forms both trap and
press; the palpi spread out in order to facilitate the rapid engulfing of
a victim, then close up under it and help to support it in the mouth;
while the labrum above closes the cavity in the other direction. The
mouth itself is a large cavity communicating very freely with the
exterior, but so completely shut off from the following parts of the
alimentary canal that it is difficult to find the orifice of communication;
the labium being much modified to form the posterior wall. For the
capture of the prey, always living but of various kinds, a mechanism
with great holding power and capable of rapid action is required. The
mouth of the terrestrial Manticora (Fig. 91, B), exhibits great
strength; some of the chitinous parts are extremely thick, the
mandibles are enormous, the palpi, however, are comparatively low
in development. In the arboreal genus Pogonostoma the palpary
structures (Fig. 91, A) attain a development scarcely equalled
elsewhere in the Insect world. The great majority of the Cicindelidae
are inhabitants of the warmer, or of the tropical regions of the world,
and very little is known as to their life-histories; they show great
diversity in their modes of hunting their prey. Some are wingless;
others are active on the wing; and of both of these divisions there
are forms that are found only on trees or bushes. Some, it is
believed, frequent only the mounds of Termites. The characteristic
feature common to all is great activity and excessive wariness. The
genus Pogonostoma, to which we have already alluded, is confined
to Madagascar, where the species are numerous, but are rare in
collections on account of the difficulty of securing them. Raffray
informs us that certain species frequent the trunks of trees, up which
they run in a spiral manner on the least alarm. The only way he
could obtain specimens was by the aid of an assistant; the two
approached a tree very quietly from opposite sides, and when near
it, made a rush, and joined hands as high up the trunk as they could,
so as to embrace the tree, when the Pogonostoma fell to the ground
and was captured.

Fig. 92—Leistus spinibarbis. A, Larva (after Schiödte); B, imago.

Britain.

Fam. 5. Carabidae (Ground-beetles).—Clypeus not extending

laterally in front of the antennae. Maxillae with the outer lobe
destitute of an articulated hook at the tip. Antennae covered (except
the basal joints) with a minute pubescence. Hind legs not very
different from the middle pair, formed for running, as usual in beetles.
This is one of the largest and most important of the families of
Coleoptera, including as it does 12,000 or 13,000 described species.
In this country Carabidae are nearly entirely terrestrial in habits, and
are scarcely ever seen on the wing; many of the species indeed
have merely rudimentary wings; in the tropics there are, however,
many arboreal forms that take wing with more or less alertness. The
larvae (Fig. 92, A) are usually elongate in form and run freely; they
may be known by their tarsi ending in two claws, by the exserted,
sharp, calliper-like mandibles, by the body ending in two processes
(sometimes jointed) and a tube of varying length projecting
backwards. The pupae usually have the hind pair of legs so
arranged that the tips of the tarsi project behind, beyond the
extremity of the body. The Carabidae are carnivorous and
predaceous both as larvae and perfect Insects; they attack living
Insects, worms, or other small, soft creatures, but do not disdain
dead specimens. Some species of Carabus, found in North Africa
where snails abound, are specially formed for attacking these
molluscs, having the head long and slender so that it can be thrust
into the shell of the snail. A few species have been detected eating
growing corn, and even the young seeds of some Umbelliferae;
these belong chiefly to the genera Harpalus, Zabrus, and Amara.
Some species of the abundant genera Pterostichus and Harpalus,
are said to be fond of ripe strawberries. The most anomalous forms
of Carabidae are the Pseudomorphides, a sub-family almost peculiar
to Australia, the members of which live under bark, and have but
little resemblance to other Carabids owing to their compact forms
and continuous outlines. The genus Mormolyce is one of the
wonders of the Insect world on account of the extraordinary shape of
its members; the sides of the elytra form large crinkled expansions,
and the head is unusually elongate. These Insects live on the
underside of fallen trees in the Malay Archipelago and Peninsula; no
reason whatever can be at present assigned for their remarkable
shape.

There are a considerable number of blind members of this family:

some of them live in caverns; these belong chiefly to the genus
Anophthalmus, species of which have been detected in the caves of
the Pyrenees, of Austria, and of North America. It has been shown
that the optic nerves and lobes, as well as the external organs of
vision, are entirely wanting in some of these cave Carabidae; the
tactile setae have, however, a larger development than usual, and
the Insects are as skilful in running as if they possessed eyes.
Anophthalmus is closely related to our British genus Trechus, the
species of which are very much given to living in deep crevices in the
earth, or under large stones, and have some of them very small
eyes. In addition to these cavernicolous Anophthalmus, other blind
Carabidae have been discovered during recent years in various
parts of the world, where they live under great stones deeply
embedded in the earth; these blind lapidicolous Carabidae are of
extremely minute size and of most sluggish habits; the situations in
which they are found suggest that many successive generations are
probably passed under the same stone. Not a single specimen has
ever been found above ground. The minute Carabids of the genus
Aëpus, that pass a large part of their lives under stones below high-
water mark (emerging only when the tide uncovers them), on the
borders of the English Channel and elsewhere, are very closely
allied to these blind Insects, and have themselves only very small
eyes, which, moreover, according to Hammond and Miall, are
covered in larger part by a peculiar shield.[87] A few Carabidae, of
the genera Glyptus and Orthogonius, are believed to live in the nests
of Termites. Savage found the larva of G. sculptilis in the nests of
Termes bellicosus; it has been described by Horn, and is said to bear
so great a resemblance to young queens of the Termites as to have
been mistaken for them.[88] Mr. Haviland found Rhopalomelus
angusticollis in Termites' nests in South Africa. Péringuey states that
it emits a very strong and disagreeable odour. It is probable that it
preys on the Termites, and this also is believed to be the habit of the
Ceylonese Helluodes taprobanae. Some species of the
Mediterranean genus Siagona stridulate by means of a file on the
under surface of the prothorax, rubbed by a striate area, adapted in
form, on the anterior femora.

A valuable memoir on the classification of this important family is due

to the late Dr. G. H. Horn;[89] he arranges Carabidae in three sub-
families; we think it necessary to add a fourth for Mormolyce:

1. Middle coxal cavities enclosed externally by the junction of the meso- and
meta-sternum; neither epimeron nor episternum attaining the cavity.
Head beneath, with a deep groove on each side near the eye for the
reception of the antennae or a part thereof. .......... Sub-fam. 3.
Pseudomorphides.
Head without antennal grooves. .......... Sub-fam. 2. Harpalides.
2. Middle coxal cavities attained on the outside by the tips of the episterna
and epimera. .......... Sub-fam. 4. Mormolycides.
3. Middle coxal cavities attained on the outside by the tips of the epimera,
but not by those of the episterna. .......... Sub-fam. 1. Carabides.

These four sub-families are of extremely different extent and nature.

The Harpalides are the dominant forms, and include upwards of
10,000 known species; while the various tribes into which the sub-
family is divided include, as a rule, each many genera; the Carabides
are next in importance, with upwards of 2000 species, but are
divided into a comparatively large number of tribes, each of which
averages a much smaller number of genera than do the tribes of
Harpalides; Pseudomorphides includes only about 100 species; and
Mormolycides consists of the single genus Mormolyce with three
species.

Fam. 6. Amphizoidae.—Antennae destitute of pubescence: outer

lobe of maxilla not jointed; metasternum with a short transverse
impressed line on the middle behind. Hind legs slender, not formed
for swimming. This family is limited to the genus Amphizoa; the
species of which may be briefly described as lowly organised
Carabidae that lead an aquatic life. The geographical distribution is
highly remarkable, there being but three species, two of which live in
Western North America, the third in Eastern Tibet. The habits of
American Amphizoa are known; they pass a life of little activity in
very cold, rapid streams; they do not swim, but cling to stones and
timber. The larva was recently discovered in Utah by Messrs.
Hubbard and Schwarz:[90] it has the same habits as the perfect
Insect, and in general form resembles the larvae of the genus
Carabus; but it has no terminal tube to the body, the abdomen
consisting of eight segments and a pair of short terminal
appendages; the spiracles are obsolete, with the exception of a pair
placed near to one another at the termination of the eighth
abdominal segment. As regards the mouth this larva is Carabid, as
regards the abdomen and stigmata Dytiscid of a primitive type.

Fig. 93—Amphizoa lecontei. North America. A, Larva; B, imago.

 Fig. 94—Pelobius tardus. Britain. A, Young larva; B, adult larva; C,
imago. (A and B after Schiödte.)

Fam. 7. Pelobiidae.—Antennae destitute of pubescence: outer lobe

of maxilla jointed, metasternum with a short transverse impressed
line on the middle behind. Hind legs rather slender, formed for
swimming, the tarsi longer than the tibiae. This family is limited to the
one genus Pelobius (Hygrobia of some authors). Like Amphizoa, to
which it is in several respects analogous, it has a singular
geographical distribution; there are only four known species, one
lives in Britain and the Mediterranean region, one in Chinese Tibet,
two in Australia. Pelobius may be briefly described as a Carabid
adapted to a considerable extent for living in and swimming about in
water; differing thus from Amphizoa, which has no special adaptation
for swimming. The larva of Pelobius is remarkable; it breathes by
means of branchial filaments on the under surface of the body, the
spiracles being present, though those of the abdomen are very
minute and the others small. The head is very large, the mandibles
are not tube-like, the food being taken after the manner of the
Carabidae; the 8th abdominal segment ends in three long
processes; the small 9th segment is retracted beneath them. The
adult Pelobius tardus is remarkable for its loud stridulation. The
sound is produced by an apparatus described correctly by Charles
Darwin;[91] there is a file on the inside of the wing-cases, and the
Insect turns up the tip of the abdomen and scrapes the file therewith.
The Insects are called squeakers in the Covent Garden market,
where they are sold.
 Fig. 95—Cnemidotus caesus. England. A, Imago; B, larva, highly
magnified. (After Schiödte.)

Fam. 8. Haliplidae.—Antennae bare, ten-jointed; metasternum

marked by a transverse line; posterior coxae prolonged as plates,
covering a large part of the lower surface of the abdomen; the
slender, but clubbed, hind femora move between these plates and
the abdomen. The Haliplidae are aquatic, and are all small, not
exceeding four or five millimetres in length. The ventral plates are
peculiar to the Insects of this family, but their function is not known.
The larvae are remarkable on account of the fleshy processes
disposed on their bodies; but they exhibit considerable variety in this
respect; their mandibles are grooved so that they suck their prey. In
the larva of Haliplus, according to Schiödte, there are eight pairs of
abdominal spiracles, but in Cnemidotus (Fig. 95, B), there are no
spiracles, and air is obtained by means of a trachea traversing each
of the long filaments. The Insects of these two genera are so similar
in the imaginal instar that it is well worthy of note that their larvae
should be distinguished by such important characters. Haliplidae is a
small family consisting of three genera, having about 100 species; it
is very widely distributed. We have 13 species in Britain, all the
genera being represented.
Welcome to our website – the ideal destination for book lovers and
knowledge seekers. With a mission to inspire endlessly, we offer a
vast collection of books, ranging from classic literary works to
specialized publications, self-development books, and children's
literature. Each book is a new journey of discovery, expanding
knowledge and enriching the soul of the reade

Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.

Let us accompany you on the journey of exploring knowledge and

personal growth!

textbookfull.com