Download the Full Version of ebook for Fast Typing at ebookmass.

com

Handbook of statistical analysis and data mining

applications Second Edition Elder

https://ebookmass.com/product/handbook-of-statistical-
analysis-and-data-mining-applications-second-edition-elder/

OR CLICK BUTTON

DOWNLOAD NOW

Download More ebook Instantly Today - Get Yours Now at ebookmass.com

 Recommended digital products (PDF, EPUB, MOBI) that
you can download immediately if you are interested.

Predictive Modeling in Biomedical Data Mining and Analysis

Sudipta Roy

https://ebookmass.com/product/predictive-modeling-in-biomedical-data-
mining-and-analysis-sudipta-roy/

ebookmass.com

Handbook of Regression Analysis With Applications in R,

Second Edition Samprit Chatterjee

https://ebookmass.com/product/handbook-of-regression-analysis-with-
applications-in-r-second-edition-samprit-chatterjee/

ebookmass.com

Statistical Methods for Survival Data Analysis 3rd Edition

Lee

https://ebookmass.com/product/statistical-methods-for-survival-data-
analysis-3rd-edition-lee/

ebookmass.com

An Introduction to Statistical Methods and Data Analysis

7th Edition, (Ebook PDF)

https://ebookmass.com/product/an-introduction-to-statistical-methods-
and-data-analysis-7th-edition-ebook-pdf/

ebookmass.com
Data Mining for Business Analytics: Concepts, Techniques
and Applications in Python eBook

https://ebookmass.com/product/data-mining-for-business-analytics-
concepts-techniques-and-applications-in-python-ebook/

ebookmass.com

Intelligent Data Mining and Analysis in Power and Energy

Systems : Models and Applications for Smarter Efficient
Power Systems 1st Edition Zita A. Vale
https://ebookmass.com/product/intelligent-data-mining-and-analysis-in-
power-and-energy-systems-models-and-applications-for-smarter-
efficient-power-systems-1st-edition-zita-a-vale/
ebookmass.com

(eBook PDF) Data Mining and Predictive Analytics 2nd

Edition

https://ebookmass.com/product/ebook-pdf-data-mining-and-predictive-
analytics-2nd-edition/

ebookmass.com

Statistical Topics and Stochastic Models for Dependent

Data With Applications Vlad Stefan Barbu

https://ebookmass.com/product/statistical-topics-and-stochastic-
models-for-dependent-data-with-applications-vlad-stefan-barbu/

ebookmass.com

Spatial analysis using big data: methods and urban

applications Yamagata

https://ebookmass.com/product/spatial-analysis-using-big-data-methods-
and-urban-applications-yamagata/

ebookmass.com
HANDBOOK OF STATISTICAL ANALYSIS
AND DATA MINING APPLICATIONS
 HANDBOOK OF
STATISTICAL
ANALYSIS AND
DATA MINING
APPLICATIONS
SECOND EDITION
AUTHORS

Robert Nisbet, Ph.D.

University of California, Predictive Analytics Certificate Program, Santa Barbara, Goleta, California, USA

Gary Miner, Ph.D.

University of California, Predictive Analytics Certificate Program, Tulsa, Oklahoma and Rome, Georgia, USA

Ken Yale, D.D.S., J.D.

University of California, Predictive Analytics Certificate Program; and Chief Clinical Officer,
Delta Dental Insurance, San Francisco, California, USA

GUEST AUTHORS of selected CHAPTERS

John Elder IV, Ph.D.

Chairman of the Board, Elder Research, Inc., Charlottesville, Virginia, USA

Andy Peterson, Ph.D.

VP for Educational Innovation and Global Outreach, Western Seminary, Charlotte, North Carolina, USA
Academic Press is an imprint of Elsevier
125 London Wall, London EC2Y 5AS, United Kingdom
525 B Street, Suite 1800, San Diego, CA 92101-4495, United States
50 Hampshire Street, 5th Floor, Cambridge, MA 02139, United States
The Boulevard, Langford Lane, Kidlington, Oxford OX5 1GB, United Kingdom

© 2018 Elsevier Inc. All rights reserved.

No part of this publication may be reproduced or transmitted in any form or by any means, electronic or
mechanical, including photocopying, recording, or any information storage and retrieval system, without
permission in writing from the publisher. Details on how to seek permission, further information about the
Publisher’s permissions policies and our arrangements with organizations such as the Copyright Clearance Center
and the Copyright Licensing Agency, can be found at our website: www.elsevier.com/permissions.

This book and the individual contributions contained in it are protected under copyright by the Publisher (other
than as may be noted herein).

Notices
Knowledge and best practice in this field are constantly changing. As new research and experience broaden our
understanding, changes in research methods, professional practices, or medical treatment may become necessary.

Practitioners and researchers must always rely on their own experience and knowledge in evaluating and using
any information, methods, compounds, or experiments described herein. In using such information or methods
they should be mindful of their own safety and the safety of others, including parties for whom they have a
professional responsibility.

To the fullest extent of the law, neither the Publisher nor the authors, contributors, or editors, assume any liability
for any injury and/or damage to persons or property as a matter of products liability, negligence or otherwise, or
from any use or operation of any methods, products, instructions, or ideas contained in the material herein.

Library of Congress Cataloging-in-Publication Data

A catalog record for this book is available from the Library of Congress

British Library Cataloguing-in-Publication Data

A catalogue record for this book is available from the British Library

ISBN 978-0-12-416632-5

For information on all Academic Press publications

visit our website at https://www.elsevier.com/books-and-journals

Publisher: Candice Janco

Acquisition Editor: Graham Nisbet
Editorial Project Manager: Susan Ikeda
Production Project Manager: Paul Prasad Chandramohan
Cover Designer: Alan Studholme

Typeset by SPi Global, India

 List of Tutorials on the Elsevier
Companion Web Page

Note: This list includes all the extra tuto- 8. TUTORIAL “W”—Diabetes Control in
rials published with the 1st edition of this Patients [Field: Medical Informatics]
handbook (2009). These can be considered 9. TUTORIAL “X”—Independent
“enrichment” tutorials for readers of this 2nd Component Analysis [Field: Separating
edition. Since the 1st edition of the handbook Competing Signals]
will not be available after the release of the 10. TUTORIAL “Y”—NTSB Aircraft
2nd edition, these extra tutorials are carried Accidents Reports [Field: Engineering—
over in their original format/versions of soft- Air Travel—Text Mining]
ware, as they are still very useful in learning 11. TUTORIAL “Z”—Obesity Control in
and understanding data mining and predic- Children [Field: Preventive Health Care]
tive analytics, and many readers will want to 12. TUTORIAL “AA”—Random Forests
take advantage of them. Example [Field: Statistics—Data Mining]
List of Extra Enrichment Tutorials that 13. TUTORIAL “BB”—Response
are only on the ELSEVIER COMPANION Optimization [Field: Data Mining—
web page, with data sets as appropriate, for Response Optimization]
downloading and use by readers of this 2nd 14. TUTORIAL “CC”—Diagnostic Tooling
edition of handbook: and Data Mining: Semiconductor Industry
[Field: Industry—Quality Control]
1. TUTORIAL “O”—Boston Housing
15. TUTORIAL “DD”—Titanic—Survivors
Using Regression Trees [Field:
of Ship Sinking [Field: Sociology]
Demographics]
16. TUTORIAL “EE”—Census Data
2. TUTORIAL “P”—Cancer Gene [Field:
Analysis [Field: Demography—Census]
Medical Informatics & Bioinformatics]
17. TUTORIAL “FF”—Linear & Logistic
3. TUTORIAL “Q”—Clustering of Shoppers
Regression—Ozone Data [Field:
[Field: CRM—Clustering Techniques]
Environment]
4. TUTORIAL “R”—Credit Risk
18. TUTORIAL “GG”—R-Language
using Discriminant Analysis [Field:
Integration—DISEASE SURVIVAL
Financial—Banking]
ANALYSIS Case Study [Field: Survival
5. TUTORIAL “S”—Data Preparation and
Analysis—Medical Informatics]
Transformation [Field: Data Analysis]
19. TUTORIAL “HH”—Social Networks
6. TUTORIAL “T”—Model Deployment
Among Community Organizations
on New Data [Field: Deployment of
[Field: Social Networks—Sociology &
Predictive Models]
Medical Informatics]
7. TUTORIAL “V”—Heart Disease Visual
20. TUTORIAL “II”—Nairobi, Kenya
Data Mining Methods [Field: Medical
Baboon Project: Social Networking
Informatics]

xi
xii LIST OF TUTORIALS ON THE ELSEVIER COMPANION WEB PAGE

Among Baboon Populations in Kenya 22. TUTORIAL “KK”—Dahlia Mosaic

on the Laikipia Plateau [Field: Social Virus: A DNA Microarray Analysis of 10
Networks] Cultivars from a Single Source: Dahlia
21. TUTORIAL “JJ”—Jackknife and Garden in Prague, Czech Republic
Bootstrap Data Miner Workspace and [Field: Bioinformatics]
MACRO [Field: Statistics Resampling
Methods]

The final companion site URL will be https://www.elsevier.com/books-and-journals/

book-companion/9780124166325.
 Foreword 1 for 1st Edition

This book will help the novice user be- • Asking the wrong question—when
come familiar with data mining. Basically, looking for a rare phenomenon, it may
data mining is doing data analysis (or statis- be helpful to identify the most common
tics) on data sets (often large) that have been pattern. These may lead to complex
obtained from potentially many sources. As analyses, as in item 3, but they may also
such, the miner may not have control of the be conceptually simple. Again, you may
input data, but must rely on sources that have need to take care that you don't overfit
gathered the data. As such, there are prob- the data.
lems that every data miner must be aware of • Don't become enamored with the data.
as he or she begins (or completes) a mining There may be a substantial history from
operation. I strongly resonated to the mate- earlier data or from domain experts that
rial on “The Top 10 Data Mining Mistakes,” can help with the modeling.
which give a worthwhile checklist: • Be wary of using an outcome variable (or
• Ensure you have a response variable and one highly correlated with the outcome
predictor variables—and that they are variable) and becoming excited about the
correctly measured. result. The predictors should be “proper”
• Beware of overfitting. With scads of predictors in the sense that they (a) are
variables, it is easy with most statistical measured prior to the outcome and (b)
programs to fit incredibly complex are not a function of the outcome.
models, but they cannot be reproduced. It • Do not discard outliers without solid
is good to save part of the sample to use justification. Just because an observation
to test the model. Various methods are is out of line with others is insufficient
offered in this book. reason to ignore it. You must check the
• Don't use only one method. Using only circumstances that led to the value. In
linear regression can be a problem. any event, it is useful to conduct the
Try dichotomizing the response or analysis with the observation(s) included
categorizing it to remove nonlinearities and excluded to determine the sensitivity
in the response variable. Often, there are of the results to the outlier.
clusters of values at zero, which messes • Extrapolating is a fine way to go
up any normality assumption. This, of broke; the best example is the stock
course, loses information, so you may market. Stick within your data, and
want to categorize a continuous response if you must go outside, put plenty
variable and use an alternative to of caveats. Better still, restrain the
regression. Similarly, predictor variables impulse to extrapolate. Beware that
may need to be treated as factors rather pictures are often far too simple and
than linear predictors. A classic example we can be misled. Political campaigns
is using marital status or race as a linear oversimplify complex problems (“my
predictor when there is no order. opponent wants to raise taxes”; “my

xiii
xiv FOREWORD 1 FOR 1st EDITION

opponent will take us to war”) when using mean replacement, almost the
the realities may imply we have same set of predictor variables surfaced,
some infrastructure needs that can be but the residual sum of squares was 20.
handled only with new funding or we I then used multiple imputation and
have been attacked by some bad guys. found approximately the same set of
Be wary of your data sources. If you are predictors but had a residual sum of
combining several sets of data, they need squares (median of 20 imputations) of
to meet a few standards: 25. I find that mean replacement is rather
• The definitions of variables that are optimistic but surely better than relying
being merged should be identical. Often, on only complete cases. Using stepwise
they are close but not exact (especially regression, I find it useful to replicate
in metaanalysis where clinical studies it with a bootstrap or with multiple
may have somewhat different definitions imputations. However, with large data
due to different medical institutions or sets, this approach may be expensive
laboratories). computationally.
• Be careful about missing values. Often, To conclude, there is a wealth of material
when multiple data sets are merged, in this handbook that will repay study.
missing values can be induced: one
variable isn't present in another data set; Peter A. Lachenbruch
what you thought was a unique variable Oregon State University, Corvallis, OR,
name was slightly different in the two United States
sets, so you end up with two variables American Statistical Association,
that both have a lot of missing values. Alexandria, VA, United States
• How you handle missing values can be Johns Hopkins University, Baltimore,
crucial. In one example, I used complete MD, United States
cases and lost half of my sample; all UCLA, Los Angeles, CA, United States
variables had at least 85% completeness, University of Iowa, Iowa City, IA,
but when put together, the sample lost United States
half of the data. The residual sum of University of North Carolina, Chapel
squares from a stepwise regression was Hill, NC, United States
about 8. When I included more variables
 Foreword 2 for 1st Edition

A November 2008 search on https:// This book is different; it is organized

www.amazon.com/ for “data mining” with the practitioner in mind. The volume
books yielded over 15,000 hits—including 72 is divided into four parts. Part I provides
to be published in 2009. Most of these books an overview of analytics from a historical
either describe data mining in very technical perspective and frameworks from which to
and mathematical terms, beyond the reach approach data mining, including CRISP-DM
of most individuals, or approach data min- and SEMMA. These chapters will provide a
ing at an introductory level without suffi- novice analyst an excellent overview by de-
cient detail to be useful to the practitioner. fining terms and methods to use and will
The Handbook of Statistical Analysis and Data provide program managers a framework
Mining Applications is the book that strikes from which to approach a wide variety of
the right balance between these two treat- data mining problems. Part II describes algo-
ments of data mining. rithms, though without extensive mathemat-
This volume is not a theoretical treatment ics. These will appeal to practitioners who
of the subject—the authors themselves recom- are or will be involved with day-to-day an-
mend other books for this—but rather contains alytics and need to understand the qualita-
a description of data mining principles and tive aspects of the algorithms. The inclusion
techniques in a series of “knowledge-­transfer” of a chapter on text mining is particularly
sessions, where examples from real data mining timely, as text mining has shown tremen-
projects illustrate the main ideas. This aspect dous growth in recent years.
of the book makes it most valuable for practi- Part III provides a series of tutorials that
tioners, whether novice or more experienced. are both domain-specific and software-­
While it would be easier for everyone if specific. Any instructor knows that examples
data mining were merely a matter of find- make the abstract concept more concrete,
ing and applying the correct mathematical and these tutorials accomplish exactly that.
equation or approach for any given problem, In addition, each tutorial shows how the
the reality is that both “art” and “science” ­solutions were developed using popular data
are necessary. The “art” in data mining re- mining software tools, such as Clementine,
quires experience: when one has seen and Enterprise Miner, Weka, and STATISTICA.
overcome the difficulties in finding solutions The step-by-step specifics will assist practi-
from among the many possible approaches, tioners in learning not only how to approach
one can apply newfound wisdom to the next a wide variety of problems but also how
project. However, this process takes consid- to use these software products effectively.
erable time, and particularly for data min- Part IV presents a look at the future of data
ing novices, the iterative process inevitable mining, including a treatment of model
in data mining can lead to discouragement ensembles and “The Top 10 Data Mining
when a “textbook” approach doesn't yield a Mistakes,” from the popular presentation by
good solution. Dr. Elder.

xv
xvi FOREWORD 2 FOR 1st EDITION

However, the book is best read a few the ­excellent “History of Statistics and Data
chapters at a time while actively doing Mining” chapter and chapters 16, 17, and
the data mining rather than read cover to 18. These are broadly applicable and should
cover (a daunting task for a book this size). be read by even the most experienced data
Practitioners will appreciate tutorials that miners.
match their business objectives and choose The Handbook of Statistical Analysis and
to ignore other tutorials. They may choose Data Mining Applications is an exceptional
to read sections on a particular algorithm to book that should be on every data miner's
increase insight into that algorithm and then bookshelf or, better yet, found lying open
decide to add a second algorithm after the next to their computer.
first is mastered. For those new to a partic-
ular software tool highlighted in the tutori- Dean Abbott
als section, the step-by-step approach will Abbott Analytics, San Diego, CA,
operate much like a user's manual. Many United States
chapters stand well on their own, such as
 Preface

Much has happened in the professional • Operations research—decision science

discipline known previously as data mining and optimization of business processes
since the first edition of this book was written • Stock market quants—focused on
in 2008. This discipline has broadened and stock market trading and portfolio
deepened to a very large extent, requiring a optimization.
major reorganization of its elements. A new • Data engineering—focused on
parent discipline was formed, data science, optimizing data flow through memories
which includes previous subjects and activi- and storage structures
ties in data mining and many new elements • Business intelligence—focused primarily
of the scientific study of data, including stor- on descriptive aspects of data but
age structures optimized for analytic use, predictive aspects are coming
data ethics, and performance of many ac- • Business analytics—focused primarily
tivities in business, industry, and education. on the predictive aspects of data but is
Analytic aspects that used to be included in merging with descriptives
data mining have broadened considerably to (based on an article by Vincent

include image analysis, facial recognition, in- Granville published in http://www.
dustrial performance and control, threat de- datasciencecentral.com/profiles/
tection, fraud detection, astronomy, national blogs/17-analytic-disciplines-compared.)
security, weather forecasting, and financial In this book, we will use the terms “data
forensics. Consequently, several subdisci- mining” and “predictive analytics” synony-
plines have been erected to contain various mously, even though data mining includes
specialized data analytic applications. These many descriptive operations also.
subdisciplines of data science include the Modern data mining tools, like the ones
following: featured in this book, permit ordinary busi-
• Machine learning—analytic algorithm ness analysts to follow a path through the
design and optimization data mining process to create models that
• Data mining—generally restricted in are “good enough.” These less-than-optimal
scope now to pattern recognition apart models are far better in their ability to lever-
from causes and interpretation age faint patterns in databases to solve prob-
• Predictive analytics—using algorithms to lems than the ways it used to be done. These
predict things, rather than describe them tools provide default configurations and
or manage them automatic operations, which shield the user
• Statistical analysis—use of parametric from the technical complexity underneath.
statistical algorithms for analysis and They provide one part in the crude analogy
prediction to the automobile interface. You don't have
• Industrial statistical analysis—analytic to be a chemical engineer or physicist who
techniques to control and direct industrial understands moments of force to be able to
operations operate a car. All you have to do is learn to

xvii
xviii PREFACE

turn the key in the ignition, step on the gas is enough here to permit you to construct
and the brake at the right times, and turn the “smart enough” business operations with a
wheel to change direction in a safe manner, relatively small amount of the right informa-
and voilà, you are an expert user of the very tion. James Taylor developed this concept
complex technology under the hood. The for automating operational ­decision-making
other half of the story is the instruction man- in the area of enterprise decision man-
ual and the driver's education course that agement (Raden and Taylor, 2007). Taylor
help you to learn how to drive. recognized that companies need decision-
This book provides the instruction man- making systems that are automated enough
ual and a series of tutorials to train you how to keep up with the volume and time-­critical
to do data mining in many subject areas. We nature of modern business operations.
provide both the right tools and the right These decisions should be deliberate, pre-
intuitive explanations (rather than formal cise, and consistent across the enterprise;
mathematical definitions) of the data mining smart enough to serve immediate needs
process and algorithms, which will enable appropriately; and agile enough to adapt
even beginner data miners to understand the to new opportunities and challenges in the
basic concepts necessary to understand what company. The same concept can be applied
they are doing. In addition, we provide many to nonoperational systems for customer re-
tutorials in many different industries and lationship management (CRM) and market-
businesses (using many of the most common ing support. Even though a CRM model for
data mining tools) to show how to do it. cross sell may not be optimal, it may enable
several times the response rate in prod-
uct sales following a marketing campaign.
OVERALL ORGANIZATION Models like this are “smart enough” to drive
OF THIS BOOK companies to the next level of sales. When
models like this are proliferated through-
We have divided the chapters in this book out the enterprise to lift all sales to the next
into four parts to guide you through the as- level, more refined models can be developed
pects of predictive analytics. Part I covers the to do even better. This e­ nterprise-wide “lift”
history and process of predictive analytics. in intelligent operations can drive a com-
Part II discusses the algorithms and methods pany through evolutionary rather than rev-
used. Part III is a group of tutorials, which olutionary changes to reach long-term goals.
serve in principle as Rome served—as the Companies can leverage “smart enough”
central governing influence. Part IV presents decision systems to do likewise in their pur-
some advanced topics. The central theme of suit of optimal profitability in their business.
this book is the education and training of Clearly, the use of this book and these tools
beginning data mining practitioners, not the will not make you experts in data mining.
rigorous academic preparation of algorithm Nor will the explanations in the book per-
scientists. Hence, we located the tutorials in mit you to understand the complexity of the
the middle of the book in Part III, flanked by theory behind the algorithms and methodol-
topical chapters in Parts I, II, and IV. ogies so necessary for the academic student.
This approach is “a mile wide and an inch But we will conduct you through a relatively
deep” by design, but there is a lot packed into thin slice across the wide practice of data
that inch. There is enough here to stimulate mining in many industries and disciplines.
you to take deeper dives into theory, and there We can show you how to create powerful
 PREFACE xix
predictive models in your own organization Coauthor Gary Miner wishes to thank his
in a relatively short period of time. In addi- wife, Linda A. Winters-Miner, PhD, who has
tion, this book can function as a springboard been working with Gary on similar books over
to launch you into higher-level studies of the the past 30 years and wrote several of the tu-
theory behind the practice of data mining. torials included in this book, using ­real-world
If we can accomplish those goals, we will data. Gary also wishes to thank the following
have succeeded in taking a significant step in people from his office who helped in various
bringing the practice of data mining into the ways, including Angela Waner, Jon Hillis, Greg
mainstream of business analysis. Sergeant, and Dr. Thomas Hill, who gave per-
The three coauthors could not have done mission to use and also edited a group of the
this book completely by themselves, and tutorials that had been written over the years
we wish to thank the following individuals, by some of the people listed as guest authors in
with the disclaimer that we apologize if, by this book. Dr. Dave Dimas, of the University of
our neglect, we have left out of this “thank- California—Irvine, has also been very helpful
you list” anyone who contributed. in providing suggestions for enhancements for
Foremost, we would like to thank ac- this second edition—THANK YOU DAVE !!!
quisitions editor (name to use?) and others Without all the help of the people men-
(names). Bob Nisbet would like to honor tioned here and maybe many others we failed
and thank his wife, Jean Nisbet, PhD, who to specifically mention, this book would never
blasted him off in his technical career by re- have been completed. Thanks to you all!
typing his PhD dissertation five times (be-
fore word processing) and assumed much
of the family's burdens during the writing Bob Nisbet
of this book. Bob also thanks Dr. Daniel B. Gary Miner
Botkin, the famous global ecologist, for in- Ken Yale
troducing him to the world of modeling and
exposing him to the distinction between
viewing the world as machine and viewing Reference
it as ­organism. And thanks are due to Ken Raden, N., Taylor, J., 2007. Smart Enough Systems: How to
Reed, PhD, for inducting Bob into the prac- Deliver Competitive Advantage by Automating Hidden
Decisions. Prentice Hall, NJ, ISBN: 9780132713061.
tice of data mining.
 Introduction

Often, data analysts are asked, “What very different ways of arriving at the same
are statistical analysis and data mining?” In conclusion, a decision. We will introduce
this book, we will define what data mining some basic analytic history and theory in
is from a procedural standpoint. But most Chapters 1 and 2.
people have a hard time relating what we The basic process of analytic modeling is
tell them to the things they know and under- presented in Chapter 3. But it may be diffi-
stand. Before moving on into the book, we cult for you to relate what is happening in
would like to provide a little background for the process without some sort of tie to the
data mining that everyone can relate to. The real world that you know and enjoy. In many
Preface describes the many changes in ac- ways, the decisions served by analytic mod-
tivities related to data mining since the first eling are similar to those we make every day.
edition of this book was published in 2009. These decisions are based partly on patterns
Now, it is time to dig deeper and discuss the of action formed by experience and partly by
differences between statistical analysis and intuition.
data mining (aka predictive analytics).
Statistical analysis and data mining are PATTERNS OF ACTION
two methods for simulating the unconscious
operations that occur in the human brain to A pattern of action can be viewed in
provide a rationale for decision-making and terms of the activities of a hurdler on a
actions. Statistical analysis is a very directed race track. The runner must start success-
rationale that is based on norms. We all think fully and run to the first hurdle. He must
and make decisions on the basis of norms. decide very quickly how high to jump to
For example, we consider (unconsciously) clear the hurdle. He must decide when and
what the norm is for dress in a certain situa- in what sequence to move his legs to clear
tion. Also, we consider the acceptable range the hurdle with minimum effort and with-
of variation in dress styles in our culture. out knocking it down. Then, he must run
Based on these two concepts, the norm and a specified distance to the next hurdle and
the variation around that norm, we render do it all over again several times, until he
judgments like “that man is inappropriately crosses the finish line. Analytic modeling is
dressed.” Using similar concepts of mean a lot like that.
and standard deviation, statistical analy- The training of the hurdler's “model” of
sis proceeds in a very logical way to make action to run the race happens in a series of
very similar judgments (in principle). On operations:
the other hand, data mining learns case by
case and does not use means or standard • Run slow at first.
deviations. Data mining algorithms build • Practice takeoff from different positions
patterns, clarifying the pattern as each case to clear the hurdle.
is submitted for processing. These are two • Practice different ways to move the legs.

xxi
xxii INTRODUCTION

• Determine the best ways to do each activity. two classes (for dichotomous keys) and those
• Practice the best ways for each activity who don't. Along with this joke is a similar
over and over again. recognition from the outside that taxono-
mists are divided also into two classes: the
This practice trains the sensory and motor
“lumpers” (who combine several species into
neurons to function together most efficiently.
one) and the “splitters” (who divide one spe-
Individual neurons in the brain are “trained”
cies into many). These distinctions point to
in practice by adjusting signal strengths and
a larger dichotomy in the way people think.
firing thresholds of the motor nerve cells. The
In ecology, there used to be two schools
performance of a successful hurdler follows
of thought: autoecologists (chemistry, phys-
the “model” of these activities and the process
ics, and mathematics explain all) and the
of coordinating them to run the race. Creation
synecologists (organism relationships in
of an analytic “model” of a business process to
their environment explain all). It wasn't until
predict a desired outcome follows a very simi-
the 1970s that these two schools of thought
lar path to the training regimen of a hurdler. We
learned that both perspectives were needed
will explore this subject further in Chapter 3
to understand the complexities in ecosys-
and apply it to develop a data mining process
tems (but more about that later). In business,
that expresses the basic activities and tasks per-
there are the “big picture” people versus
formed in creating an analytic model.
“detail” people. Some people learn by fol-
lowing an intuitive pathway from general to
HUMAN INTUITION specific (deduction). Often, we call them “big
picture” people. Other people learn by fol-
In humans, the right side of the brain is lowing an intuitive pathway from specific to
the center for visual and esthetic sensibil- general (inductive). Often, we call them “de-
ities. The left side of the brain is the center tail” people. Similar distinctions are reflected
for quantitative and time-regulated sensi- in many aspects of our society. In Chapter 1,
bilities. Human intuition is a blend of both we will explore this distinction to a greater
sensibilities. This blend is facilitated by the depth in regards to the development of sta-
neural connections between the right side tistical and data mining theory through time.
of the brain and the left side. In women, the Many of our human activities involve
number of neural connections between the finding patterns in the data input to our sen-
right and left sides of the brain is 20% greater sory systems. An example is the mental pat-
(on average) than in men. This higher con- tern that we develop by sitting in a chair in
nectivity of women's brains enables them to the middle of a shopping mall and making
exercise intuitive thinking to a greater extent some judgment about patterns among its cli-
than men. Intuition “builds” a model of re- entele. In one mall, people of many ages and
ality from both quantitative building blocks races may intermingle. You might conclude
and visual sensibilities (and memories). from this pattern that this mall is located in
an ethnically diverse area. In another mall,
you might see a very different pattern. In
PUTTING IT ALL one mall in Toronto, a great many of the
TOGETHER stores had Chinese titles and script on the
windows. One observer noticed that he was
Biological taxonomy students claim (in the only non-Asian seen for a half hour. This
jest) that there are two kinds of people in led to the conclusion that the mall catered
­taxonomy—those who divide things up into to the Chinese community and was owned
Discovering Diverse Content Through
Random Scribd Documents
accessible by the widest array of equipment including outdated
equipment. Many small donations ($1 to $5,000) are particularly
important to maintaining tax exempt status with the IRS.

The Foundation is committed to complying with the laws

regulating charities and charitable donations in all 50 states of
the United States. Compliance requirements are not uniform
and it takes a considerable effort, much paperwork and many
fees to meet and keep up with these requirements. We do not
solicit donations in locations where we have not received written
confirmation of compliance. To SEND DONATIONS or
determine the status of compliance for any particular state visit
www.gutenberg.org/donate.

While we cannot and do not solicit contributions from states

where we have not met the solicitation requirements, we know
of no prohibition against accepting unsolicited donations from
donors in such states who approach us with offers to donate.

International donations are gratefully accepted, but we cannot

make any statements concerning tax treatment of donations
received from outside the United States. U.S. laws alone swamp
our small staff.

Please check the Project Gutenberg web pages for current

donation methods and addresses. Donations are accepted in a
number of other ways including checks, online payments and
credit card donations. To donate, please visit:
www.gutenberg.org/donate.

Section 5. General Information About Project

Gutenberg™ electronic works
Professor Michael S. Hart was the originator of the Project
Gutenberg™ concept of a library of electronic works that could
be freely shared with anyone. For forty years, he produced and
distributed Project Gutenberg™ eBooks with only a loose
network of volunteer support.

Project Gutenberg™ eBooks are often created from several

printed editions, all of which are confirmed as not protected by
copyright in the U.S. unless a copyright notice is included. Thus,
we do not necessarily keep eBooks in compliance with any
particular paper edition.

Most people start at our website which has the main PG search
facility: www.gutenberg.org.

This website includes information about Project Gutenberg™,

including how to make donations to the Project Gutenberg
Literary Archive Foundation, how to help produce our new
eBooks, and how to subscribe to our email newsletter to hear
about new eBooks.
Welcome to our website – the ideal destination for book lovers and
knowledge seekers. With a mission to inspire endlessly, we offer a
vast collection of books, ranging from classic literary works to
specialized publications, self-development books, and children's
literature. Each book is a new journey of discovery, expanding
knowledge and enriching the soul of the reade

Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.

Let us accompany you on the journey of exploring knowledge and

personal growth!

ebookmass.com