Full download ebook at ebookmass.

com

Introduction to algorithms for data mining

and machine learning Yang

https://ebookmass.com/product/introduction-to-
algorithms-for-data-mining-and-machine-learning-
yang/

Download more ebook from https://ebookmass.com

More products digital (pdf, epub, mobi) instant
download maybe you interests ...

Introduction to Algorithms for Data Mining and Machine

Learning Xin-She Yang

https://ebookmass.com/product/introduction-to-algorithms-for-data-
mining-and-machine-learning-xin-she-yang/

ebookmass.com

Fundamentals of Machine Learning for Predictive Data

Analytics: Algorithms,

https://ebookmass.com/product/fundamentals-of-machine-learning-for-
predictive-data-analytics-algorithms/

ebookmass.com

Machine Learning for Signal Processing: Data Science,

Algorithms, and Computational Statistics Max A. Little

https://ebookmass.com/product/machine-learning-for-signal-processing-
data-science-algorithms-and-computational-statistics-max-a-little/

ebookmass.com

BIG DATA ANALYTICS: Introduction to Hadoop, Spark, and

Machine-Learning Raj Kamal

https://ebookmass.com/product/big-data-analytics-introduction-to-
hadoop-spark-and-machine-learning-raj-kamal/

ebookmass.com
Machine Learning for Biometrics: Concepts, Algorithms and
Applications (Cognitive Data Science in Sustainable
Computing) Partha Pratim Sarangi
https://ebookmass.com/product/machine-learning-for-biometrics-
concepts-algorithms-and-applications-cognitive-data-science-in-
sustainable-computing-partha-pratim-sarangi/
ebookmass.com

Machine Learning Algorithms for Signal and Image

Processing Suman Lata Tripathi

https://ebookmass.com/product/machine-learning-algorithms-for-signal-
and-image-processing-suman-lata-tripathi/

ebookmass.com

Learn Data Mining Through Excel: A Step-by-Step Approach

for Understanding Machine Learning Methods, 2nd Edition
Hong Zhou
https://ebookmass.com/product/learn-data-mining-through-excel-a-step-
by-step-approach-for-understanding-machine-learning-methods-2nd-
edition-hong-zhou/
ebookmass.com

Absolute Beginner’s Guide to Algorithms: A Practical

Introduction to Data Structures and Algorithms in
JavaScript (for True Epub) Kirupa Chinnathambi
https://ebookmass.com/product/absolute-beginners-guide-to-algorithms-
a-practical-introduction-to-data-structures-and-algorithms-in-
javascript-for-true-epub-kirupa-chinnathambi/
ebookmass.com

Absolute Beginner's Guide to Algorithms: A Practical

Introduction to Data Structures and Algorithms in
JavaScript Kirupa Chinnathambi
https://ebookmass.com/product/absolute-beginners-guide-to-algorithms-
a-practical-introduction-to-data-structures-and-algorithms-in-
javascript-kirupa-chinnathambi/
ebookmass.com
Introduction to Algorithms for Data Mining and
Machine Learning
Introduction to
Algorithms for Data
Mining and Machine
Learning

Xin-She Yang
Middlesex University
School of Science and Technology
London, United Kingdom
Academic Press is an imprint of Elsevier
125 London Wall, London EC2Y 5AS, United Kingdom
525 B Street, Suite 1650, San Diego, CA 92101, United States
50 Hampshire Street, 5th Floor, Cambridge, MA 02139, United States
The Boulevard, Langford Lane, Kidlington, Oxford OX5 1GB, United Kingdom
Copyright © 2019 Elsevier Inc. All rights reserved.
No part of this publication may be reproduced or transmitted in any form or by any means, electronic or
mechanical, including photocopying, recording, or any information storage and retrieval system, without
permission in writing from the publisher. Details on how to seek permission, further information about the
Publisher’s permissions policies and our arrangements with organizations such as the Copyright Clearance Center
and the Copyright Licensing Agency, can be found at our website: www.elsevier.com/permissions.
This book and the individual contributions contained in it are protected under copyright by the Publisher (other
than as may be noted herein).
Notices
Knowledge and best practice in this field are constantly changing. As new research and experience broaden our
understanding, changes in research methods, professional practices, or medical treatment may become necessary.
Practitioners and researchers must always rely on their own experience and knowledge in evaluating and using
any information, methods, compounds, or experiments described herein. In using such information or methods
they should be mindful of their own safety and the safety of others, including parties for whom they have a
professional responsibility.
To the fullest extent of the law, neither the Publisher nor the authors, contributors, or editors, assume any liability
for any injury and/or damage to persons or property as a matter of products liability, negligence or otherwise, or
from any use or operation of any methods, products, instructions, or ideas contained in the material herein.

Library of Congress Cataloging-in-Publication Data

A catalog record for this book is available from the Library of Congress

British Library Cataloguing-in-Publication Data

A catalogue record for this book is available from the British Library

ISBN: 978-0-12-817216-2

For information on all Academic Press publications

visit our website at https://www.elsevier.com/books-and-journals

Publisher: Candice Janco

Acquisition Editor: J. Scott Bentley
Editorial Project Manager: Michael Lutz
Production Project Manager: Nilesh Kumar Shah
Designer: Miles Hitchen
Typeset by VTeX
About the author

Xin-She Yang obtained his PhD in Applied Mathematics from the University of Ox-
ford. He then worked at Cambridge University and National Physical Laboratory (UK)
as a Senior Research Scientist. Now he is Reader at Middlesex University London, and
an elected Bye-Fellow at Cambridge University.
He is also the IEEE Computer Intelligence Society (CIS) Chair for the Task Force
on Business Intelligence and Knowledge Management, Director of the International
Consortium for Optimization and Modelling in Science and Industry (iCOMSI), and
an Editor of Springer’s Book Series Springer Tracts in Nature-Inspired Computing
(STNIC).
With more than 20 years of research and teaching experience, he has authored
10 books and edited more than 15 books. He published more than 200 research pa-
pers in international peer-reviewed journals and conference proceedings with more
than 36 800 citations. He has been on the prestigious lists of Clarivate Analytics and
Web of Science highly cited researchers in 2016, 2017, and 2018. He serves on the
Editorial Boards of many international journals including International Journal of
Bio-Inspired Computation, Elsevier’s Journal of Computational Science (JoCS), In-
ternational Journal of Parallel, Emergent and Distributed Systems, and International
Journal of Computer Mathematics. He is also the Editor-in-Chief of the International
Journal of Mathematical Modelling and Numerical Optimisation.
Preface

Both data mining and machine learning are becoming popular subjects for university
courses and industrial applications. This popularity is partly driven by the Internet and
social media because they generate a huge amount of data every day, and the under-
standing of such big data requires sophisticated data mining techniques. In addition,
many applications such as facial recognition and robotics have extensively used ma-
chine learning algorithms, leading to the increasing popularity of artificial intelligence.
From a more general perspective, both data mining and machine learning are closely
related to optimization. After all, in many applications, we have to minimize costs,
errors, energy consumption, and environment impact and to maximize sustainabil-
ity, productivity, and efficiency. Many problems in data mining and machine learning
are usually formulated as optimization problems so that they can be solved by opti-
mization algorithms. Therefore, optimization techniques are closely related to many
techniques in data mining and machine learning.
Courses on data mining, machine learning, and optimization are often compulsory
for students, studying computer science, management science, engineering design, op-
erations research, data science, finance, and economics. All students have to develop
a certain level of data modeling skills so that they can process and interpret data for
classification, clustering, curve-fitting, and predictions. They should also be familiar
with machine learning techniques that are closely related to data mining so as to carry
out problem solving in many real-world applications. This book provides an introduc-
tion to all the major topics for such courses, covering the essential ideas of all key
algorithms and techniques for data mining, machine learning, and optimization.
Though there are over a dozen good books on such topics, most of these books are
either too specialized with specific readership or too lengthy (often over 500 pages).
This book fills in the gap with a compact and concise approach by focusing on the key
concepts, algorithms, and techniques at an introductory level. The main approach of
this book is informal, theorem-free, and practical. By using an informal approach all
fundamental topics required for data mining and machine learning are covered, and
the readers can gain such basic knowledge of all important algorithms with a focus
on their key ideas, without worrying about any tedious, rigorous mathematical proofs.
In addition, the practical approach provides about 30 worked examples in this book
so that the readers can see how each step of the algorithms and techniques works.
Thus, the readers can build their understanding and confidence gradually and in a
step-by-step manner. Furthermore, with the minimal requirements of basic high school
mathematics and some basic calculus, such an informal and practical style can also
enable the readers to learn the contents by self-study and at their own pace.
This book is suitable for undergraduates and graduates to rapidly develop all the
fundamental knowledge of data mining, machine learning, and optimization. It can
 Visit https://ebookmass.com
now to explore a rich
collection of eBooks and enjoy
exciting offers!
xii Preface

also be used by students and researchers as a reference to review and refresh their
knowledge in data mining, machine learning, optimization, computer science, and data
science.

Xin-She Yang
January 2019 in London
Acknowledgments

I would like to thank all my students and colleagues who have given valuable feedback
and comments on some of the contents and examples of this book. I also would like to
thank my editors, J. Scott Bentley and Michael Lutz, and the staff at Elsevier for their
professionalism. Last but not least, I thank my family for all the help and support.

Xin-She Yang
January 2019
Introduction to optimization
Contents
1.1 Algorithms
1 1
1.1.1 Essence of an algorithm 1
1.1.2 Issues with algorithms 3
1.1.3 Types of algorithms 3
1.2 Optimization 4
1.2.1 A simple example 4
1.2.2 General formulation of optimization 7
1.2.3 Feasible solution 9
1.2.4 Optimality criteria 10
1.3 Unconstrained optimization 10
1.3.1 Univariate functions 11
1.3.2 Multivariate functions 12
1.4 Nonlinear constrained optimization 14
1.4.1 Penalty method 15
1.4.2 Lagrange multipliers 16
1.4.3 Karush–Kuhn–Tucker conditions 17
1.5 Notes on software 18

This book introduces the most fundamentals and algorithms related to optimization,
data mining, and machine learning. The main requirement is some understanding of
high-school mathematics and basic calculus; however, we will review and introduce
some of the mathematical foundations in the first two chapters.

1.1 Algorithms
An algorithm is an iterative, step-by-step procedure for computation. The detailed
procedure can be a simple description, an equation, or a series of descriptions in
combination with equations. Finding the roots of a polynomial, checking if a natu-
ral number is a prime number, and generating random numbers are all algorithms.

1.1.1 Essence of an algorithm

In essence, an algorithm can be written as an iterative equation or a set of iterative
equations. For example, to find a square root of a > 0, we can use the following
iterative equation:
1 a
xk+1 = xk + , (1.1)
2 xk
where k is the iteration counter (k = 0, 1, 2, . . . ) starting with a random guess x0 = 1.
Introduction to Algorithms for Data Mining and Machine Learning. https://doi.org/10.1016/B978-0-12-817216-2.00008-9
Copyright © 2019 Elsevier Inc. All rights reserved.
2 Introduction to Algorithms for Data Mining and Machine Learning

Example 1
As an example, if x0 = 1 and a = 4, then we have

1 4
x1 = (1 + ) = 2.5. (1.2)
2 1

Similarly, we have

1 4 1 4
x2 = (2.5 + ) = 2.05, x3 = (2.05 + ) ≈ 2.0061, (1.3)
2 2.5 2 2.05
x4 ≈ 2.00000927, (1.4)
√
which is very close to the true value of 4 = 2. The accuracy of this iterative formula or algorithm
is high because it achieves the accuracy of five decimal places after four iterations.

The convergence is very quick if we start from different initial values such as
x0 = 10 and even x0 = 100. However, for an obvious reason, we cannot start with
x0 = 0 due to division by
√zero.
Find the root of x = a is equivalent to solving the equation

f (x) = x 2 − a = 0, (1.5)

which is again equivalent to finding the roots of a polynomial f (x). We know that
Newton’s root-finding algorithm can be written as

f (xk )
xk+1 = xk − , (1.6)
f  (xk )

where f  (x) is the first derivative or gradient of f (x). In this case, we have
f  (x) = 2x. Thus, Newton’s formula becomes

(xk2 − a)
xk+1 = xk − , (1.7)
2xk

which can be written as

xk a 1 a
xk+1 = (xk − )+ = xk + ). (1.8)
2 2xk 2 xk

This is exactly what we have in Eq. (1.1).

Newton’s method has rigorous mathematical foundations, which has a guaranteed
convergence under certain conditions. However, in general, Eq. (1.6) is more general,
and the gradient information f  (x) is needed. In addition, for the formula to be valid,
we must have f  (x) = 0.
Introduction to optimization 3

1.1.2 Issues with algorithms

The advantage of the algorithm given in Eq. (1.1) is that√it converges very quickly.
However, careful readers may have asked: we know that 4 = ±2, how can we find
the other root −2 in addition to +2?
Even if we use different initial value x0 = 10 or x0 = 0.5, we can only reach x∗ = 2,
not −2.
What happens if we start with x0 < 0? From x0 = −1, we have
1 4 1 4
x1 = (−1 + ) = −2.5, x 2 = (−2.5 + ) = −2.05, (1.9)
2 −1 2 −2.5
x3 ≈ −2.0061, x4 ≈ −2.00000927, (1.10)
which is approaching −2 very quickly. If we start from x0 = −10 or x0 = −0.5, then
we can always get x∗ = −2, not +2.
This highlights a key issue here: the final solution seems to depend on the initial
starting point for this algorithm, which is true for many algorithms.
Now the relevant question is: how do we know where to start to get a particular
solution? The general short answer is “we do not know”. Thus, some knowledge of
the problem under consideration or an educated guess may be useful to find the final
solution.
In fact, most algorithms may depend on the initial configuration, and such algo-
rithms are often carrying out search moves locally. Thus, this type of algorithm is
often referred to as local search. A good algorithm should be able to “forget” its initial
configuration though such algorithms may not exist at all for most types of problems.
What we need in general is the global search, which attempts to find final solutions
that are less sensitive to the initial starting point(s).
Another important issue in our discussions is that the gradient information f  (x) is
necessary for some algorithms such as Newton’s method given in Eq. (1.6). This poses
certain requirements on the smoothness of the function f (x). For example, we know
that |x| is not differentiable at x = 0. Thus, we cannot directly use Newton’s method
to find the roots of f (x) = |x|x 2 − a = 0 for a > 0. Some modifications are needed.
There are other issues related to algorithms such as the setting of parameters, the
slow rate of convergence, condition numbers, and iteration structures. All these make
algorithm designs and usage somehow challenging, and we will discuss these issues
in more detail later in this book.

1.1.3 Types of algorithms

An algorithm can only do a specific computation task (at most a class of computational
tasks), and no algorithms can do all the tasks. Thus, algorithms can be classified due
to their purposes. An algorithm to find roots of a polynomial belongs to root-finding
algorithms, whereas an algorithm for ranking a set of numbers belongs to sorting
algorithms. There are many classes of algorithms for different purposes. Even for the
same purpose such as sorting, there are many different algorithms such as the merge
sort, bubble sort, quicksort, and others.
4 Introduction to Algorithms for Data Mining and Machine Learning

We can also categorize algorithms in terms of their characteristics. The root-finding

algorithms we just introduced are deterministic algorithms because the final solutions
are exactly the same if we start from the same initial guess. We obtain the same set of
solutions every time we run the algorithm. On the other hand, we may introduce some
randomization into the algorithm, for example, using purely random initial points.
Every time we run the algorithm, we use a new random initial guess. In this case, the
algorithm can have some nondeterministic nature, and such algorithms are referred
to as stochastic.√Sometimes, using randomness may be advantageous. For example, in
the example of 4 = ±2 using Eq. (1.1), random initial values (both positive and neg-
ative) can allow the algorithm to find both roots. In fact, a major trend in the modern
metaheuristics is using some randomization to suit different purposes.
For algorithms to be introduced in this book, we are mainly concerned with al-
gorithms for data mining, optimization, and machine learning. We use a relatively
unified approach to link algorithms in data mining and machine learning to algorithms
for optimization.

1.2 Optimization

Optimization is everywhere, from engineering design to business planning. After all,

time and resources are limited, and optimal use of such valuable resources is crucial.
In addition, designs of products have to maximize the performance, sustainability, and
energy efficiency and to minimize the costs. Therefore, optimization is important for
many applications.

1.2.1 A simple example

Let us start with a very simple example to design a container with volume capacity
V0 = 10 m3 . As the main cost is related to the cost of materials, the main aim is to
minimize the total surface area S.
The first thing we have to decide is the shape of the container (cylinder, cubic,
sphere or ellipsoid, or more complex geometry). For simplicity, let us start with a
cylindrical shape with radius r and height h (see Fig. 1.1).
The total surface area of a cylinder is

S = 2(πr 2 ) + 2πrh, (1.11)

and the volume is

V = πr 2 h. (1.12)

There are only two design variables r and h and one objective function S to be min-
imized. Obviously, if there is no capacity constraint, then we can choose not to build
the container, and then the cost of materials is zero for r = 0 and h = 0. However,
Introduction to optimization 5

Figure 1.1 Design of a cylindric container.

the constraint requirement means that we have to build a container with fixed volume
V0 = πr 2 h = 10 m3 . Therefore, this optimization problem can be written as

minimize S = 2πr 2 + 2πrh, (1.13)

subject to the equality constraint

πr 2 h = V0 = 10. (1.14)

To solve this problem, we can first try to use the equality constraint to reduce the
number of design variables by solving h. So we have
V0
h= . (1.15)
πr 2
Substituting it into (1.13), we get

S = 2πr 2 + 2πrh
V0 2V0
= 2πr 2 + 2πr 2 = 2πr 2 + . (1.16)
πr r
This is a univariate function. From basic calculus we know that the minimum or max-
imum can occur at the stationary point, where the first derivative is zero, that is,
dS 2V0
= 4πr − 2 = 0, (1.17)
dr r
which gives

V0 3 V0
r3 = , or r = . (1.18)
2π 2π
Thus, the height is

h V0 /(πr 2 ) V0
= = 3 = 2. (1.19)
r r πr
 Visit https://ebookmass.com
now to explore a rich
collection of eBooks and enjoy
exciting offers!
6 Introduction to Algorithms for Data Mining and Machine Learning

This means that the height is twice the radius: h = 2r. Thus, the minimum surface is

S∗ = 2πr 2 + 2πrh = 2πr 2 + 2πr(2r) = 6πr 2

 V 2/3 6π
0 2/3
= 6π =√3
V0 . (1.20)
2π 4π 2

For V0 = 10, we have

 
3 V0 3 10
r= = ≈ 1.1675, h = 2r = 2.335,
(2π) 2π

and the total surface area

S∗ = 2πr 2 + 2πrh ≈ 25.69.

It is worth pointing out that this optimal solution is based on the assumption or re-
quirement to design a cylindrical container. If we decide to use a sphere with radius R,
we know that its volume and surface area is
4π 3
V0 = R , S = 4πR 2 . (1.21)
3
We can solve R directly

3V0 3 3V0
R =
3
, or R = , (1.22)
4π 4π
which gives the surface area
 3V 2/3 √
0 4π 3 9 2/3
S = 4π =√ 3
V0 . (1.23)
4π 16π 2
√3 √ √ 3
Since 6π/ 4π 2 ≈ 5.5358 and 4π 3 9/ 16π 2 ≈ 4.83598, we have S < S∗ , that is, the
surface area of a sphere is smaller than the minimum surface area of a cylinder with
the same volume. In fact, for the same V0 = 10, we have
√
4π 3 9 2/3
S(sphere) = √ 3
V0 ≈ 22.47, (1.24)
16π 2
which is smaller than S∗ = 25.69 for a cylinder.
This highlights the importance of the choice of design type (here in terms of shape)
before we can do any truly useful optimization. Obviously, there are many other fac-
tors that can influence the choice of design, including the manufacturability of the
design, stability of the structure, ease of installation, space availability, and so on. For
a container, in most applications, a cylinder may be much easier to produce than a
sphere, and thus the overall cost may be lower in practice. Though there are so many
factors to be considered in engineering design, for the purpose of optimization, here
we will only focus on the improvement and optimization of a design with well-posed
mathematical formulations.
Introduction to optimization 7

1.2.2 General formulation of optimization

Whatever the real-world applications may be, it is usually possible to formulate an
optimization problem in a generic form [49,53,160]. All optimization problems with
explicit objectives can in general be expressed as a nonlinearly constrained optimiza-
tion problem

maximize/minimize f (x), x = (x1 , x2 , . . . , xD )T ∈ RD ,

subject to φj (x) = 0 (j = 1, 2, . . . , M),
ψk (x) ≤ 0 (k = 1, . . . , N), (1.25)

where f (x), φj (x), and ψk (x) are scalar functions of the design vector x. Here the
components xi of x = (x1 , . . . , xD )T are called design or decision variables, and they
can be either continuous, discrete, or a mixture of these two. The vector x is often
called the decision vector, which varies in a D-dimensional space RD .
It is worth pointing out that we use a column vector here for x (thus with trans-
pose T ). We can also use a row vector x = (x1 , . . . , xD ) and the results will be the
same. Different textbooks may use slightly different formulations. Once we are aware
of such minor variations, it should cause no difficulty or confusion.
In addition, the function f (x) is called the objective function or cost function,
φj (x) are constraints in terms of M equalities, and ψk (x) are constraints written as
N inequalities. So there are M + N constraints in total. The optimization problem
formulated here is a nonlinear constrained problem. Here the inequalities ψk (x) ≤ 0
are written as “less than”, and they can also be written as “greater than” via a simple
transformation by multiplying both sides by −1.
The space spanned by the decision variables is called the search space RD , whereas
the space formed by the values of the objective function is called the objective or
response space, and sometimes the landscape. The optimization problem essentially
maps the domain RD or the space of decision variables into the solution space R (or
the real axis in general).
The objective function f (x) can be either linear or nonlinear. If the constraints φj
and ψk are all linear, it becomes a linearly constrained problem. Furthermore, when
φj , ψk , and the objective function f (x) are all linear, then it becomes a linear pro-
gramming problem [35]. If the objective is at most quadratic with linear constraints,
then it is called a quadratic programming problem. If all the values of the decision
variables can be only integers, then this type of linear programming is called integer
programming or integer linear programming.
On the other hand, if no constraints are specified and thus xi can take any values
in the real axis (or any integers), then the optimization problem is referred to as an
unconstrained optimization problem.
As a very simple example of optimization problems without any constraints, we
discuss the search of the maxima or minima of a univariate function.
8 Introduction to Algorithms for Data Mining and Machine Learning

2
Figure 1.2 A simple multimodal function f (x) = x 2 e−x .

Example 2
For example, to find the maximum of a univariate function f (x)

f (x) = x 2 e−x ,
2
−∞ < x < ∞, (1.26)

is a simple unconstrained problem, whereas the following problem is a simple constrained mini-
mization problem:

f (x1 , x2 ) = x12 + x1 x2 + x22 , (x1 , x2 ) ∈ R2 , (1.27)

subject to

x1 ≥ 1, x2 − 2 = 0. (1.28)

It is worth pointing out that the objectives are explicitly known in all the optimiza-
tion problems to be discussed in this book. However, in reality, it is often difficult to
quantify what we want to achieve, but we still try to optimize certain things such as the
degree of enjoyment or service quality on holiday. In other cases, it may be impossible
to write the objective function in any explicit form mathematically.
From basic calculus we know that, for a given curve described by f (x), its gradient
f  (x) describes the rate of change. When f  (x) = 0, the curve has a horizontal tangent
at that particular point. This means that it becomes a point of special interest. In fact,
the maximum or minimum of a curve occurs at
f  (x∗ ) = 0, (1.29)

which is a critical condition or stationary condition. The solution x∗ to this equation

corresponds to a stationary point, and there may be multiple stationary points for a
given curve.
To see if it is a maximum or minimum at x = x∗ , we have to use the information of
its second derivative f  (x). In fact, f  (x∗ ) > 0 corresponds to a minimum, whereas
f  (x∗ ) < 0 corresponds to a maximum. Let us see a concrete example.

Example 3
To find the minimum of f (x) = x 2 e−x (see Fig. 1.2), we have the stationary condition
2

f  (x) = 0 or

f  (x) = 2x × e−x + x 2 × (−2x)e−x = 2(x − x 3 )e−x = 0.

2 2 2
Introduction to optimization 9

Figure 1.3 (a) Feasible domain with nonlinear inequality constraints ψ1 (x) and ψ2 (x) (left) and linear
inequality constraint ψ3 (x). (b) An example with an objective of f (x) = x 2 subject to x ≥ 2 (right).

As e−x > 0, we have

2

x(1 − x 2 ) = 0, or x = 0 and x = ±1.

The second derivative is given by

f  (x) = 2e−x (1 − 5x 2 + 2x 4 ),
2

which is an even function with respect to x.

So at x = ±1, f  (±1) = 2[1 − 5(±1)2 + 2(±1)4 ]e−(±1) = −4e−1 < 0. Thus, there are
2

two maxima that occur at x∗ = ±1 with fmax = e−1 . At x = 0, we have f  (0) = 2 > 0, thus
the minimum of f (x) occurs at x∗ = 0 with fmin (0) = 0.

Whatever the objective is, we have to evaluate it many times. In most cases, the
evaluations of the objective functions consume a substantial amount of computational
power (which costs money) and design time. Any efficient algorithm that can reduce
the number of objective evaluations saves both time and money.
In mathematical programming, there are many important concepts, and we will
first introduce a few related concepts: feasible solutions, optimality criteria, the strong
local optimum, and weak local optimum.

1.2.3 Feasible solution

A point x that satisfies all the constraints is called a feasible point and thus is a feasible
solution to the problem. The set of all feasible points is called the feasible region (see
Fig. 1.3).
For example, we know that the domain f (x) = x 2 consists of all real numbers. If
we want to minimize f (x) without any constraint, all solutions such as x = −1, x = 1,
and x = 0 are feasible. In fact, the feasible region is the whole real axis. Obviously,
x = 0 corresponds to f (0) = 0 as the true minimum.
However, if we want to find the minimum of f (x) = x 2 subject to x ≥ 2, then it
becomes a constrained optimization problem. The points such as x = 1 and x = 0 are
no longer feasible because they do not satisfy x ≥ 2. In this case the feasible solutions
are all the points that satisfy x ≥ 2. So x = 2, x = 100, and x = 108 are all feasible. It
is obvious that the minimum occurs at x = 2 with f (2) = 22 = 4, that is, the optimal
solution for this problem occurs at the boundary point x = 2 (see Fig. 1.3).
10 Introduction to Algorithms for Data Mining and Machine Learning

Figure 1.4 Local optima, weak optima, and global optimality.

1.2.4 Optimality criteria

A point x ∗ is called a strong local maximum of the nonlinearly constrained op-
timization problem if f (x) is defined in a δ-neighborhood N (x ∗ , δ) and satisfies
f (x ∗ ) > f (u) for u ∈ N (x ∗ , δ), where δ > 0 and u = x ∗ . If x ∗ is not a strong lo-
cal maximum, then the inclusion of equality in the condition f (x ∗ ) ≥ f (u) for all
u ∈ N (x ∗ , δ) defines the point x ∗ as a weak local maximum (see Fig. 1.4). The local
minima can be defined in a similar manner when > and ≥ are replaced by < and ≤,
respectively.
Fig. 1.4 shows various local maxima and minima. Point A is a strong local max-
imum, whereas point B is a weak local maximum because there are many (in fact,
infinite) different values of x that will lead to the same value of f (x ∗ ). Point D is the
global maximum, and point E is the global minimum. In addition, point F is a strong
local minimum. However, point C is a strong local minimum, but it has a discontinuity
in f  (x ∗ ). So the stationary condition for this point f  (x ∗ ) = 0 is not valid. We will
not deal with these types of minima or maxima in detail.
As we briefly mentioned before, for a smooth curve f (x), optimal solutions usu-
ally occur at stationary points where f  (x) = 0. This is not always the case because
optimal solutions can also occur at the boundary, as we have seen in the previous ex-
ample of minimizing f (x) = x 2 subject to x ≥ 2. In our present discussion, we will
assume that both f (x) and f  (x) are always continuous or f (x) is everywhere twice
continuously differentiable. Obviously, the information of f  (x) is not sufficient to
determine whether a stationary point is a local maximum or minimum. Thus, higher-
order derivatives such as f  (x) are needed, but we do not make any assumption at this
stage. We will further discuss this in detail in the next section.

1.3 Unconstrained optimization

Optimization problems can be classified as either unconstrained or constrained. Un-

constrained optimization problems can in turn be subdivided into univariate and mul-
tivariate problems.
Introduction to optimization 11

1.3.1 Univariate functions

The simplest optimization problem without any constraints is probably the search for
the maxima or minima of a univariate function f (x). For unconstrained optimization
problems, the optimality occurs at the critical points given by the stationary condition
f  (x) = 0.
However, this stationary condition is just a necessary condition, but it is not a suf-
ficient condition. If f  (x∗ ) = 0 and f  (x∗ ) > 0, it is a local minimum. Conversely, if
f  (x∗ ) = 0 and f  (x∗ ) < 0, then it is a local maximum. However, if f  (x∗ ) = 0 and
f  (x∗ ) = 0, care should be taken because f  (x) may be indefinite (both positive and
negative) when x → x∗ , then x∗ corresponds to a saddle point.
For example, for f (x) = x 3 , we have

f  (x) = 3x 2 , f  (x) = 6x. (1.30)

The stationary condition f  (x) = 3x 2 = 0 gives x∗ = 0. However, we also have

f  (x∗ ) = f  (0) = 0.

In fact, f (x) = x 3 has a saddle point x∗ = 0 because f  (0) = 0 but f  changes sign
from f  (0+) > 0 to f  (0−) < 0 as x moves from positive to negative.

Example 4
For example, to find the maximum or minimum of a univariate function

f (x) = 3x 4 − 4x 3 − 12x 2 + 9, −∞ < x < ∞,

we first have to find its stationary points x∗ when the first derivative f  (x) is zero, that is,

f  (x) = 12x 3 − 12x 2 − 24x = 12(x 3 − x 2 − 2x) = 0.

Since f  (x) = 12(x 3 − x 2 − 2x) = 12x(x + 1)(x − 2) = 0, we have

x∗ = −1, x∗ = 2, x∗ = 0.

The second derivative of f (x) is simply

f  (x) = 36x 2 − 24x − 24.

From the basic calculus we know that the maximum requires f  (x∗ ) ≤ 0 whereas the minimum
requires f  (x∗ ) ≥ 0.
At x∗ = −1, we have

f  (−1) = 36(−1)2 − 24(−1) − 24 = 36 > 0,

so this point corresponds to a local minimum

f (−1) = 3(−1)4 − 4(−1)3 − 12(−1)2 + 9 = 4.

Random documents with unrelated
content Scribd suggests to you:
but for all that exceedingly peremptory, way, insists in putting on his
readers—a huge pair of blinkers. We are to regard the late
seventeenth and the whole of the eighteenth century as an Age of
Prose: and we are to regard Johnson, whether he was speaking of
the poets of this age or of others, as the spokesman of an age of
prose. Far be it from me to deny that there is an element of truth in
this: but it is not the whole truth, and the critic must strive, though he
may not boast, to “find the whole.”
The whole truth, as it seems to me, about Johnson is that he was
very much more than the critic of an age of prose, though he was not
(who has been? even Longinus? even Coleridge?)
 “The King who ruled, as he thought fit,
The universal monarchy of wit”

as regards poetic criticism. He saw far beyond prose, as in those few

words of the concluding and reconciling eulogy of Gray which have been
quoted above. It is poetry and not prose which has the gift of putting new
things so that the man who reads them ingenuously thinks that they are
merely a neat statement of what he has always thought. And Johnson
was far more than merely a critic of the eighteenth-century Neo-Classic
theory, though he was this. A most noteworthy passage in the Rambler
(No. 156), which I have purposely kept for comment in this place, though
it is delivered on the wrong side, shows us, as the great critics always do
show us, what a range of sight the writer had. In this he expresses a
doubt whether we ought “to judge genius merely by the event,” and,
applying this to Shakespeare, takes the odd, but for an eighteenth-
century critic most tell-tale and interesting, line that if genius succeeds by
means which are wrong according to rule, we may think higher of the
genius but less highly of the work. It is hardly necessary to point out that
this is, though in no way a discreditable, a transparent evasion of the
difficulty which is pressing on the defenders of the Rules. “Show me,” one
may without irreverence retort, “thy genius without thy works; and I will
show thee my genius by my works.” If Shakespeare shows genius in
neglecting the Rules, the inexorable voice of Logic, greater than Fortune,
greater than all other things save Fate, will point out that the Rules are
evidently not necessary, and, with something like the Lucretian Te sequar,
will add, “Then for what are they necessary?” But Johnson’s power is only
a little soured and not at all quenched by this. He has seen what others
refused—perhaps were unable—to see, and what some flatly denied,—
that a process of literary judgment “by the event” is possible, and that its
verdicts, in some respects at any rate, cannot be challenged or reversed.
These great critical aperçus, though sometimes delivered half unwillingly
or on the wrong side, establish Johnson’s claim to a place not often to be
given to critics; but they do not establish it more certainly than his surveys
of his actual subjects. It was an unfortunate consequence of Mr Arnold’s
generous impatience of all but “the chief and principal things,” and of his
curious dislike to literary history as such, that he should have swept away
the minor Lives. One may not care for Stepney or Yalden, Duke or King,
much more, or at all more, than he did. But with a really great member of
the craft his admissions and omissions, his paradoxes, his
extravagances, his very mistakes pure and simple, are all critically
edifying. How does he apply his own critical theory? is what we must ask:
and, with Johnson, I think we shall never ask it in vain.
His idea of English poetry was the application to certain classes of
subjects, not rigidly limited to, but mainly arranged by, the canons of the
classical writers—of what seemed to him and his generation the supreme
form of English language and metre, brought in by Mr Waller and
perfected by Mr Pope, yet not so as to exclude from admiration the
Allegro of Milton and the Elegy of Gray. We may trace his applications of
this, if we have a real love of literature and a real sense of criticism,
nearly as profitably and pleasantly in relation to John Pomfret as in
relation to Alexander Pope. We may trace his failures (as we are pleased,
quite rightly in a way, to call them), the failures arising from the
inadequacy, not of his genius, but of his scheme, not less agreeably in
relation to Dyer than in relation to Dryden. We are not less informed by
his passing the Castle of Indolence almost sub silentio than we are by
that at first sight astounding criticism of Lycidas. This Cæsar never does
wrong but with just cause—to use the phrase which was too much for the
equanimity or the intelligence of his great namesake Ben, in the work of
one whom both admired yet could not quite stomach.
Now, this it is which makes the greatness of a critic. That Johnson
might have been greater still at other times need not necessarily be
denied; though it is at least open to doubt whether any other time would
have suited his whole disposition better. But, as he is, he is great. The
critics who deserve that name are not those who, like, for instance,
Christopher North and Mr Ruskin, are at the mercy of different kinds of
caprice—with whom you must be always on the qui vive to be certain
what particular watchword they have adopted, what special side they are
taking. It may even be doubted whether such a critic as Lamb, though
infinitely delightful, is exactly “great” because of the singular gaps and
arbitrariness of his likes and dislikes. Nay, Hazlitt, one of the greatest
critics of the world on the whole, goes near to forfeit his right to the title by
the occasional outbursts of almost insane prejudice that cloud his vision.
Johnson is quite as prejudiced; but his prejudice is not in the least insane.
His critical calculus is perfectly sound on its own postulates and axioms;
and you have only to apply checks and correctives (which are easily
ascertained, and kept ready) to adjust it to absolute critical truth. And,
what is more, he has not merely flourished and vapoured critical
abstractions, but has left us a solid reasoned body of critical judgment; he
has not judged literature in the exhausted receiver of mere art, and yet
has never neglected the artistic criterion; he has kept in constant touch
with life, and yet has never descended to mere gossip. We may freely
disagree with his judgments, but we can never justly disable his
judgment; and this is the real criterion of a great critic.
Johnson is so much the eighteenth-century orthodox critic in
quintessence (though, as I have tried to show, in transcendence also) that
he will dispense us from saying very much more about the rank and file,
Minor the ordinary or inferior examples, of the kind. If we were
Criticism: able to devote this Book, or even this volume, to the
Periodical and subject of the present chapter, there would be no lack of
other.
material. Critical exercitations of a kind formed now, of
course, a regular part of the work of literature, and a very large part of its
hack-work. The Gentleman’s Magazine devoted much attention to the
subject; and for a great part of the century two regular Reviews, the
Critical and the Monthly,[640] were recognised organs of literary
censorship, and employed some really eminent hands, notably Smollett
and Goldsmith. The periodicals which, now in single spies, now (about
the middle of the century) in battalions, endeavoured to renew the
success of the Tatler and Spectator, were critical by kind; and dozens,
scores, hundreds probably, of separate critical publications, large and
small, issued from the press.[641] But, with the rarest exceptions, they
must take the non-benefit of the warning which was laid down in the
Preface to the First Volume. Something we must say of Goldsmith; then
we may take two contrasted examples, Knox and Scott of Amwell, of the
critic in Johnson’s last days who inclined undoubtingly to the classical,
and of the critic of the same time who had qualms and stirrings of
Romanticism, but was hardly yet a heretic. And then, reserving summary,
we may close the record.
Of Goldsmith as a critic little need be said, though his pen was not
much less prolific in this than in other departments. But the angel is too
Goldsmith. often absent, and Poor Poll distressingly in evidence.
The Inquiry into the Present State of Polite Learning in
Europe is simply “prodigious.” It is admirably written—Macaulay owes
something to its style, which he only hardened and brazened. The author
apes the fashionable philosophastering of the time, and throws in cheap
sciolism like the prince of journalists that he was. It is almost always
interesting; it is, where it touches life, not literature, sometimes excellently
acute; but there is scarcely a critical dictum in it which is other than
ridiculous. So in the Citizen of the World the Author’s Club is of course
delightful; but why should a sneer at Drayton have been put in the mouth
of Lien Chi Altangi? And the miscellaneous Essays, including the Bee,
which contain so much of Goldsmith’s best work, are perhaps the best
evidences of his nullity here. When one thinks how little it would cost
anybody of Goldsmith’s genius (to find such an one I confess would cost
more) to write a literary parallel to the magnificent Reverie, which would
be even finer, it is enough to draw iron tears down the critic’s cheek.
Goldsmith on Taste, Poetry, Metaphor, &c.,[642] is still the Goldsmith of the
Inquiry. His “Account of the Augustan Age,”[643] though much better, and
(unless I mistake) resorted to by some recent critics as a source of
criticism different from that mostly prevalent in the nineteenth century, has
all the limitations of its own period. And the Essay on Versification,[644]
though it contains expressions which, taken by themselves, might seem
to show that Goldsmith had actually emancipated himself from the
tyranny of the fixed number of syllables, contains others totally
irreconcilable with these, supports English hexameters and sapphics,[645]
and as a whole forces on us once more the reluctant belief that he simply
had no clear ideas, no accurate knowledge, on the subject.
Vicesimus Knox[646] is a useful figure in this critical Transition Period. A
Vicesimus scholar and a schoolmaster, he had some of the
Knox. advantages of the first state and some of the defects of
the less gracious second, accentuated in both cases by the dying
influences of a “classical” tradition which had not the slightest idea that it
was moribund. He carries his admiration for Pope to such a point as to
assure us somewhere that Pope was a man of exemplary piety and
goodness, while Gay was “uncontaminated with the vices of the world,”
which is really more than somewhat blind, and more than a little kind,
even if we admit that it is wrong to call Pope a bad man, and that Gay
had only tolerable vices. He thinks, in his Fourteenth Essay on the
“Fluctuations of Taste,” that the Augustans “arrived at that standard of
perfection which,” &c.; that the imitators of Ariosto, Spenser, and the
smaller poems of Milton are “pleasingly uncouth” [compare Scott, infra,
on the metrical renaissance of Dyer], depreciates Gray, and dismisses
the Elegy as “a confused heap of splendid ideas”; is certain that Milton’s
sonnets “bear no mark of his genius,” and in discussing the versions of
“the sensible[647] Sappho” decides that Catullus is much inferior to—
Philips! “The Old English Poets [Essay Thirty-Nine] are deservedly
forgotten.” Chaucer, Gower, Lydgate, and Occleve “seem to have thought
that rhyme was poetry, and even this constituent they applied with
extreme negligence”—the one charge which is unfair against even
Occleve, and which, in reference to Chaucer, is proof of utter ignorance.
Patriotism probably made him more favourable to Dunbar, Douglas, and
Lyndsay, though he groans over the necessity of a glossary in their case
also. In fact, Knox is but a Johnson without the genius. Let it, however, be
counted to him for righteousness that he defended classical education,
including verse-writing, against its enemies, who even then imagined vain
things.
John Scott of Amwell, once praised by good wits, now much forgotten,
Scott of was a very respectable critic and a poet of “glimmerings.”
Amwell. In fact, I am not at all sure that he does not deserve to be
promoted and postponed to the next volume, as a representative of the
rising, not the falling, tide. His Essays on poetry[648] exhibit in a most
interesting way the “know-not-what-to-think-of-it” state of public opinion
about the later years of Johnson. He defends Lycidas against the
Dictator; yet he finds fault with the “daystar” for acting both as a person
and an orb of radiance, and admits the “incorrectness” of the poem,
without giving us a hint of the nature or authority of “correctness.” He
boldly attacks the consecrated Cooper’s Hill, and sets the rival eminence
of Grongar against it, pronouncing Dyer “a sublime but strangely
neglected poet,” yet picking very niggling holes in this poet himself. He
often anticipates, and oftener seems to be going to anticipate,
Wordsworth, who no doubt owed him a good deal; yet he thinks Pope’s
famous epigram on Wit “the most concise and just definition of Poetry.” In
Grongar Hill itself he thinks the “admixture of metre [its second, certainly,
if not its first great charm] rather displeasing to a nice ear”; and though he
defends Gray against Knox, he is altogether yea-nay about Windsor
Forest, and attacks Thomson’s personifications, without remembering
that Gray is at least an equal sinner, and without giving the author of the
Seasons, and still more of the Castle of Indolence, any just compensation
for his enthusiasm of nature. In fact, Scott is a man walking in twilight,
who actually sees the line of dawn, but dares not step out into it.

552. An interesting monograph on our subject, before and after 1700, is

Herr Paul Hamelius’s Die Kritik in der Engl. Literatur des 17 und 18
Jahrhunderts (Leipsic, 1897). I was able, as I always prefer to do, to
postpone the reading of this till I had finished the English part of this
volume, and I do not think I owe Herr Hamelius much. I am all the more
glad to find that we agree on the Romantic element in Dryden (though not
as to that in Dennis), and as to reducing the importance of French
influence in England.
 553. The excessively rare Parliament of Critics (London, 1702), a copy
of which has been kindly lent me by Mr Gregory Smith, is more of what it
calls itself, a “banter,” than of a serious composition. But it connects itself
not obscurely with the Collier quarrel.
554. See Mr Swinburne’s William Blake, p. 130 note, for the sortes
Bysshianæ of Blake and his wife.
555. My copy is the Third Edition, “with large improvements,” London,
1708. Some put the first at 1702, not 1700. Before Bysshe, Joshua Poole,
a schoolmaster, had given posthumously (1657: I have ed. 2, London,
1677), with a short dedication and a curious verse proem of his own, and
an Institution signed J. D., The English Parnassus. This contains a double
gradus of epithets and passages (the authors named only in a general
list), an “Alphabet of [Rhyming] Monosyllables,” and some “Forms of
Compliment,” &c. The Institution stoutly defends “Rhythm” [i.e., rhyme],
notices Sidney, Daniel, Puttenham, &c., shortly defines Kinds, objects to
excessive enjambment (note the time, 1657) and to polysyllables, but is
sensible if rather general and scrappy.
556. Addison, Atterbury, Beaumont and Fletcher, Afra Behn,
Blackmore, Tom Brown, Buckingham, Cleveland, Congreve, Cowley,
Creech, Davenant (2), Denham, Dennis, Dorset, Dryden, Duke, Garth,
Halifax, Harvey, Sir R. Howard, Hudibras, Jonson, Lee, Milton, Mulgrave,
Oldham, Otway, Prior, Ratcliff, Rochester, Roscommon, Rowe, Sedley,
Shakespeare, Southern, Sprat, Stafford, Stepney, Suckling, Tate, Walsh,
Waller, Wycherley, and Yalden. Observe that no non-dramatic poet earlier
than Cowley is admitted.
557. London, 1718.
558. The Lives and Characters of the English Dramatic Poets, &c., First
begun by Mr Langbain, improved and continued down to this time by a
Careful Hand (London, printed for Tho. Leigh, &c. No date in my copy, but
the Dict. Nat. Biog. gives 1699).
559. I hope the passing suspicion is not illiberal. But why should he call
the Palmyrene “Zenobie” in English? Cela sent furieusement son
Français. (For the critical work of yet another who felt the lash of Pope—
James Ralph—v. inf., p. 554 note.)
560. 2 vols., London, 1718.
561. See, among others, Herr Hamelius, op. cit. Yet it is interesting to
find that the passage of Dennis to which his panegyrist gives the single
and signal honour of extract in an appendix is purely ethical: it is all on
“the previous question.”
 562. Had Dryden let his Cambridge admirer see the Heads? (v. supra,
pp. 373, 397 notes.)
563. Although Dennis’s fun is heavy enough, there are some
interesting touches, as this: “Port [then a novelty in England, remember]
is not so well tasted as Claret: and intoxicates sooner.”
564. It appeared in the very year of the Short View (1698). I have a
reprint of it, issued many years later (1725), but long before Dennis’s
death, together with The Advancement and Reformation of Modern
Poetry and the tragedy of Rinaldo and Armida, all separately titled, but
continuously paged.
565. This is from the Advancement and Reformation, which contains its
author’s full definition of Poetry itself—not the worst of such definitions.
“Poetry is an Imitation of Nature by a pathetic and numerous speech.”
566. London, 1702.
567. London, 1712.
568. London, 1728.
569. The Advancement and Reformation of Poetry, 1701; A Large
Account of the Taste in Poetry, next year; and Grounds of Criticism in
Poetry, 1704.
570. Mr W. Basil Worsfold in his Principles of Criticism (London, 1897).
I hope that nothing which, in a politely controversial tone, I may have to
say here, will be taken as disparagement of a very interesting and
valuable essay.
571. The most convenient edition of Addison’s Works is that of Bohn,
with Hurd’s editorial matter and a good deal more (London, 6 vols., 1862).
572. It is fair to say that he never published this, and that, as Pope told
Spence, he used himself to call it “a poor thing,” and admitted that he
spoke of some of the poets only “on hearsay.” Now when Pope speaks to
Addison’s credit it is not as “what the soldier said.” It is evidence, and of
the strongest.
573.

“In vain he jests in his unpolished strain,

And tries to make his readers laugh in vain.”
574. “His moral lay so bare that it lost the effect” (Ess. on Po., iii. 420,
ed. cit. sup.) Indeed it has been suggested that Addison’s debt to Temple
here is not confined to this.
575. He proposes to give an account of “all the Muse possessed”
between Chaucer and Dryden; and, as a matter of fact, mentions nobody
but Spenser between Chaucer and Cowley.
 576. In the last paragraph of Sp. 409. The whole paper has been
occupied by thoughts on Taste and Criticism: it contains the excellent
comparison of a critic to a tea-taster, and it ends with this retrospect, and
the promise of the “Imagination” Essays (v. ed. cit., iii. 393).
577. Sp. 58-63.
578. Sp. 39, 40, 42, 44, 45.
579. These began in Sp. 267, and were the regular Saturday feature of
the paper for many weeks. References to Milton outside of them will be
found in the excellent index of the ed. cit. or in that of Mr Gregory Smith’s
exact and elegant reproduction of the Spectator (8 vols., London, 1897).
580. Sp. 411, ed. cit., iii. 394.
581. This phrase is originally Dryden’s (dedication to King Arthur, viii.
136, ed. cit.), who, however, has “kind” for “way”.
582. Op. cit., pp. 93-107, and more largely pp. 55-93.
583. Students of the Stagirite may be almost equally surprised to find
Aristotle regarded as mainly, if not wholly, a critic of Form as opposed to
Thought.
584. See vol. i. p. 165 sq.
585. It would be unfair to lay too much stress on his identification of
Imagination and Fancy; but there is something tell-tale in it.
586. See vol. i. p. 118 sq.
587. Herr Hamelius, op. cit. sup., p. 103, and elsewhere, thinks much
more highly of Steele than I do, and even makes him a “Romantic before
Romanticism.” Steele’s temperament was undoubtedly Romantic, and
both in essays and plays he displayed it; but he was not really critical.
588. In his Preface to the Second Part of the Poems (1690).
589. Of course he might, to some extent, have sheltered himself under
Dryden’s own authority for all this.
590. I have thought it useless to give references to particular editions of
the better known writings of Swift and Pope, as they are so numerous. Of
their whole works there is, in the former case, no real standard, Scott’s
being much inferior to his Dryden; but in the latter that of the late Mr Elwin
and Mr Courthope is not likely soon to be superseded.
591. V. inf., p. 553 note.
592. The most important of these is the sentence on Crashaw (with
whom Pope has some points of sympathy), that he is wanting in “design,
form, fable, which is the soul of poetry,” and “exactness or consent of
parts, which is the body,” while he grants him “pretty conceptions, fine
metaphors, glittering expressions, and something of a neat cast of verse,
which are properly the dress, gems, or loose ornaments” of it. See my
friend Mr Courthope (in his Life, ed. cit. of the Works, v. 63), with whom,
for once, I am in irreconcilable disagreement.
593. Spence (whose Anecdotes were printed partly by Malone, and
completely by Singer in 1820, reprinted from the latter edition in 1858,
and re-selected by Mr Underhill (London, n. d.) in the last decade of the
nineteenth century) has sometimes received praise as a critic himself. His
Polymetis usefully brought together classical art and letters, and the
Anecdotes themselves are not without taste. But his elaborate criticism of
Pope’s Odyssey, published in 1726, is of little value, neither praising nor
blaming its subject for the right things, and characterised as a whole by a
pottering and peddling kind of censorship.
594. Selecta Poemata Italorum qui Latine Scripserunt. Cura cujusdam
Anonymi anno 1684 congesta, iterum in lucem data, una cum aliorum
Italorum operibus. Acccurante A. Pope. 2 vols., London, 1740. The title-
page contains absolutely all the ostensible editorial matter, and, as I have
not got hold of the work of the Anonymus, I do not know how much Pope
added. But his collection, as I can testify from some little knowledge of
the subject, is good.
595. Ep. to Aug., l. 263.
596. Pope, v. supra, p. 454, actually admitted this as regards Aristotle
and Shakespeare; yet the admission practically revokes most of the
Essay.
597. Individual preference, in the case of the famous pair of epigrams
on the books and the troop of horse sent by George I. to Cambridge and
to Oxford respectively, may be biassed by academical and by political
partisanship. But while it is matter of opinion whether “Tories own no
argument but force,” and whether, in certain circumstances, a University
may not justifiably “want loyalty,” no one can ever maintain that it is not
disgraceful to a university to “want learning.” This it is which gives the
superior wing and sting to Trapp’s javelin.
598. Prælectiones Poeticæ, London, 3rd ed., 1736. The first of the first
batch was printed as early as 1711, and an English translation (not by the
author) was published in 1742. I hope to give in the next volume, as a
prelude to notice of Mr Arnold’s work in the Oxford Chair, a survey of all
the more noteworthy of his predecessors.
599. The first ed. is that of Edinburgh, 1783: mine is that of London,
1823.
600. I have it with The Poems of Ossian, 2 vols., London, 1796. Blair
had taken Macpherson under his wing as early as 1760.
 601. It had reached its eighth edition in 1807, the date of my copy.
Perhaps some may think that Kames, as being mainly an æsthetician,
ought to be postponed with Shaftesbury, Hume, &c. My reason for not
postponing is the large amount of positive literary criticism in his book.
602. Boswell, Globe ed., p. 132. He was elsewhere more, and less,
kind.
603. Vol. i. chap. iii., on “Beauty”; i. 195 ed. cit.
604. i. 77.
605. i. 26.
606. i. 288, note. Kames had just before, in his chapter on “Motion and
Force” (i. 250-255), referred complacently to his own indulgence in this
foible, and had accumulated others of the same kind.
607. i. 359.
608. i. 405, 410, 411, 416, 417.
609. ii. 3.
610. ii. 163.
611. ii. 129.
612. i. 33.
613. Hurd is reserved for the next volume.
614. ii. 457.
615. Kames has this spelling, which is indeed so universal that any
other may seem pedantic. Yet it is needless to say that the word so spelt
is a vox nihili, and should be “parallelepipedon.”
616. I use the Tegg edition, London, 1850.
617. He had, of course, good authority for it, including that of Dryden;
but it is obviously better to limit it in the modern sense than to use it
equivocally. Mason (not Gray’s friend, but an interesting and little-known
person to whom I hope to recur in the next volume) had already seen this,
and expressly referred to it.
618. Works, Oxford, 1841.
619. Note to Pt. II. chap. vii. of the Enquiries, p. 433, ed. cit.
620. Harris deserves a good word for his prosodic studies, which may
entitle him to reappear in the next volume.
621. “There never was a time when rules did not exist; they always
made a part of that immutable truth,” &c.—P. 450.
622. The best known is Cowper’s, in Table Talk, ll. 384, 385—

“The inestimable Estimate of Brown

Rose like a paper-kite and charmed the town.”
See also Chesterfield, to the Bishop of Waterford, April 14, 1758.
Chesterfield was no Bottom, but, being melancholy at the time, he was
tickled.
623. London, 1757, 8vo.
624. London, 1763, 4to.
625. Newcastle, 1764, 8vo.
626. His birth-year was 1709; Thomson’s 1700; Dyer’s perhaps the
same; Shenstone’s 1714; Gray’s 1716. Lady Winchelsea had been born
as far back as 1660.
627. He was perhaps the last man of very great power who entertained
the Renaissance superstition of Latin. He was horrified at the notion of an
English epitaph; and in the first agony of his stroke in 1783 he rallied and
racked his half-paralysed brains to make Latin verses as the best test of
his sanity.
628. Let it be noted, however, that in Johnson, as in most strong men,
there were certain leanings to the other side, certain evidences of the
“identity of contradictories.” His sense of mystery, his religiosity, his strong
passions, his tendency to violence in taste and opinions—were all rather
Romantic than Classical.
629. The Allegory on Criticism (daughter of Labour and Truth, who
gives up her task to Time, but is temporarily personated by Flattery and
Malevolence) in No. 3 almost speaks itself in the parenthetical description
just given. Cf. also 4, on Ancient and Modern Romances; 22, another
Allegory on Wit and Learning; 23, on the Contrariety of Criticism; and 36,
37, on “Pastoral Poetry.”
630. He was no doubt thinking also of Gilbert West, in his Life of whom
he introduces a caveat against West’s Imitations of Spenser as
“successful” indeed and “amusing” but “only pretty.”
631. “The warmest admirers of the great Mantuan poet can extol him
for little more than the skill with which he has ... united the beauties of the
Iliad and Odyssey,” and he adds a longish exposure of the way in which
Virgil, determined to imitate at all costs, has put in his borrowed matter
without regard to keeping.
632. The chief remaining critical loci in the Rambler are the unlucky
strictures in No. 168 on “dun,” “knife,” and “blanket” in Macbeth as “low”;
and the remarks on unfriendly criticism in 176.
633. There are, of course, other passages in the Idler touching on
Criticism,—59 on the Causes of Neglect of Books, 68, 69 on Translation,
77 on “Essay Writing,” 85 on Compilations. But they contain nothing of
exceptional importance.
 634. “Jonson, ... who besides that he had no imaginable temptation to
falsehood, wrote at a time when the character and acquisitions of
Shakespeare were known to multitudes. His evidence ought therefore to
decide the controversy, unless some testimony of equal force could be
opposed.”
635. With Johnson, as with some other writers, I have not thought it
necessary to specify editions. I must, however, mention Mr J. H. Millar’s
issue of the Lives (London, 1896) for the sake of the excellent
Introduction on Johnson’s criticism.
636. There are blind attempts at it even in antiquity; but Dryden’s Lives
of Lucian and Plutarch are, like other things of his elsewhere, the real
originals here.
637. Let me draw special attention to “John.” I once, unwittingly or
carelessly, called him “Thomas,” and I am afraid that I even neglected to
correct the error in a second edition of the guilty book. A man who writes
“Thomas” for “John,” in the case of a minor poet, can, I am aware,
possess no virtues, and must expect no pardon. But I shall always
henceforth remember to call him “Pomfret, Mr John.” “Let this expiate,” as
was remarked in another case of perhaps not less mortal sin.
638. It was of course probably suggested by Dryden (Essay on Satire,
“Donne ... affects the metaphysics”), but in Johnson’s hands is much
altered and extended.
639. It must be remembered that this word had no unfavourable
connotation with Johnson. It meant intelligent and scholarly interest.
640. Johnson’s relative estimates of the two (Boswell, Globe ed., pp.
186, 364) are well known; as is his apology for the Critical Reviewers’
habit [he had been one himself] of not reading the books through, as the
“duller” Monthly fellows were glad to do. Later generations have perhaps
contrived to be dull and not to read.
641. For instance, here is one which I have hunted for years—Essay
on the New Species of Writing founded by Fielding, with a word or two on
Modern Criticism (London? 1751). The better-known Canons of Criticism
of Thomas Edwards (4th ed., London, 1750) may serve as a specimen of
another kind. It is an attack on Warburton’s Shakespeare, uncommonly
shrewd in all senses of the word, but, as Johnson (Boswell, Globe ed., p.
87 note) justly enough said, of the gad-fly kind mainly. A curious little
book, which I do not remember to have seen cited anywhere, is the
Essay upon Poetry and Painting of Charles Lamotte (Dudlin (sic), 1742).
La Motte, who was an F.S.A., a D.D., and chaplain to the Duke of
Montagu, and who has the still rarer honour of not appearing in the Dict.
Nat. Biog., never, I think, refers to his namesake, but quotes Voltaire and
Du Bos frequently. He is very anxious for “propriety” in all senses, and
seems a little more interested in Painting than in Poetry. As to the latter,
he is a good example of the devouring appetite for sense and fact which
had seized on the critics of this time (save a few rebels) throughout
Europe. The improbabilities of Tasso and of “Camoenus, the Homer and
Virgil of the Portuguese,” afflict him more, because they amuse him less,
than they do in Voltaire’s own case, and to any liberty with real or
supposed history he is simply Rhadamanthine. “That which jars with
probability—that which shocks Sense and Reason—can never be
excused in Poetry.” Mrs Barbauld and The Ancient Mariner sixty years
before date!
642. Essays, xii.-xvii.
643. The Bee, viii.
644. Essay xviii.
645. It is perhaps only fair to hope that this fancy, as later with Southey
and others, was a blind motion for freedom. Yet Goldsmith commits
himself to the hemistich theory of decasyllables.
646. Essays, Moral and Literary, 2nd ed., London, 1774, 8vo.
647. This is perhaps the most delightful instance in (English) existence
of the change which has come over the meaning of the word.
648. Critical Essays, London, 1785, 8vo.
 CHAPTER II.

THE CONTEMPORARIES OF VOLTAIRE.

CLOSE CONNECTION OF FRENCH SEVENTEENTH AND EIGHTEENTH CENTURY
CRITICISM: FONTENELLE—EXCEPTIONAL CHARACTER OF HIS CRITICISM—
HIS ATTITUDE TO THE “ANCIENT AND MODERN” QUARREL—THE
‘DIALOGUES DES MORTS’—OTHER CRITICAL WORK—LA MOTTE—HIS
“UNITY OF INTEREST”—ROLLIN—BRUMOY—RÉMOND DE SAINT-MARD—L.
RACINE—DU BOS—STIMULATING BUT DESULTORY CHARACTER OF HIS
‘RÉFLEXIONS’—MONTESQUIEU—VOLTAIRE: DISAPPOINTMENT OF HIS
CRITICISM—EXAMPLES OF IT—CAUSES OF HIS FAILURE—OTHERS:
BUFFON—“STYLE AND THE MAN”—VAUVENARGUES—BATTEUX—HIS
ADJUSTMENT OF RULES AND TASTE—HIS INCOMPLETENESS—
MARMONTEL—ODDITIES AND QUALITIES OF HIS CRITICISM—OTHERS:
THOMAS, SUARD, ETC.—LA HARPE—HIS ‘COURS DE LITTÉRATURE’—HIS
CRITICAL POSITION AS “ULTIMUS SUORUM”—THE ACADEMIC ESSAY—
RIVAROL.

The later seventeenth and at least the earlier eighteenth century in

France are perhaps more closely connected than any other literary
periods, if, indeed, they are not practically one, like the two halves of our
Close connection of own so-called “Elizabethan” time. And this
French seventeenth and connection we can duly demonstrate, as far as
eighteenth century criticism is concerned. Boileau himself outlived
criticism. Fontenelle.
the junction of the centuries by more than a
decade: and the birth of Voltaire preceded it by more than a lustrum. The
Quarrel of Ancients and Moderns—a very poor thing certainly—revived in
the new century, as if on purpose to show the connection with the old.
And, lastly, the prolonged life of one remarkable and representative critic
was almost equally distributed over the two. Fontenelle is one of the most
interesting, if not exactly one of the most important, figures in our whole
long gallery; and if he has never yet held quite his proper place in literary
history, this is due to the facts, first, that he was a critic more than he was
anything else; and, secondly, that he forgot the great “Thou shalt not”
which Criticism lays upon her sons, and would lay (if she had any) on her
daughters. No critic is in the least bound to produce good work, or any
work, of the constructive kind: but he is bound not to produce that which
is not good. The author of Aspar and the Lettres du Chevalier d’Her ...
forgot this, and paid the penalty.[649]
 Yet his attractions are so great that few people who have paid him
much attention have failed to be smitten with them. M. Rigault,[650] who
does not approve of him generally, is a conspicuous example of this. But
Exceptional what we must look to is what he has actually written
character of himself. His utterances are almost too tempting. In such
his criticism. a book as this the expatiation which they invite must be
perforce denied them. Yet one may break proportion a little in order to do
something like justice to a critic whose like, for suggestiveness, delicacy,
and range, we shall hardly meet in the French eighteenth century. It is
indeed curious that of the three men of his own earliest years from whom
Voltaire inherits—Saint-Evremond, Hamilton, and Fontenelle—every one
should have surpassed him in the finer traits, while all fall short of him in
force and, as he himself said, diable au corps. Saint-Evremond we have
dealt with; Hamilton[651] does not come into our story. Fontenelle is for the
moment ours.
It must be confessed that he is an elusive if an agreeable possession.
From wisdom, from worldly-wisdom, from whim, or from what not, he
seems to have wished to be an enigma; and—to borrow one of Scott’s
great sentences—“the wish of his heart was granted to his loss, and the
hope of his pride has destroyed him”—at least has certainly made him
rank lower than he would otherwise have ranked. However délié—to use
a word of his own language for which we have no single English
equivalent—however watchful, mercurial, sensitive the reader’s spirit may
be, he will, over and over again in Fontenelle, meet passages where he
cannot be sure whether his author is writing merely with tongue in cheek,
or applying an all-dissolving irony, hardly inferior to Swift’s in power, and
almost superior in quietness and subtlety. Moreover, his critical position is
a very peculiar one, and constantly liable to be misunderstood—if,
indeed, it be not safer to say that it is almost always difficult to apprehend
with any certainty of escaping misprision. The good folk who magisterially
rebuke Dryden as to Gorboduc, because he made mistakes about the
form of the verse and the sex of the person—even those (one regrets to
say this includes M. Rigault himself) who are shocked at that great critic’s
laudatory citations of, and allusions to, Le Bossu—need never hope to
understand Fontenelle.
Few things (except that he was the author of that Plurality of Worlds
which happily does not concern us) are better known concerning him than
His attitude to that he was a champion of the Moderns. Yet, when we
the “Ancient come to examine his numerous and elusive writings on
the subject, the one principle of his that does emerge is a
and Modern” principle which, if it chastises the Ancients with whips,
Quarrel. chastises the Moderns with scorpions. A man writing, as
M. Rigault wrote, in 1856, would have been a wonderful person if he had
not been misled by the great idol of Progress. But Fontenelle was at least
as far from the delusion as he was from the date. His argument is just the
contrary—that as human wisdoms and human follies, human powers and
human weaknesses, are always the same, it is absurd to suppose that
any one period can have general and intrinsic superiority over any other.
The Dialogues Assuredly no “modern,” whether of his days or of our
des Morts. own, can find aught but confusion of face in the quiet
axiom of Laura at the end of her controversy with Sappho,[652] “Croyez
moi, après qu’on a bien raisonné ou sur l’amour, ou sur telle autre
matière qu’on voudra, on trouve au bout du compte que les choses sont
bien comme elles sont, et que la réforme qu’on prétendroit y apporter
gâterait tout.” Pulveris exigui jactus! but one with a fatally magical effect
in the quarrels of criticism as of other things. And the same is the lesson
of the dialogue which follows immediately—the best of the whole, and
almost a sovereign document of our library,—that between Socrates and
Montaigne. Not only is there no example in the literature of the dialogue,
from Plato to Mr Traill, much more apt than the “maieutic” feat of
Socrates, by which he induces Montaigne to commit himself to the
dogma, “Partout où il y a des hommes, il y a des sottises, et les mêmes
sottises”; but the rest of the piece is as powerfully, though as quietly,
worked out as this crisis of it. There is no Progress; there is no
Degeneration. The distribution may vary: the sum will not. Erasistratus
maintains the same thesis on a different matter a little later in his dialogue
with Harvey,[653] laying down the doctrine, outrageous to all the Royal
Societies of the world (though they were glad to welcome Fontenelle as
populariser, and have perhaps never had such an one since, except Mr
Huxley), that “the things which are not necessary perhaps do get
discovered in the course of ages, the others not.” And Charles V.
preaches no very different sermon when he “makes a hare” of Erasmus
by pointing out to that dilettante republican that les biens de l’esprit are
just as much things of time and chance as crown and sceptre.[654]
It is, however, in Fontenelle’s actual concrete deliverances of criticism
Other critical that the resemblance to Dryden comes in most. Those
work. who insist that such deliverances shall be Medic-Persian,
unalterable, mathematical, true without relation and adjustment, will not
like him. To take his utterances down in a notebook, and reproduce them
at the next examination (to provide for which process seems to be held
the be-all and end-all of modern criticism), would not do at all. When
Fontenelle praises Corneille at the expense of Racine, you have to think
whether he is speaking what he thinks or merely as le neveu de son
oncle; when he says other things, whether he is a “Modern” at the time
and to the extent of saying something which he knows will cause the
“Ancients” grinding torments; when he sketches[655] a theory of poetic
criticism of the most sweeping a priori kind from Principles of Beauty
down through Kinds to Rules, whether he really means this, or is
conciliating somebody, or laughing in his sleeve at somebody, or the like.
But this—at least for some tastes—only adds piquancy to his
observations, and they have now and then surprising justice, freshness,
freedom from the prejudices of time, country, and circumstance. The
Histoire du Théâtre Français, for instance, which he has prefixed to his
Vie de Corneille, may be based on second-hand information, and, with
our fuller knowledge, it may not be very hard to pick holes in it. But it is an
extraordinary production for a representative man of letters at a time
when hardly any such man, in any country of Europe, was free from
ignorant contempt of the early vernaculars. The brief eleven-articled
“parallel between Racine and Corneille” is of course somewhat partisan;
but it will give the partisans on the other side some trouble to prove it
unjust. The “Remarks on Aristophanes,” and on the Greek theatre
generally, are obviously “modern” and intended to tease; but they are
uncommonly shrewd, and so are the Réflexions sur la Poétique and
those on “Poetry in General.” It is wonderful that even an antagonist of
Boileau, and a sworn paradoxer, should, at this time, have been able to
see the beauty of the Père Le Moyne’s splendid couplet on the Sicilian
Vespers,—

“Quand du Gibel ardent les noires Euménides

Sonneront de leur cor ces Vêpres homicides,”—

where we are more than half-way from Du Bartas and Aubigné to Victor
Hugo. The mere image—this new “vision of the guarded mount,” with the
black Furies silhouetted against the flaming cone, and the explosions of
the volcano deepening the bugle-call to massacre—is fine: the means
taken to make it poetical are finer. The use of the proper names, and the
cunning arrangement of epithet and noun in noires Euménides and
Vêpres homicides, and the sharp blasts of the long and short o's in the
second line, are more than Hugonian, they are positively Miltonic: and the
couplet will serve to keep a man in Mr Arnold’s “torpid and dismal” stage
of later middle life cheerful for an evening, and whensoever he
remembers it afterwards. True, Fontenelle admits demurely that he knows
“vespers” and “Eumenides” are something of an anachronism in
conjunction, and proposes a slight alteration to suit this objection of
“correctness.” But this is his way; and the wonderful thing is that he
should have admired it at all—should have actually tasted this heady
wine of poetry. As he finishes the paragraph in his own quaint style,[656] “Il
était bien aisé, même à de grands poètes, de ne pas trouver” this couplet:
and in his time it would have been still easier even for great critics not to
do justice to it, and not to see that it is to these things “so easy for the
poet not to find” that it is the critic’s business to look.
The general remarks on Comedy which he prefixed to a collection of
his efforts in that kind are not negligible; but in those on Eclogue,[657] and
still more in the Digression sur Les Anciens et Les Modernes, the curse,
or at least the gainsaying, of the Quarrel is upon him, and the main drift is
not merely digressive but aggressive and excessive. In the Digression he
anticipates (as he did in so many things) the materialist-rationalist
explanations of the later eighteenth century by climate, fibres of the brain,
&c. Here he becomes scientific, and therefore necessarily ceases to be of
importance in literature.
But he always regains that importance before long—in his Discourse of
the Origin of Fable, in his Academic Discourses and Replies, in many a
fragment and isolated remark. Even in his Eloges—mostly devoted (there
are nearly two volumes of them) to scientific personages from Leibniz and
Newton downwards—the unconquerable critical power of the man shows
itself, subject to the limitations noted. The world is sometimes not allowed
to know anything of its greatest critics, and Fontenelle is an example of
this. But those who have won something of that knowledge of criticism
which it is the humble purpose of this book to facilitate, will not slight the
man who, at the junction of the seventeenth and eighteenth centuries,
could flirt in the face of Ancients and Moderns alike the suggestion (which
Mr Rigmarole doubtless borrowed from him) that all times are “pretty
much like our own,” and could see and hear the sable sisters sounding
the tocsin on the flaming crest of Mongibel.
Fontenelle is elusive, but comprehensible by the imagination. La Motte,
[658]
his inseparable companion in the renewed sacrilege of the Moderns,
seems an easier, but is really a harder, personage to lay hold of. It is
La Motte. indeed not extremely difficult to explain his attitude to the
Ancients by the fact that he knew no Greek; and his
exaltation of prose by a consciousness (wherein he has left a family by no
means extinct) that his own verses were worth very little. But it is so easy
not to write verses if you cannot; and not to write about Greek if you do
not know it! And the problem is further complicated by the facts that at
least some judges, who are not exactly the first comers, such as
Fontenelle himself and Voltaire, maintained that La Motte could write
verses,—and that, so far from being “a fellow who had failed,” he had
obtained the greatest scenic success of the early eighteenth century with
Inès de Castro, and, what is more, had deserved it. But for once, as also
again in Pope’s case, the dangerous explanation of physical defects and
constitutional weakness seems to have some validity. The invulnerable
nonchalance of his friend Fontenelle had met the damnation of Aspar by
a cool tearing up of the piece, and an undismayed advance upon the fate
of the plusquam semel damnatus; La Motte, at twenty or at little more, felt
the similar misfortune of Les Originaux so severely that he actually went
to La Trappe for a time. Before middle life he was blind and a cripple. The
irritability which did not show itself in his temper (for he was the most
amiable of men) would seem to have transferred itself to his literary
attitude, not affecting his politeness of expression, but inducing a sort of
“rash” of paradox.
To trace the vagaries of this might not be unamusing, but would
His “Unity of certainly be excessive here. La Motte, it seems to me,
Interest.” had considerably less natural literary taste than
Fontenelle; and of the controversy[659] (it was not his antagonist’s fault if it
was not a very acrimonious one) between him and Madame Dacier one
cannot say much more than that the lady is very aggressive, very erudite,
and very unintelligent; the gentleman very suave, rather ignorant, and of
an intelligence better, but not much better, directed; while both are
sufficiently distant from any true critical point of view. Yet once, as was
not unnatural in the case of a very clever man who was at least
endeavouring to form independent conclusions, La Motte did hit upon a
great critical truth when,[660] discussing the Three Unities, he laid it down
that there is after all only one Unity which is of real importance, and that
this is the “Unity of Interest,” to which all the others are subsidiary, and
but as means to an end. “Self-evident,” some one may say; but in how
many critics have we found the fact acknowledged hitherto? and by how
many has it been frankly acknowledged since? That the aim of the poet is
to please, to satisfy the thirst for pleasure—that is to say, to interest—all
but the extremest ethical prudery will admit. But critics, especially
classical and neo-classical critics, have always been in the mood of