Download the full version of the ebook now at ebooksecure.

com

Introduction to Algorithms for Data Mining

and Machine Learning 1st edition - eBook PDF

https://ebooksecure.com/download/introduction-to-
algorithms-for-data-mining-and-machine-learning-
ebook-pdf/

Explore and download more ebook at https://ebooksecure.com

 Recommended digital products (PDF, EPUB, MOBI) that
you can download immediately if you are interested.

Machine Learning for Biometrics: Concepts, Algorithms and

Applications (Cognitive Data Science in Sustainable
Computing) 1st Edition - eBook PDF
https://ebooksecure.com/download/machine-learning-for-biometrics-
concepts-algorithms-and-applications-cognitive-data-science-in-
sustainable-computing-ebook-pdf/
ebooksecure.com

BIG DATA ANALYTICS: Introduction to Hadoop, Spark, and

Machine-Learning 1st Edition - eBook PDF

https://ebooksecure.com/download/big-data-analytics-introduction-to-
hadoop-spark-and-machine-learning-ebook-pdf/

ebooksecure.com

(eBook PDF) Introduction to Machine Learning with Python:

A Guide for Data Scientists

http://ebooksecure.com/product/ebook-pdf-introduction-to-machine-
learning-with-python-a-guide-for-data-scientists/

ebooksecure.com

(eBook PDF) Introduction to Business Data Mining 1st

Edition

http://ebooksecure.com/product/ebook-pdf-introduction-to-business-
data-mining-1st-edition/

ebooksecure.com
(eBook PDF) Machine Learning Refined: Foundations,
Algorithms, and Applications

http://ebooksecure.com/product/ebook-pdf-machine-learning-refined-
foundations-algorithms-and-applications/

ebooksecure.com

(eBook PDF) Introduction to Data Mining, Global Edition

2nd Edition

http://ebooksecure.com/product/ebook-pdf-introduction-to-data-mining-
global-edition-2nd-edition/

ebooksecure.com

Big Data Mining for Climate Change 1st edition - eBook PDF

https://ebooksecure.com/download/big-data-mining-for-climate-change-
ebook-pdf/

ebooksecure.com

(eBook PDF) Introduction to Data Mining 2nd Edition by

Pang-Ning Tan

http://ebooksecure.com/product/ebook-pdf-introduction-to-data-
mining-2nd-edition-by-pang-ning-tan/

ebooksecure.com

Machine Learning for Planetary Science 1st Edition - eBook

PDF

https://ebooksecure.com/download/machine-learning-for-planetary-
science-ebook-pdf/

ebooksecure.com
 Xin-She Yang

Introduction to
Algorithms for Data Mining
and Machine Learning
Introduction to Algorithms for Data Mining and
Machine Learning
This page intentionally left blank
Introduction to
Algorithms for Data
Mining and Machine
Learning

Xin-She Yang
Middlesex University
School of Science and Technology
London, United Kingdom
Academic Press is an imprint of Elsevier
125 London Wall, London EC2Y 5AS, United Kingdom
525 B Street, Suite 1650, San Diego, CA 92101, United States
50 Hampshire Street, 5th Floor, Cambridge, MA 02139, United States
The Boulevard, Langford Lane, Kidlington, Oxford OX5 1GB, United Kingdom
Copyright © 2019 Elsevier Inc. All rights reserved.
No part of this publication may be reproduced or transmitted in any form or by any means, electronic or
mechanical, including photocopying, recording, or any information storage and retrieval system, without
permission in writing from the publisher. Details on how to seek permission, further information about the
Publisher’s permissions policies and our arrangements with organizations such as the Copyright Clearance Center
and the Copyright Licensing Agency, can be found at our website: www.elsevier.com/permissions.
This book and the individual contributions contained in it are protected under copyright by the Publisher (other
than as may be noted herein).
Notices
Knowledge and best practice in this field are constantly changing. As new research and experience broaden our
understanding, changes in research methods, professional practices, or medical treatment may become necessary.
Practitioners and researchers must always rely on their own experience and knowledge in evaluating and using
any information, methods, compounds, or experiments described herein. In using such information or methods
they should be mindful of their own safety and the safety of others, including parties for whom they have a
professional responsibility.
To the fullest extent of the law, neither the Publisher nor the authors, contributors, or editors, assume any liability
for any injury and/or damage to persons or property as a matter of products liability, negligence or otherwise, or
from any use or operation of any methods, products, instructions, or ideas contained in the material herein.

Library of Congress Cataloging-in-Publication Data

A catalog record for this book is available from the Library of Congress

British Library Cataloguing-in-Publication Data

A catalogue record for this book is available from the British Library

ISBN: 978-0-12-817216-2

For information on all Academic Press publications

visit our website at https://www.elsevier.com/books-and-journals

Publisher: Candice Janco

Acquisition Editor: J. Scott Bentley
Editorial Project Manager: Michael Lutz
Production Project Manager: Nilesh Kumar Shah
Designer: Miles Hitchen
Typeset by VTeX
Contents

About the author ix

Preface xi
Acknowledgments xiii

1 Introduction to optimization 1
1.1 Algorithms 1
1.1.1 Essence of an algorithm 1
1.1.2 Issues with algorithms 3
1.1.3 Types of algorithms 3
1.2 Optimization 4
1.2.1 A simple example 4
1.2.2 General formulation of optimization 7
1.2.3 Feasible solution 9
1.2.4 Optimality criteria 10
1.3 Unconstrained optimization 10
1.3.1 Univariate functions 11
1.3.2 Multivariate functions 12
1.4 Nonlinear constrained optimization 14
1.4.1 Penalty method 15
1.4.2 Lagrange multipliers 16
1.4.3 Karush–Kuhn–Tucker conditions 17
1.5 Notes on software 18

2 Mathematical foundations 19
2.1 Convexity 20
2.1.1 Linear and affine functions 20
2.1.2 Convex functions 21
2.1.3 Mathematical operations on convex functions 22
2.2 Computational complexity 22
2.2.1 Time and space complexity 24
2.2.2 Complexity of algorithms 25
2.3 Norms and regularization 26
2.3.1 Norms 26
2.3.2 Regularization 28
2.4 Probability distributions 29
2.4.1 Random variables 29
2.4.2 Probability distributions 30
vi Contents

2.4.3 Conditional probability and Bayesian rule 32

2.4.4 Gaussian process 34
2.5 Bayesian network and Markov models 35
2.6 Monte Carlo sampling 36
2.6.1 Markov chain Monte Carlo 37
2.6.2 Metropolis–Hastings algorithm 37
2.6.3 Gibbs sampler 39
2.7 Entropy, cross entropy, and KL divergence 39
2.7.1 Entropy and cross entropy 39
2.7.2 DL divergence 40
2.8 Fuzzy rules 41
2.9 Data mining and machine learning 42
2.9.1 Data mining 42
2.9.2 Machine learning 42
2.10 Notes on software 42

3 Optimization algorithms 45
3.1 Gradient-based methods 45
3.1.1 Newton’s method 45
3.1.2 Newton’s method for multivariate functions 47
3.1.3 Line search 48
3.2 Variants of gradient-based methods 49
3.2.1 Stochastic gradient descent 50
3.2.2 Subgradient method 51
3.2.3 Conjugate gradient method 52
3.3 Optimizers in deep learning 53
3.4 Gradient-free methods 56
3.5 Evolutionary algorithms and swarm intelligence 58
3.5.1 Genetic algorithm 58
3.5.2 Differential evolution 60
3.5.3 Particle swarm optimization 61
3.5.4 Bat algorithm 61
3.5.5 Firefly algorithm 62
3.5.6 Cuckoo search 62
3.5.7 Flower pollination algorithm 63
3.6 Notes on software 64

4 Data fitting and regression 67

4.1 Sample mean and variance 67
4.2 Regression analysis 69
4.2.1 Maximum likelihood 69
4.2.2 Liner regression 70
4.2.3 Linearization 75
4.2.4 Generalized linear regression 77
4.2.5 Goodness of fit 80
Contents vii

4.3 Nonlinear least squares 81

4.3.1 Gauss–Newton algorithm 82
4.3.2 Levenberg–Marquardt algorithm 85
4.3.3 Weighted least squares 85
4.4 Overfitting and information criteria 86
4.5 Regularization and Lasso method 88
4.6 Notes on software 90

5 Logistic regression, PCA, LDA, and ICA 91

5.1 Logistic regression 91
5.2 Softmax regression 96
5.3 Principal component analysis 96
5.4 Linear discriminant analysis 101
5.5 Singular value decomposition 104
5.6 Independent component analysis 105
5.7 Notes on software 108

6 Data mining techniques 109

6.1 Introduction 110
6.1.1 Types of data 110
6.1.2 Distance metric 110
6.2 Hierarchy clustering 111
6.3 k-Nearest-neighbor algorithm 112
6.4 k-Means algorithm 113
6.5 Decision trees and random forests 115
6.5.1 Decision tree algorithm 115
6.5.2 ID3 algorithm and C4.5 classifier 116
6.5.3 Random forest 120
6.6 Bayesian classifiers 121
6.6.1 Naive Bayesian classifier 121
6.6.2 Bayesian networks 123
6.7 Data mining for big data 124
6.7.1 Characteristics of big data 124
6.7.2 Statistical nature of big data 125
6.7.3 Mining big data 125
6.8 Notes on software 127

7 Support vector machine and regression 129

7.1 Statistical learning theory 129
7.2 Linear support vector machine 130
7.3 Kernel functions and nonlinear SVM 133
7.4 Support vector regression 135
7.5 Notes on software 137
viii Contents

8 Neural networks and deep learning 139

8.1 Learning 139
8.2 Artificial neural networks 140
8.2.1 Neuron models 140
8.2.2 Activation models 141
8.2.3 Artificial neural networks 143
8.3 Back propagation algorithm 146
8.4 Loss functions in ANN 147
8.5 Optimizers and choice of optimizers 149
8.6 Network architecture 149
8.7 Deep learning 151
8.7.1 Convolutional neural networks 151
8.7.2 Restricted Boltzmann machine 157
8.7.3 Deep neural nets 158
8.7.4 Trends in deep learning 159
8.8 Tuning of hyperparameters 160
8.9 Notes on software 161

Bibliography 163

Index 171
About the author

Xin-She Yang obtained his PhD in Applied Mathematics from the University of Ox-
ford. He then worked at Cambridge University and National Physical Laboratory (UK)
as a Senior Research Scientist. Now he is Reader at Middlesex University London, and
an elected Bye-Fellow at Cambridge University.
He is also the IEEE Computer Intelligence Society (CIS) Chair for the Task Force
on Business Intelligence and Knowledge Management, Director of the International
Consortium for Optimization and Modelling in Science and Industry (iCOMSI), and
an Editor of Springer’s Book Series Springer Tracts in Nature-Inspired Computing
(STNIC).
With more than 20 years of research and teaching experience, he has authored
10 books and edited more than 15 books. He published more than 200 research pa-
pers in international peer-reviewed journals and conference proceedings with more
than 36 800 citations. He has been on the prestigious lists of Clarivate Analytics and
Web of Science highly cited researchers in 2016, 2017, and 2018. He serves on the
Editorial Boards of many international journals including International Journal of
Bio-Inspired Computation, Elsevier’s Journal of Computational Science (JoCS), In-
ternational Journal of Parallel, Emergent and Distributed Systems, and International
Journal of Computer Mathematics. He is also the Editor-in-Chief of the International
Journal of Mathematical Modelling and Numerical Optimisation.
Visit https://ebooksecure.com
now to explore a rich
collection of ebook and enjoy
exciting offers!
This page intentionally left blank
Preface

Both data mining and machine learning are becoming popular subjects for university
courses and industrial applications. This popularity is partly driven by the Internet and
social media because they generate a huge amount of data every day, and the under-
standing of such big data requires sophisticated data mining techniques. In addition,
many applications such as facial recognition and robotics have extensively used ma-
chine learning algorithms, leading to the increasing popularity of artificial intelligence.
From a more general perspective, both data mining and machine learning are closely
related to optimization. After all, in many applications, we have to minimize costs,
errors, energy consumption, and environment impact and to maximize sustainabil-
ity, productivity, and efficiency. Many problems in data mining and machine learning
are usually formulated as optimization problems so that they can be solved by opti-
mization algorithms. Therefore, optimization techniques are closely related to many
techniques in data mining and machine learning.
Courses on data mining, machine learning, and optimization are often compulsory
for students, studying computer science, management science, engineering design, op-
erations research, data science, finance, and economics. All students have to develop
a certain level of data modeling skills so that they can process and interpret data for
classification, clustering, curve-fitting, and predictions. They should also be familiar
with machine learning techniques that are closely related to data mining so as to carry
out problem solving in many real-world applications. This book provides an introduc-
tion to all the major topics for such courses, covering the essential ideas of all key
algorithms and techniques for data mining, machine learning, and optimization.
Though there are over a dozen good books on such topics, most of these books are
either too specialized with specific readership or too lengthy (often over 500 pages).
This book fills in the gap with a compact and concise approach by focusing on the key
concepts, algorithms, and techniques at an introductory level. The main approach of
this book is informal, theorem-free, and practical. By using an informal approach all
fundamental topics required for data mining and machine learning are covered, and
the readers can gain such basic knowledge of all important algorithms with a focus
on their key ideas, without worrying about any tedious, rigorous mathematical proofs.
In addition, the practical approach provides about 30 worked examples in this book
so that the readers can see how each step of the algorithms and techniques works.
Thus, the readers can build their understanding and confidence gradually and in a
step-by-step manner. Furthermore, with the minimal requirements of basic high school
mathematics and some basic calculus, such an informal and practical style can also
enable the readers to learn the contents by self-study and at their own pace.
This book is suitable for undergraduates and graduates to rapidly develop all the
fundamental knowledge of data mining, machine learning, and optimization. It can
xii Preface

also be used by students and researchers as a reference to review and refresh their
knowledge in data mining, machine learning, optimization, computer science, and data
science.

Xin-She Yang
January 2019 in London
Acknowledgments

I would like to thank all my students and colleagues who have given valuable feedback
and comments on some of the contents and examples of this book. I also would like to
thank my editors, J. Scott Bentley and Michael Lutz, and the staff at Elsevier for their
professionalism. Last but not least, I thank my family for all the help and support.

Xin-She Yang
January 2019
This page intentionally left blank
Introduction to optimization
Contents
1.1 Algorithms
1 1
1.1.1 Essence of an algorithm 1
1.1.2 Issues with algorithms 3
1.1.3 Types of algorithms 3
1.2 Optimization 4
1.2.1 A simple example 4
1.2.2 General formulation of optimization 7
1.2.3 Feasible solution 9
1.2.4 Optimality criteria 10
1.3 Unconstrained optimization 10
1.3.1 Univariate functions 11
1.3.2 Multivariate functions 12
1.4 Nonlinear constrained optimization 14
1.4.1 Penalty method 15
1.4.2 Lagrange multipliers 16
1.4.3 Karush–Kuhn–Tucker conditions 17
1.5 Notes on software 18

This book introduces the most fundamentals and algorithms related to optimization,
data mining, and machine learning. The main requirement is some understanding of
high-school mathematics and basic calculus; however, we will review and introduce
some of the mathematical foundations in the first two chapters.

1.1 Algorithms
An algorithm is an iterative, step-by-step procedure for computation. The detailed
procedure can be a simple description, an equation, or a series of descriptions in
combination with equations. Finding the roots of a polynomial, checking if a natu-
ral number is a prime number, and generating random numbers are all algorithms.

1.1.1 Essence of an algorithm

In essence, an algorithm can be written as an iterative equation or a set of iterative
equations. For example, to find a square root of a > 0, we can use the following
iterative equation:
1 a
xk+1 = xk + , (1.1)
2 xk
where k is the iteration counter (k = 0, 1, 2, . . . ) starting with a random guess x0 = 1.
Introduction to Algorithms for Data Mining and Machine Learning. https://doi.org/10.1016/B978-0-12-817216-2.00008-9
Copyright © 2019 Elsevier Inc. All rights reserved.
2 Introduction to Algorithms for Data Mining and Machine Learning

Example 1
As an example, if x0 = 1 and a = 4, then we have

1 4
x1 = (1 + ) = 2.5. (1.2)
2 1

Similarly, we have

1 4 1 4
x2 = (2.5 + ) = 2.05, x3 = (2.05 + ) ≈ 2.0061, (1.3)
2 2.5 2 2.05
x4 ≈ 2.00000927, (1.4)
√
which is very close to the true value of 4 = 2. The accuracy of this iterative formula or algorithm
is high because it achieves the accuracy of five decimal places after four iterations.

The convergence is very quick if we start from different initial values such as
x0 = 10 and even x0 = 100. However, for an obvious reason, we cannot start with
x0 = 0 due to division by
√zero.
Find the root of x = a is equivalent to solving the equation

f (x) = x 2 − a = 0, (1.5)

which is again equivalent to finding the roots of a polynomial f (x). We know that
Newton’s root-finding algorithm can be written as

f (xk )
xk+1 = xk − , (1.6)
f  (xk )

where f  (x) is the first derivative or gradient of f (x). In this case, we have
f  (x) = 2x. Thus, Newton’s formula becomes

(xk2 − a)
xk+1 = xk − , (1.7)
2xk

which can be written as

xk a 1 a
xk+1 = (xk − )+ = xk + ). (1.8)
2 2xk 2 xk

This is exactly what we have in Eq. (1.1).

Newton’s method has rigorous mathematical foundations, which has a guaranteed
convergence under certain conditions. However, in general, Eq. (1.6) is more general,
and the gradient information f  (x) is needed. In addition, for the formula to be valid,
we must have f  (x) = 0.
Introduction to optimization 3

1.1.2 Issues with algorithms

The advantage of the algorithm given in Eq. (1.1) is that√it converges very quickly.
However, careful readers may have asked: we know that 4 = ±2, how can we find
the other root −2 in addition to +2?
Even if we use different initial value x0 = 10 or x0 = 0.5, we can only reach x∗ = 2,
not −2.
What happens if we start with x0 < 0? From x0 = −1, we have
1 4 1 4
x1 = (−1 + ) = −2.5, x 2 = (−2.5 + ) = −2.05, (1.9)
2 −1 2 −2.5
x3 ≈ −2.0061, x4 ≈ −2.00000927, (1.10)
which is approaching −2 very quickly. If we start from x0 = −10 or x0 = −0.5, then
we can always get x∗ = −2, not +2.
This highlights a key issue here: the final solution seems to depend on the initial
starting point for this algorithm, which is true for many algorithms.
Now the relevant question is: how do we know where to start to get a particular
solution? The general short answer is “we do not know”. Thus, some knowledge of
the problem under consideration or an educated guess may be useful to find the final
solution.
In fact, most algorithms may depend on the initial configuration, and such algo-
rithms are often carrying out search moves locally. Thus, this type of algorithm is
often referred to as local search. A good algorithm should be able to “forget” its initial
configuration though such algorithms may not exist at all for most types of problems.
What we need in general is the global search, which attempts to find final solutions
that are less sensitive to the initial starting point(s).
Another important issue in our discussions is that the gradient information f  (x) is
necessary for some algorithms such as Newton’s method given in Eq. (1.6). This poses
certain requirements on the smoothness of the function f (x). For example, we know
that |x| is not differentiable at x = 0. Thus, we cannot directly use Newton’s method
to find the roots of f (x) = |x|x 2 − a = 0 for a > 0. Some modifications are needed.
There are other issues related to algorithms such as the setting of parameters, the
slow rate of convergence, condition numbers, and iteration structures. All these make
algorithm designs and usage somehow challenging, and we will discuss these issues
in more detail later in this book.

1.1.3 Types of algorithms

An algorithm can only do a specific computation task (at most a class of computational
tasks), and no algorithms can do all the tasks. Thus, algorithms can be classified due
to their purposes. An algorithm to find roots of a polynomial belongs to root-finding
algorithms, whereas an algorithm for ranking a set of numbers belongs to sorting
algorithms. There are many classes of algorithms for different purposes. Even for the
same purpose such as sorting, there are many different algorithms such as the merge
sort, bubble sort, quicksort, and others.
4 Introduction to Algorithms for Data Mining and Machine Learning

We can also categorize algorithms in terms of their characteristics. The root-finding

algorithms we just introduced are deterministic algorithms because the final solutions
are exactly the same if we start from the same initial guess. We obtain the same set of
solutions every time we run the algorithm. On the other hand, we may introduce some
randomization into the algorithm, for example, using purely random initial points.
Every time we run the algorithm, we use a new random initial guess. In this case, the
algorithm can have some nondeterministic nature, and such algorithms are referred
to as stochastic.√Sometimes, using randomness may be advantageous. For example, in
the example of 4 = ±2 using Eq. (1.1), random initial values (both positive and neg-
ative) can allow the algorithm to find both roots. In fact, a major trend in the modern
metaheuristics is using some randomization to suit different purposes.
For algorithms to be introduced in this book, we are mainly concerned with al-
gorithms for data mining, optimization, and machine learning. We use a relatively
unified approach to link algorithms in data mining and machine learning to algorithms
for optimization.

1.2 Optimization

Optimization is everywhere, from engineering design to business planning. After all,

time and resources are limited, and optimal use of such valuable resources is crucial.
In addition, designs of products have to maximize the performance, sustainability, and
energy efficiency and to minimize the costs. Therefore, optimization is important for
many applications.

1.2.1 A simple example

Let us start with a very simple example to design a container with volume capacity
V0 = 10 m3 . As the main cost is related to the cost of materials, the main aim is to
minimize the total surface area S.
The first thing we have to decide is the shape of the container (cylinder, cubic,
sphere or ellipsoid, or more complex geometry). For simplicity, let us start with a
cylindrical shape with radius r and height h (see Fig. 1.1).
The total surface area of a cylinder is

S = 2(πr 2 ) + 2πrh, (1.11)

and the volume is

V = πr 2 h. (1.12)

There are only two design variables r and h and one objective function S to be min-
imized. Obviously, if there is no capacity constraint, then we can choose not to build
the container, and then the cost of materials is zero for r = 0 and h = 0. However,
Introduction to optimization 5

Figure 1.1 Design of a cylindric container.

the constraint requirement means that we have to build a container with fixed volume
V0 = πr 2 h = 10 m3 . Therefore, this optimization problem can be written as

minimize S = 2πr 2 + 2πrh, (1.13)

subject to the equality constraint

πr 2 h = V0 = 10. (1.14)

To solve this problem, we can first try to use the equality constraint to reduce the
number of design variables by solving h. So we have
V0
h= . (1.15)
πr 2
Substituting it into (1.13), we get

S = 2πr 2 + 2πrh
V0 2V0
= 2πr 2 + 2πr 2 = 2πr 2 + . (1.16)
πr r
This is a univariate function. From basic calculus we know that the minimum or max-
imum can occur at the stationary point, where the first derivative is zero, that is,
dS 2V0
= 4πr − 2 = 0, (1.17)
dr r
which gives

V0 3 V0
r3 = , or r = . (1.18)
2π 2π
Thus, the height is

h V0 /(πr 2 ) V0
= = 3 = 2. (1.19)
r r πr
Visit https://ebooksecure.com
now to explore a rich
collection of ebook and enjoy
exciting offers!
6 Introduction to Algorithms for Data Mining and Machine Learning

This means that the height is twice the radius: h = 2r. Thus, the minimum surface is

S∗ = 2πr 2 + 2πrh = 2πr 2 + 2πr(2r) = 6πr 2

 V 2/3 6π
0 2/3
= 6π =√3
V0 . (1.20)
2π 4π 2

For V0 = 10, we have

 
3 V0 3 10
r= = ≈ 1.1675, h = 2r = 2.335,
(2π) 2π

and the total surface area

S∗ = 2πr 2 + 2πrh ≈ 25.69.

It is worth pointing out that this optimal solution is based on the assumption or re-
quirement to design a cylindrical container. If we decide to use a sphere with radius R,
we know that its volume and surface area is
4π 3
V0 = R , S = 4πR 2 . (1.21)
3
We can solve R directly

3V0 3 3V0
R =
3
, or R = , (1.22)
4π 4π
which gives the surface area
 3V 2/3 √
0 4π 3 9 2/3
S = 4π =√ 3
V0 . (1.23)
4π 16π 2
√3 √ √ 3
Since 6π/ 4π 2 ≈ 5.5358 and 4π 3 9/ 16π 2 ≈ 4.83598, we have S < S∗ , that is, the
surface area of a sphere is smaller than the minimum surface area of a cylinder with
the same volume. In fact, for the same V0 = 10, we have
√
4π 3 9 2/3
S(sphere) = √ 3
V0 ≈ 22.47, (1.24)
16π 2
which is smaller than S∗ = 25.69 for a cylinder.
This highlights the importance of the choice of design type (here in terms of shape)
before we can do any truly useful optimization. Obviously, there are many other fac-
tors that can influence the choice of design, including the manufacturability of the
design, stability of the structure, ease of installation, space availability, and so on. For
a container, in most applications, a cylinder may be much easier to produce than a
sphere, and thus the overall cost may be lower in practice. Though there are so many
factors to be considered in engineering design, for the purpose of optimization, here
we will only focus on the improvement and optimization of a design with well-posed
mathematical formulations.
Introduction to optimization 7

1.2.2 General formulation of optimization

Whatever the real-world applications may be, it is usually possible to formulate an
optimization problem in a generic form [49,53,160]. All optimization problems with
explicit objectives can in general be expressed as a nonlinearly constrained optimiza-
tion problem

maximize/minimize f (x), x = (x1 , x2 , . . . , xD )T ∈ RD ,

subject to φj (x) = 0 (j = 1, 2, . . . , M),
ψk (x) ≤ 0 (k = 1, . . . , N), (1.25)

where f (x), φj (x), and ψk (x) are scalar functions of the design vector x. Here the
components xi of x = (x1 , . . . , xD )T are called design or decision variables, and they
can be either continuous, discrete, or a mixture of these two. The vector x is often
called the decision vector, which varies in a D-dimensional space RD .
It is worth pointing out that we use a column vector here for x (thus with trans-
pose T ). We can also use a row vector x = (x1 , . . . , xD ) and the results will be the
same. Different textbooks may use slightly different formulations. Once we are aware
of such minor variations, it should cause no difficulty or confusion.
In addition, the function f (x) is called the objective function or cost function,
φj (x) are constraints in terms of M equalities, and ψk (x) are constraints written as
N inequalities. So there are M + N constraints in total. The optimization problem
formulated here is a nonlinear constrained problem. Here the inequalities ψk (x) ≤ 0
are written as “less than”, and they can also be written as “greater than” via a simple
transformation by multiplying both sides by −1.
The space spanned by the decision variables is called the search space RD , whereas
the space formed by the values of the objective function is called the objective or
response space, and sometimes the landscape. The optimization problem essentially
maps the domain RD or the space of decision variables into the solution space R (or
the real axis in general).
The objective function f (x) can be either linear or nonlinear. If the constraints φj
and ψk are all linear, it becomes a linearly constrained problem. Furthermore, when
φj , ψk , and the objective function f (x) are all linear, then it becomes a linear pro-
gramming problem [35]. If the objective is at most quadratic with linear constraints,
then it is called a quadratic programming problem. If all the values of the decision
variables can be only integers, then this type of linear programming is called integer
programming or integer linear programming.
On the other hand, if no constraints are specified and thus xi can take any values
in the real axis (or any integers), then the optimization problem is referred to as an
unconstrained optimization problem.
As a very simple example of optimization problems without any constraints, we
discuss the search of the maxima or minima of a univariate function.
8 Introduction to Algorithms for Data Mining and Machine Learning

2
Figure 1.2 A simple multimodal function f (x) = x 2 e−x .

Example 2
For example, to find the maximum of a univariate function f (x)

f (x) = x 2 e−x ,
2
−∞ < x < ∞, (1.26)

is a simple unconstrained problem, whereas the following problem is a simple constrained mini-
mization problem:

f (x1 , x2 ) = x12 + x1 x2 + x22 , (x1 , x2 ) ∈ R2 , (1.27)

subject to

x1 ≥ 1, x2 − 2 = 0. (1.28)

It is worth pointing out that the objectives are explicitly known in all the optimiza-
tion problems to be discussed in this book. However, in reality, it is often difficult to
quantify what we want to achieve, but we still try to optimize certain things such as the
degree of enjoyment or service quality on holiday. In other cases, it may be impossible
to write the objective function in any explicit form mathematically.
From basic calculus we know that, for a given curve described by f (x), its gradient
f  (x) describes the rate of change. When f  (x) = 0, the curve has a horizontal tangent
at that particular point. This means that it becomes a point of special interest. In fact,
the maximum or minimum of a curve occurs at
f  (x∗ ) = 0, (1.29)

which is a critical condition or stationary condition. The solution x∗ to this equation

corresponds to a stationary point, and there may be multiple stationary points for a
given curve.
To see if it is a maximum or minimum at x = x∗ , we have to use the information of
its second derivative f  (x). In fact, f  (x∗ ) > 0 corresponds to a minimum, whereas
f  (x∗ ) < 0 corresponds to a maximum. Let us see a concrete example.

Example 3
To find the minimum of f (x) = x 2 e−x (see Fig. 1.2), we have the stationary condition
2

f  (x) = 0 or

f  (x) = 2x × e−x + x 2 × (−2x)e−x = 2(x − x 3 )e−x = 0.

2 2 2
Introduction to optimization 9

Figure 1.3 (a) Feasible domain with nonlinear inequality constraints ψ1 (x) and ψ2 (x) (left) and linear
inequality constraint ψ3 (x). (b) An example with an objective of f (x) = x 2 subject to x ≥ 2 (right).

As e−x > 0, we have

2

x(1 − x 2 ) = 0, or x = 0 and x = ±1.

The second derivative is given by

f  (x) = 2e−x (1 − 5x 2 + 2x 4 ),
2

which is an even function with respect to x.

So at x = ±1, f  (±1) = 2[1 − 5(±1)2 + 2(±1)4 ]e−(±1) = −4e−1 < 0. Thus, there are
2

two maxima that occur at x∗ = ±1 with fmax = e−1 . At x = 0, we have f  (0) = 2 > 0, thus
the minimum of f (x) occurs at x∗ = 0 with fmin (0) = 0.

Whatever the objective is, we have to evaluate it many times. In most cases, the
evaluations of the objective functions consume a substantial amount of computational
power (which costs money) and design time. Any efficient algorithm that can reduce
the number of objective evaluations saves both time and money.
In mathematical programming, there are many important concepts, and we will
first introduce a few related concepts: feasible solutions, optimality criteria, the strong
local optimum, and weak local optimum.

1.2.3 Feasible solution

A point x that satisfies all the constraints is called a feasible point and thus is a feasible
solution to the problem. The set of all feasible points is called the feasible region (see
Fig. 1.3).
For example, we know that the domain f (x) = x 2 consists of all real numbers. If
we want to minimize f (x) without any constraint, all solutions such as x = −1, x = 1,
and x = 0 are feasible. In fact, the feasible region is the whole real axis. Obviously,
x = 0 corresponds to f (0) = 0 as the true minimum.
However, if we want to find the minimum of f (x) = x 2 subject to x ≥ 2, then it
becomes a constrained optimization problem. The points such as x = 1 and x = 0 are
no longer feasible because they do not satisfy x ≥ 2. In this case the feasible solutions
are all the points that satisfy x ≥ 2. So x = 2, x = 100, and x = 108 are all feasible. It
is obvious that the minimum occurs at x = 2 with f (2) = 22 = 4, that is, the optimal
solution for this problem occurs at the boundary point x = 2 (see Fig. 1.3).
10 Introduction to Algorithms for Data Mining and Machine Learning

Figure 1.4 Local optima, weak optima, and global optimality.

1.2.4 Optimality criteria

A point x ∗ is called a strong local maximum of the nonlinearly constrained op-
timization problem if f (x) is defined in a δ-neighborhood N (x ∗ , δ) and satisfies
f (x ∗ ) > f (u) for u ∈ N (x ∗ , δ), where δ > 0 and u = x ∗ . If x ∗ is not a strong lo-
cal maximum, then the inclusion of equality in the condition f (x ∗ ) ≥ f (u) for all
u ∈ N (x ∗ , δ) defines the point x ∗ as a weak local maximum (see Fig. 1.4). The local
minima can be defined in a similar manner when > and ≥ are replaced by < and ≤,
respectively.
Fig. 1.4 shows various local maxima and minima. Point A is a strong local max-
imum, whereas point B is a weak local maximum because there are many (in fact,
infinite) different values of x that will lead to the same value of f (x ∗ ). Point D is the
global maximum, and point E is the global minimum. In addition, point F is a strong
local minimum. However, point C is a strong local minimum, but it has a discontinuity
in f  (x ∗ ). So the stationary condition for this point f  (x ∗ ) = 0 is not valid. We will
not deal with these types of minima or maxima in detail.
As we briefly mentioned before, for a smooth curve f (x), optimal solutions usu-
ally occur at stationary points where f  (x) = 0. This is not always the case because
optimal solutions can also occur at the boundary, as we have seen in the previous ex-
ample of minimizing f (x) = x 2 subject to x ≥ 2. In our present discussion, we will
assume that both f (x) and f  (x) are always continuous or f (x) is everywhere twice
continuously differentiable. Obviously, the information of f  (x) is not sufficient to
determine whether a stationary point is a local maximum or minimum. Thus, higher-
order derivatives such as f  (x) are needed, but we do not make any assumption at this
stage. We will further discuss this in detail in the next section.

1.3 Unconstrained optimization

Optimization problems can be classified as either unconstrained or constrained. Un-

constrained optimization problems can in turn be subdivided into univariate and mul-
tivariate problems.
Other documents randomly have
different content
The Project Gutenberg eBook of A Yellow
Aster, Volume 2 (of 3)
This ebook is for the use of anyone anywhere in the United
States and most other parts of the world at no cost and with
almost no restrictions whatsoever. You may copy it, give it away
or re-use it under the terms of the Project Gutenberg License
included with this ebook or online at www.gutenberg.org. If you
are not located in the United States, you will have to check the
laws of the country where you are located before using this
eBook.

Title: A Yellow Aster, Volume 2 (of 3)

Author: Mrs. Mannington Caffyn

Release date: November 22, 2020 [eBook #63841]

Language: English

Credits: Sonya Schermann and the Online Distributed

Proofreading
Team at https://www.pgdp.net (This file was produced
from
images generously made available by The Internet
Archive)

*** START OF THE PROJECT GUTENBERG EBOOK A YELLOW

ASTER, VOLUME 2 (OF 3) ***
This ebook was created in honour of Distributed Proofreaders’ 20th
Anniversary.
 A Yellow Aster

BY
Iota
“And if this fought-for climax is ever reached
and science, creeping along the path of experiment,
so invades the realm of Nature that a blue chrysanthemum
or A Yellow Aster can be produced at
will, the question still remains, has Nature been
made more beautiful thereby?”

IN THREE VOLUMES

VOL. II
London 1894
HUTCHINSON & CO.
34 PATERNOSTER ROW

PRINTED AT NIMEGUEN (HOLLAND)

BY H. C. A. THIEME OF NIMEGUEN (HOLLAND)
AND
TALBOT HOUSE, ARUNDEL STREET
LONDON, W.C.
 CONTENTS

PAGE

CHAPTER XVIII. 1
CHAPTER XIX. 19
CHAPTER XX. 33
CHAPTER XXI. 59
CHAPTER XXII. 75
CHAPTER XXIII. 91
CHAPTER XXIV. 109
CHAPTER XXV. 122
CHAPTER XXVI. 133
CHAPTER XXVII. 161
CHAPTER XXVIII. 174
CHAPTER XXIX. 191
CHAPTER XXX. 203

A YELLOW ASTER.
 CHAPTER XVIII.

“To look at the fellow one would never give him credit for half the
grit he has,” thought Strange as he glanced round for a cab at the
street corner. “If I had money I should send him to Paris,” he went
on as soon as he had settled himself comfortably, “the Kensington
methods are no manner of use to him. It’s the deuce of a shame
too, that he has to attempt finished work for a living when he should
be swatting over the primaries; and that colour mania—that will get
chronic and overgrow him, and then God help him!”
As it happened Lady Mary was at home and quite wide-awake. As
a rule this was not the case until much later in the day, but just now
various things combined to keep off sleep.
When Strange was announced, she was sitting well screened from
the small bright fire, gazing in soft meditation at her plump white
hands, with the corners of her mouth slightly drawn downwards, and
her smooth round forehead wrinkled up in a way that would have
gone to the heart of a stone to see in such a picture of comfort as
she was made to be.
“Humphrey!” she exclaimed, making a vain try at a spring and
flopping down again limply, “Humphrey!”
“Myself and no other,” said Strange, receiving her kiss cheerfully,
and settling himself into a chair after he had shaken it to see if it
would bear. “I needn’t ask you how you are, Aunt Moll, you look just
as you always did, like a catkin.”
“A what, Humphrey?” she enquired anxiously.
“A catkin, we used to call them goslings, soft, oval, pale gold,
silky, fluffy masses—you have a weakness for adjectives I know,
judging from the line in literature you patronize. The harshest wind
has never been known to ruffle a gosling, it always skips them, they
always feel warm to the touch, as if the sun were on them, they are
delicious things. The sun is always on you, Aunt Moll, ain’t it?”
“Ah, Humphrey, you little know, you can make but a faint guess at
my troubles, the death of my dear——”
“Aunt Moll, we’ll skip that!” interrupted Strange, with a twinkle.
He knew quite well what an unmixed relief the deceased peer’s
removal was to all his kith and kin, more especially to his wife.
“If you recollect, before I went to Algeria we agreed to let my
uncle rest undisturbed in his present retreat, which, from what we
know of his past, must be unexceptionable—whatever his faults may
have been no one can deny that he was a most exclusive person and
had a very just notion of his position.”
“Dear Humphrey! That flippancy! I had hoped that the many
dangers you have experienced, the many times you have come face
to face with death—and, Humphrey—with Eternity—would have
brought the seriousness of life before your eyes.”
“Aunt Moll, the sight of you there in that chair brings that view of
the case more clearly before me than ever the sight of death did.”
Lady Mary again looked anxious, her nephew always made her
feel like that, his eyes seemed to rake her from stem to stern and to
find some mute amusement in the process. Suddenly she gave a
little start.
“What have I been thinking of?” she murmured. “Humphrey,” she
began again, “we must speak of your prospects.”
She was bubbling over with them as it happened, besides, they
would keep him off her.
“What are you thinking of doing now?”
“What I have always been thinking of doing and have never done
yet, making the result of my face to face encounters with death—
and Eternity—of some practical value to the world in general and to
myself in particular, by filling my trousers’ pockets, which at this
present moment contain one pound six and threepence, and that’s
mostly due for beer.”
“Humphrey! Have you heard nothing? Your letters?”
“I never read them. For Heaven’s sake, speak, divulge, I’m ready
for anything!”
“Your great-uncle is dead—died last month. Before he went he
confessed a heavy sin that had lain for years on his soul, poor dear
creature. That great lanky son of his, about whom, as you know, I
always had a nasty feeling, as if he were not altogether quite right,
as if somehow he was not one of us. This now proves to have been
a quite prophetic instinct, he turns out to be—ahem—illegitimate,
and you, you, Humphrey, are the heir.”
“I say! It’s beastly hard lines on Tom!”
Strange was quite as staggered with the news, as any other
younger son in his condition would have been. It vibrated through
and through him, but as one cannot clothe thunder in harmonies any
more than one can a tumultuous muddle of sensations in speech in
the presence of a woman inclined to gush and stoutness, he
swallowed his muddle and was flippant.
“Humphrey!” said Lady Mary with dignity, wondering a little if
Humphrey himself were quite right. “This minute you have ten
thousand a year, and you, my nephew, are Sir Humphrey Strange.”
“Am I? You’ll be astonished to hear I don’t feel a bit like it, I feel
exactly as I did before. Is there any difference to the naked eye, if
so, do you mind telling me?”
Lady Mary stirred uneasily and crossed her hands.
“Dear Humphrey!” she cried at last, with a soft wailing bleat, “I
confess I did expect some show of proper feeling from you on this
occasion. It is a shock to me to see you in your present frame of
mind, it seems like flying in the face of Providence, and may end in
bringing down a judgment on your head.”
Lady Mary sighed and continued, lowering her voice to a coo,
“When I heard the news, Humphrey, I went down on my knees and
prayed that my poor sinful uncle might be forgiven for foisting that
counterfeit young man off on our family, and that you, my nephew,
might face your responsibilities with a seriousness befitting the
occasion. My dear, if you knew what it costs me to kneel, now that I
have grown a little stout, you might perhaps appreciate this act.”
Humphrey grinned.
“Aunt Moll, my feelings are always too deep for expression, it
would upset you for a month if I were to give you the merest
glimpse of the emotions that are ravaging me this minute. These
inward upheavals are frightfully wasting, your acts of prayer and
thanksgiving are a fool to them—There doesn’t happen to be any tea
going, does there?”
“Tea! Is it five o’clock? What can have happened? Pray ring. The
misery I have to endure with servants! I wonder my hair isn’t even
greyer than it is, and my poor face more worn.”
“Your hair is as brown as a nut, and there isn’t a crease in your
dear, soft young face. What was wrong with you when I came in, the
corners of your mouth were turned the wrong way?”
Lady Mary reflected as she made his tea.
“Ah, it was Gwen, she has thrown aside another most
unexceptionable match, the third in three months.”
“Gwen, what?”
“Gwen Waring, she is with me for the season.”
“Ah, that queer, sulky, imperturbable, long-legged girl, belonging
to those wonderful young fossils at Waring Park. I shouldn’t have
thought she’d have got the chance to throw over any match, let
alone three unexceptionable ones——”
“Humphrey!”
“What’s up? Gru!—”
He sprang to his feet.
A tall superb girl with a face like a hothouse flower, was standing
in the middle of the room, looking at him with a cool aloofness that
made his blood run cold. She had heard every word, she must have,
his voice was a big one.
This magnificent dominant creature, before whom he felt as a
worm, was only an enlarged completed edition of the “sulky, long-
legged” slip he used to catch fitful glances of, in his stays with his
aunt.
If only he hadn’t classified her in such cool pleasant tones! It was
not often the fellow felt at such a disadvantage. If the girl had made
a joke now, or even looked as if she could make one! But she knew
better than to joke, she had her tactics ready to her hand, and she
was determined his impertinence should be brought home to him.
Her own classification never troubled her in the least, it was the
good-humoured sneer at her parents which touched her. Was she
always to suffer for being the product of such a house?
The next few minutes Strange felt younger than he had done for
ten years.
“Lady Mary has been telling me of your good fortune,” she
remarked kindly, sipping her tea, and looking at him in as motherly a
way as so very splendid a person could look. “You must be quite
excited—I suppose you are already making a hundred plans?
“I seem to know you quite well,” she went on, not giving him the
chance to reply, “Lady Mary is always telling anecdotes of ‘her boy’,
very entertaining ones they are too, and I should fancy
characteristic.”
She helped herself to more cream and regarded him coolly.
“When she reads prayers, she always makes a special and very full
mention of you.”
Lady Mary winced abjectly and looked deprecatingly at her
nephew, but his eyes were fastened on Gwen. His aunt felt she had
escaped for once. She settled herself into her pillows, and wondered
vaguely what would happen next.
She had a horrid feeling that there were breakers ahead
somewhere, but as she never by any chance could see farther than
her own nose, she decided not to make any effort at sighting them,
but to drift on with faith.
“Very considerate of my aunt!” said Strange, in a pause.
“Oh, that is only one instance of her consideration and the least
important. She has done much more than that for you, she is like
John the Baptist without the skins and locusts, she has ‘been
preparing the way before’ you, and you have only to appear to be
mobbed, Sir Humphrey. There’s not a matron nor a maid in London
who doesn’t babble of you; your name is rippling off a hundred
tongues at this very minute; you are the hero of a hundred teas. All
this came on after a long round of calls Lady Mary and I paid last
Monday,” she continued, scanning him. “I had only heard your name
before, in the outward world, that is—the Baronetcy never affected
Lady Mary’s prayers and anecdotes, they were always with us—in a
queer aside way, as if one hinted at dark things that had better not
be unearthed. Ah, but that is all changed! You have no notion
though how exhausting the process has been to Lady Mary.”
She stopped at last.
“No,” he said, looking at his aunt, “I certainly hadn’t perceived any
symptoms of a cave-in about her. Monday, did you say, Miss Waring?
Would you mind letting me have your visiting list for that day, Aunt
Moll? I suppose I know some of the people, and my soul’s one desire
for years has been to pose as an afternoon-tea hero. I shall just
have time to get a foretaste of the joys this afternoon. Good-bye,
Aunt Moll, pray don’t look anxious on my account, my morals are
tough enough to run the gauntlet of all the teas in London, and my
digestion is unimpaired. Good-bye, Miss Waring,” he said, bowing
gravely in her direction, “thank you for standing by my aunt on
Monday’s warpath, I am gratified to see you are in no sort of way
exhausted by the process. Damnation!” he muttered as he got out
into the street, “she smells of a hothouse with her overpowering
beauty and her insolent airs, and that cool inexorable way of hers.
Oh, Aunt Moll, you’ll rue the day you made me a by-word. To think I
had to swallow all that, and let a girl bait me!”
 He laughed aloud.
“And so I am the coming parti! Good Lord! I’ll be fine practice for
the ‘sport,’ anyway they’ll find me shy game. I’ll go home, finish a
chapter or two, dose Tolly, and then I’ll dine.
“Hullo!” he exclaimed suddenly, “things are looking up for Charlie,
he can go to Paris now when he likes. I wonder how I can reduce
his high stomach to seeing it in that light!”
 CHAPTER XIX.

Strange found the preliminaries of his induction into the rôle of an

English Squire even more unpleasant than he had expected.
During the period when he had read Roman law and knocked
about the Courts with the hope of supplementing his income by the
experience he picked up there, the technicalities of the law had
bored him to excruciation point. Now, when they were brought
specially to bear on him he found them more galling still, but being a
wise man in his way, he shirked none of them, and took good care
not to take a solitary step in the dark, till, by the time they had got
him off their hands, the solicitors of the Stranges were in a position
to congratulate themselves at last, on the fact of having found a
whole man in the family.
He had gone the rounds of his duties doggedly and had found
them insufferably dull, he had been down to Strange Hall, had left
things there in trim, and had now flown back to London.
One afternoon in June he was standing in the shadow of a deep
window, in one of his rooms in Piccadilly, lazily sharpening a pencil.
He had plenty of work to do, but somehow he had no stomach for
it, the change in his life had got into his bones, and had filled him
with unrest and a certain loss of faith in himself. When at last after a
long meditation, the truth of this broke upon him, it came with an
audible and ample, “Damn!”
“I may as well give it up and amuse myself in a mild way,” he
thought, after a hasty review of matters, “nothing can be too weak
and vapid for my present condition—I feel flabby.”
 A mild grunt at his back made him swing round. It was Tolly, just
back from the dentist, of a deeper puce than usual, and with a
terrible uncompromising row of glistening teeth shooting out
aggressively between his thin lips.
He gave a deferential duck, and stood on approval, with a
laboured attempt at an appearance of modest deprecation.
“Turn round, Tolly,” said his master, “away from me, I can’t bear it
all at once!”
He was shaking with silent laughter.
“How do you feel about them yourself, Tolly?”
“Fust-rate, sir—your wussup.”
Since his master’s rise in life he was much exercised as to the best
terms by which to give him honour, and he varied them daily.
“I can bite nails, your wussup.”
“Ah! You mustn’t play fast and loose with these tusks as you might
with ones bred and reared on the premises.”
“Lord! your wussup, I wouldn’t make that free, being, as they are
your property, sir, besides, any fool can see as how they be the real
bought article, money down, not your everyday common grinders.
There weren’t a toff I met as didn’t mention ’em, I tried to keep ’em
dark, sir.”
“I shall expect a good deal more from you,” said Strange, pointing
the moral, “now you’re complete. If anyone calls to-day say I’m out
and I won’t be home till night, and—take these to the post before I
start,” he pointed to a big heap of notes on the table, “and don’t
drop any of them, nor swallow your teeth.
“Twenty invitations in a week,” reflected Tolly’s master, “the first-
fruits of my rise in life! they used to average six a week. I’ll go and
see Lady Mary. Damn it all, why need a man lie to himself, I’ll go and
see Miss Waring!”
And he went, and somehow the next day he went again, and the
next, and the next after that. Then he and Gwen discovered a
mutual passion for riding, not up and down the Row, that seemed as
tame a pastime to the one as to the other, but in the early mornings
out on the heath at Hampstead, or sometimes far out on the Surrey
side.
Once they went as far as Surbiton, where they got drenched in a
shower and had to take refuge and have tea in an old inn.
But it is not at all to be supposed that with all this intimacy those
two got an inch nearer one another, they were intellectual
companions, nothing more, not even to be called comrades.
Gwen neither evaded nor shirked conventions, she simply swept
them aside, as she did her lovers. As for Strange, he felt her and the
rides very distinctly a boon. She was an excellent flint to make
sparks with, her ways of thought were so new, let alone startling,
her modes of expression so quaint, her tongue so remarkably sharp,
and she had such a brutal habit of speaking undiluted truths. For the
once the two agreed, they disagreed at least three times, and a
good pitched battle had to be fought to settle any question. The
sponge was never by any chance thrown up, it was forced out of the
hand of one or of the other of them. It was a most bracing and
delightful experience for Gwen, it was so satisfactory and so
absolutely free from mawkishness, and she reflected, with superb
self-congratulation, that the man had just as little capacity for that
phase as herself.
“She’s hard—hard as nails,” he reflected after an evening at Lady
Mary’s, “and yet, she wasn’t made like that, I could swear. I wonder
what the devil’s wrong with her eyes, and what’ll put them right? I
believe, upon my word I do, that a baby might do the business for
her. There’s not a man living that would have any effect upon them,
and yet there are fellows going who would take that dewiness, for
softness, hang it! it’s mere moisture, but—ah, well, the effect is
magnificent!”
He took out his watch, but his hand shook so that he could not
open it.
“God forgive me!” he muttered, “this is awful! I have had a good
deal in the way of education at women’s hands, but this is a new
experience,” he remarked after a pause, grinning, and flicking a spot
of ash off his coat, “her want of self-consciousness is next to ghastly,
it has an uncanny sexless sort of air about it that gives one the
shivers.”
The intellectual companionship continued unabated for ten more
days, then one evening at the end of June, Gwen Waring told
Strange that she and Lady Mary were going down into the country
early in July.
When he got home that night he had a difficulty in mounting the
stairs. When he succeeded, he got himself to the glass, and found
he was white to the lips. He had had a shock—he had discovered, as
he had turned out of Lady Mary’s softly-lighted hall into the street,
that he loved the girl irretrievably, and with the knowledge came
fear.
For a few minutes he leaned against the mantel-piece, his head
sunk into his hands, then he raised himself with a sigh, threw off his
light overcoat, and sat down to smoke, but he couldn’t draw a puff,
then it struck him that he was numb with cold.
He looked at the grate with a purpose to make a fire lighting in his
eyes, but with a shrug he shirked the trouble. He could not go to
bed, that was out of the question; as for sitting there freezing, that
was just as impossible. He must move, he must feel the life stir in
him again. He stood up and shook himself, then a thought struck
him, he hurried to his room, changed his clothes, and went out
round the corner to the mews where he kept the horses he had
brought up from Strange Hall.
He found the gear, saddled the freshest, and rode away through
short cuts and byways, away from the noise and hurly-burly, out into
the quiet of the country. Then he drew rein, pulled the mare aside
on to a green strip flanking the road, and let her go her own pace.
For a long time he gave her grace and smoked savagely.
“It is about the most killing blow that could have fallen on a man.
It would be bad for any fellow; but for me, who can love if I can do
anything, to have to pour it all out at the feet of a girl who couldn’t
understand what love, much less passion, means, to save her life!
It’s a beastly backhanded stroke of fate, and I don’t know that I’ve
ever done anything bad enough to deserve it. Lord, how the mare’s
sides smoke! I must have ridden like a maniac. The worst of it is,
this isn’t a thing one can clear off and forget—with the woman right
in one’s soul!—the fine, grand, proud creature! God! it’s almost
sacrilege to expect her to love, with love in the beastly state it is—to
love any man-Jack of us; it’s honour enough to love her and yet,—
yet,—when a man has once done it, done it once and for ever, the
only thing in life seems to be to get something in return. What
commercial brutes we are even in this holiest connection of all! But
let her love or not, I’ll give her my love if she’ll take it and I shall
pick up crumbs like old Lazarus.—Pah, how she dominates one!—Ah,
and when her love wakes up—but, the devil! suppose another fellow
is the instrument chosen! Ah!—ah! hold up, mare, are you stumbling
or am I reeling? It’s myself, by Jove, God help us!”
Involuntarily he drove his spurs into the beast, she started forward
angrily, unused to maniacs. Presently he came to his senses and
pulled her up with a drag on her mouth that she did not forget for
some time. She went sulky and stumbled for the next mile, small
blame to her! A Christian would have done more.
Gradually her master’s face cleared itself and softened.
“Perhaps,” he muttered, “perhaps no other fellow after all, but—
who knows?—a baby’s tender little mouth may do it.”
 CHAPTER XX.

When Strange got back to town, after baiting man and beast at a
little inn on the outskirts of Weybridge, Tolly’s greeting, which was
blasphemous and amazed, and the unusual look in his green eyes,
caused his master to glance at himself in the glass.
“Heaven!” he thought, turning away, “I’m a nice object to go
courting! One would think I had just emerged from D.T., or Bedlam!
Tolly, turn on the hot water, empty a bottle of vinegar into it, and put
out clean clothes for me. I feel like jelly. Good Lord! has love this
limping effect often?”
He turned into his bedroom. As he was wrestling with one of his
shirt buttons he muttered,
“However this goes, it’s a toss-up what the gain will be, heaven or
hell. Well, a man might do worse than face hell for her.”
He had hardly made this heroic remark when the absurdity of it
struck him; he laughed aloud. “I had better face my bath,” he said.
When he was washed and dressed, he rather thought of the Club
and a good lunch, but the game didn’t seem worth the candle. He
felt that his hands were quite sufficiently full with one woman, he
had no desire for men, more especially at feeding-time.
“I shall have my lunch here,” he said, looking up from his paper,
“get out some bread and cheese, and beer, and anything else you
can lay your hands on.”
In five minutes Tolly had covered a little round table with a cloth,
and had set out on it a mixed assortment of cheese, beer, jam, and
a freshly-opened tin of foie gras, and he stood proudly in attendance
with napkin on arm, keeping down with difficulty a grin of self-
satisfaction.
However full he was of himself, Strange never let a new
accomplishment of Tolly’s escape him, if he did, the effect on the
boy would have been disastrous. No sinner ever strove after God as
this sinner after his owner.
“Well done, Tolly, you’ll shine in life yet, the way you flourish that
damask is sublime!”
“Beggin’ your pardon, your wussup,” said Tolly, “Bill, the groom, ’e
were round after ye, a-stormin’ at me because the horse was out.
Bill always lets out at me like when he feels hisself put about in his
mind, and he thought you and the beast were lost,” sniggered Tolly.
“I told him you was big enough to take care of yourself, and that
gents often finds the nights more convenienter than the days,” he
remarked confidentially, pushing the salt under his master’s nose.
“Bill is that ignorant, sir, of loife and sich, he erstonishes me.”
Strange drank his beer with a look at the half-made creature who
had plumbed ‘loife’ from the vantage ground of her sewers.
“Very like his betters,” he thought, “we get lots of our views from
a vantage ground not one whit sweeter or cleaner than Tolly’s.”
He made a fresh dive into the pâté and his thoughts broke out on
a new track.
“I think we’re going off somehow. I believe it is a good deal the
women’s fault; this new craze for advanced talk between the sexes is
no good, the women who affect it are never clever enough nor good
enough to make a success of the thing, it’s a pose mostly, as their
smoking is, just done for effect.—Tolly, pass that jam!”
When he had rounded off his meal with a hunch of bread and
strawberry jam, he stretched himself, went to the window and
looked out, drumming gently on the pane.
“I wonder,” he thought, “I wonder if I am quite a fool or not, but—
but, God! how I love her!”
 Then he stopped drumming, and began to wonder vaguely how in
the name of Heaven he was able to eat great hunches of bread and
jam not five minutes before.
He turned and watched Tolly through the door, devouring at his
ease, with a sudden shock of disgust, more at himself than at the
fellow, with his hideous mouth all moist and jammy. He turned again
to the window and tried to steady his brain, but it reeled and
everything in the room swam before him, he dropped his head in his
hands and trembled from head to foot, when he raised it he felt
steadier and not so raging hot.
“I shall chance it,” he said, “I shall chance it.”
When he reached Lady Mary’s he was in a much more wholesome
frame of mind. He had gone there by roundabout ways, where he
saw a good deal of stark, staring, naked humanity; this helped to
crystallize his emotions, to sift the dross out and leave the clean
stuff.
He never in his life felt clearer-headed than when he went up the
stairs unannounced, and paused to look through the half opened
door at Gwen, sitting near a window in a cloudy dress of soft yellow
crêpey stuff and with her strong, long-fingered, composed hands
lying idle in her lap and the guard dropped from her eyes, showing a
good deal more of herself than he had ever seen before.
He only paused for one minute, he had no right yet to the girl’s
secrets; then he threw open the door with a little bang and brought
her back to the present.
“Oh, is it you?” she said with the ghost of a start, looking up at
him.
She felt in a vague way that he knew more of her in that one
minute than he had any business to do, and she was not quite sure
if she liked it or not. He did not offer to shake hands with her but
glanced round the room silently. Gwen laughed.
“You are looking for Lady Mary? She has a bad headache, an
abnormally bad one, and won’t be down till five.”
 He offered up a dumb thanksgiving and sat down carefully, then
he felt a horrible desire to say, “Hem!” or to mention the deuce or
the weather.
He had felt intensely reasonable the minute before, but he was
confused by the beauty of the girl sitting so close to him, with the
flickering sunshine running golden threads in and out her twisted
russet hair, and clothing her in pale molten gold.
“She shall have nothing to add to her beauty,” he thought, “I shall
not make a beast of myself to desire the least of her when it is the
greatest I want.”
He started up, and asked if he might draw down the blinds.
“Yes,” said Gwen wonderingly, as she saw his big brown hand
tremble on the blind line.
Then a sudden certainty of his intention came upon her with a
burst of angry horror, but she swept this off and waited coolly, with a
sort of sneering excitement.
Strange drew his chair farther forward and sat facing her.
“Miss Waring,” he said, “I have come to ask if you will listen to the
shady side of a man’s life.”
There was no more tremble or hesitation about him now, he
looked as cool as she did.
“It is a side that men as a rule keep to themselves and to their
male companions, no matter how near a man and a woman come to
each other, this impalpable barrier keeps them apart. This has
always struck me as a rather low form of lie and distinctly
dishonourable, especially practised, as it is, by the stronger on the
presumably weaker. If a woman is not strong and pure and
magnanimous enough to bear this knowledge, a man should find it
out and go his way before he has dared to touch her life; if she is
strong enough she should be given the opportunity of gaining this
knowledge at first hand, and taking her subsequent course
accordingly. You are immeasurably nobler than any other woman
who has crossed my path.”
 Involuntarily he lowered his head as he spoke, in a reverential way
that touched Gwen and forced her to hear him. After the first
disgusted shock her impulse had been to send him about his
business. She had half risen from her seat on the spur of this
impulse, but somehow she had sat down again, and in spite of
herself she had let him speak.
“No decent man could deceive you,” he went on, “even if every
word he spoke were to cut his own throat. May I speak to you as
man to man?”
He watched the palpitations of her throat—which unfortunately
were beyond her control—with a sort of choking sensation—
“Or more,” he added simply, “as if you were God.”
Gwen’s colour neither increased nor left her, she neither trembled
nor stirred. For a minute she was quite silent except for one quick
little swallowing sound, she was fighting with a concentrated
restrained frenzy of despair against her fate, against the
overpowering longing to hear this man, as he sat there ready to
spoil his own life sooner than lie to her even in a fashion recognized
by the use of generations.
She was quite aware she had nothing whatsoever to give him in
exchange, she knew perfectly well she was about to do him a
grievous wrong, and yet her whole being was concentrated into one
imperative demand to hear what he had to say.
“You may speak,” she said in a hard emotionless voice.
Then he told her simply, with neither condonation nor reservation,
the whole truth about his life.
It is all very well to talk glibly about the advantages of calling a
spade a spade, but when it comes to giving dozens of spades their
unvarnished titles in the presence of one virgin clean woman, and
when every fresh spade may be about to dig up the heart you would
foster, the matter is no joke.
By the time that Strange had arrived at the end of his unadorned
record, his smooth, brick-dust cheeks looked gray and haggard, and
his voice sounded tired.
 Once during the recital Gwen had lost guard over herself and had
let a flash of half-triumphant interest escape from her eyes. It was
when he had said—“Thank Heaven! I never loved one of these
women, that is, taking love in its all-round, large sense.”
When he had finished he stood up and looked at her, waiting.
She had herself still in her power, she felt, with a wild leap of her
spirit, she could yet ward off her fate and his; “his,” she thought with
a wave of soft unaccustomed pity. She had nothing to give this man,
nothing, not even the germs of a possible something—something
called Love.
She laughed aloud and looked in his face when the empty word
stirred her brain, then she lowered her eyes and turned all her
thoughts in on herself, moving a small pearl ring up and down her
finger with a swift rhythmic movement.—This man would take her
for mere hope—hope that had no foundation in fact,—it was a mean
exchange, nothing for everything,—mean and unjust; for the minute
she was hideous to herself, with her own whole life a protest against
the injustice of others.
She looked at him again, and a horrible power seemed to drag
and bind her to him, she turned her eyes away angrily and made a
little involuntary sound of trouble.
“Oh, if I only could treat him as I did the others!” she muttered
under her breath, “but I can’t, I can’t!”
She was frightened at herself—at the power which drove her to
the man inexorably,—she looked at the door and stirred in her seat,
half-rising, but she sat down again and began to move her ring with
the old movement, only quicker and with tenser fingers.
Then a cold feeling of finality came on her, she knew she must say
something and she knew she was going to say the wrong thing; an
inexplicable smile flickered across her face and touched her mouth,
she grew quite calm and ceased to move her ring.
“You have done me a very high honour,” she said; “thank you.”
He came nearer and looked down on her.
 “I have tried to be perfectly honest,” he said, “and you have no
idea what an awful grind it has been. It would be quite impossible
for me to give you any idea of how I honour you, and as for love—”
he stopped, breathing hard, “I have a heart full for you, dear, I don’t
think I know myself how much I love you.”
The girl looked at him curiously, the simple intensity of his manner
struck her, then her eyes fell and she sighed.
“Love is such a mere name to me,” she said, “it seems such a
collapsable bubbly thing and put to such feeble uses. You want me
to be your wife then, and you offer me a whole heart full of love,
whatever that may mean. I must be honest too, and tell you that I
shouldn’t know how to dispose of a whole heart full of love. I know
nothing at all practically about the matter, and theoretically it has
never interested me. My situation is hard to explain,” she exclaimed,
with a petulant sweeping movement of her hand, “in the face of all
this I want to accept your offer, I don’t know why, I really believe it
is not I, Gwen Waring, that wants this, it is something outside me
that wants it for me. I never felt so impersonal in all my life.”
He winced, her honesty, to say the least of it, was a trifle bald.
“Perhaps I am more concerned in it than I think,” she went on
with a queer intense serenity, dissecting herself audibly, “I like new
sensations, I am curious, most things are so flat and boring.”
Strange started forward and was about to speak, she raised her
hand imperiously.
“Stop!” she cried, “I must finish, I want you quite clearly to
understand that if I take you at your word and become your wife—
wife,” she repeated, “how astonishing the word sounds in connection
with me!”
She laughed in an untranslatable way and went on,
“Remember and understand that I am doing it as an experiment.”
He flushed, it was his own precise thought but it seemed less
hideous when thought than when spoken.
 “An experiment,” she repeated, “but whether it is fair to try
experiments in lives is another matter. I wish—” she cast a half-
wistful, half-provoked look at him, “I wish you were sufficiently clear
and reasonable yourself to help me to answer the question—I am so
ignorant in these matters.”
A sudden crimson rushed to her cheeks, she was furious. What
right had she to blush like a dairy-maid and mislead the man?
“I’m not blushing properly, as girls ought to blush,” she explained,
“I am merely angry, I feel caught in a trap. Why can’t I tell you to
begone and leave me at peace?” she demanded, looking at him with
curious swift repulsion, “I have never found any difficulty before,—
why don’t you help me?”
In spite of his love, Strange shook with laughter.
It was no laughing matter for Gwen, she kept her eyes fixed on
him, angry and full of pain.
“You stand there and laugh—laugh! I wish to mercy I could. Don’t
you know I am going to accept you—I, who don’t know what love
means—I, who am, I believe, sexless, don’t you know you’re mad
and don’t you think it’s rather degrading to give all you offer me for
nothing? After all, it is not absolutely necessary to my salvation that
I should make experiments on you.”
She felt a sudden tiredness come on her and nestled back in her
cushions.
“I am ready to take you with open eyes, Gwen; you are very
honest, dear; you will lose some of that when you have suffered a
little,” he added, with a ring of sadness in his voice, as he looked
tenderly down on her.
She raised her head quickly. “Suffer! Why should I suffer?”
He watched her for a minute with sombre eyes.
“I don’t know,” he said half-absently, “but you will. Then this is our
betrothal, is it, dear?”
She bowed her head.
“Oh, my darling!” he said suddenly.
 “Will he often say it?” she thought curiously, “can I stand this?”
“My darling, you have no idea how I shall enjoy giving you lessons
in love.”
“Will you?” she said grimly, “I doubt it, I tell you I have no taste
for the cult. Well, it is at least fortunate that one can be honest and
that it isn’t necessary for me to befool you for the sake of your
income. This marriage is the very perfection of an alliance from all
such points of view, and yet—do you know, Sir Humphrey, I wish
quite intensely, we were both of us in another position, in quite a
low, unknown one, then we need not marry. Engagements are
nothing; I know as much of you now as any engagement can teach
me. We might then try a preliminary experiment as to how life
together goes; if it did not do, we might each go our own way and
bury the past. I never wished for such a thing before, it follows, I
suppose,” she added with a mirthless little laugh, “that I care this
much for you or for my experiment. Have you grasped the whole
situation?” she demanded, turning her troubled eyes full on him.
“My child, you have been very explicit, I think I have quite
grasped it. When will you marry me?”
She gave a little start.
“I was wondering,” she said at last, “if this was final?”
“It is final,” he said, “you know it is.”
“Yes, I know; it was rather paltry to pretend I didn’t—oh!—”
She looked up at him with her face held in both her hands. “Final?
yes, so it is. I am one section of a puzzle moved by fate, you’re
another. It is humiliating when one comes to think of it.”
“Well?”
“I will marry you when you like.”
“The end of next month?”
“Won’t it interfere with the shooting?”
“I had forgotten that—I don’t think I shall mind—the end of July,
then.”
 He took her hands and kissed them, and he thought as he got out
into the street that he had felt them tremble. It was a pleasant
surprise, on which he felt inclined to congratulate himself.
The knowledge had a quite other effect on his betrothed. She
smote her clenched fists angrily together and scorned herself for the
feebleness of her extremities.
“Mean deceitful wretch,” she cried, “to mislead that man, when I
am only tired and wanting my tea!”