Download the full version of the ebook at

https://ebookultra.com

Mastering Machine Learning with scikit learn

2nd edition Gavin Hackeling

https://ebookultra.com/download/mastering-machine-
learning-with-scikit-learn-2nd-edition-gavin-
hackeling/

Explore and download more ebook at https://ebookultra.com

 Recommended digital products (PDF, EPUB, MOBI) that
you can download immediately if you are interested.

Machine Learning with R Cookbook 2nd Edition Bhatia

https://ebookultra.com/download/machine-learning-with-r-cookbook-2nd-
edition-bhatia/

ebookultra.com

A First Course in Machine Learning Chapman Hall CRC

Machine Learning Pattern Recognition 2nd Edition Simon
Rogers
https://ebookultra.com/download/a-first-course-in-machine-learning-
chapman-hall-crc-machine-learning-pattern-recognition-2nd-edition-
simon-rogers/
ebookultra.com

Machine Learning for Time Series Forecasting with Python

1st Edition Francesca Lazzeri

https://ebookultra.com/download/machine-learning-for-time-series-
forecasting-with-python-1st-edition-francesca-lazzeri/

ebookultra.com

Learning styles and inclusion 1st Edition Gavin Reid

https://ebookultra.com/download/learning-styles-and-inclusion-1st-
edition-gavin-reid-2/

ebookultra.com
Learning Styles and Inclusion 1st Edition Gavin Reid

https://ebookultra.com/download/learning-styles-and-inclusion-1st-
edition-gavin-reid/

ebookultra.com

Machine Learning with R Expert Techniques for Predictive

Modeling to Solve All Your Data Analysis Problems 2nd
Edition Brett Lantz
https://ebookultra.com/download/machine-learning-with-r-expert-
techniques-for-predictive-modeling-to-solve-all-your-data-analysis-
problems-2nd-edition-brett-lantz/
ebookultra.com

Python Machine Learning By Example The Easiest Way to Get

into Machine Learning 1st Edition Yuxi (Hayden) Liu

https://ebookultra.com/download/python-machine-learning-by-example-
the-easiest-way-to-get-into-machine-learning-1st-edition-yuxi-hayden-
liu/
ebookultra.com

Bayesian reasoning and machine learning Barber D.

https://ebookultra.com/download/bayesian-reasoning-and-machine-
learning-barber-d/

ebookultra.com

Data Mining Practical Machine Learning Tools and

Techniques 2nd Edition Ian H. Witten

https://ebookultra.com/download/data-mining-practical-machine-
learning-tools-and-techniques-2nd-edition-ian-h-witten/

ebookultra.com
Mastering Machine Learning with scikit learn 2nd edition
Gavin Hackeling Digital Instant Download
Author(s): Gavin Hackeling
ISBN(s): 9781788299879, 1788299876
File Details: PDF, 6.30 MB
Language: english
Mastering Machine Learning
with scikit-learn
Second Edition

-FBSOUPJNQMFNFOUBOEFWBMVBUFNBDIJOFMFBSOJOHTPMVUJPOT
XJUITDJLJUMFBSO

Gavin Hackeling

BIRMINGHAM - MUMBAI
Mastering Machine Learning with scikit-learn
Second Edition

Copyright © 2017 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or
transmitted in any form or by any means, without the prior written permission of the
publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the
information presented. However, the information contained in this book is sold without
warranty, either express or implied. Neither the author, nor Packt Publishing, and its
dealers and distributors will be held liable for any damages caused or alleged to be caused
directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the
companies and products mentioned in this book by the appropriate use of capitals.
However, Packt Publishing cannot guarantee the accuracy of this information.

First published: October 2014ȱ

econd published: July 2017ȱ

Production reference: 1200717

1VCMJTIFECZ1BDLU1VCMJTIJOH-UE
-JWFSZ1MBDF
-JWFSZ4USFFU
#JSNJOHIBN
#1#6,
ISBN 978-1-78829-987-9
XXXQBDLUQVCDPN
 Credits

Author Copy Editors

Gavin Hackeling Safis Editing
Vikrant Phadkay

Reviewer Project Coordinator

Oleg Okun Nidhi Joshi

Commissioning Editor Proofreader

Amey Varangaonkar Safis Editing

Acquisition Editor Indexer

Aman Singh Tejal Daruwale Soni

Content Development Editor Graphics

Aishwarya Pandere Tania Dutta

Technical Editor Production Coordinator

Suwarna Patil Arvindkumar Gupta
 About the Author
Gavin Hackeling is a data scientist and author. He was worked on a variety of machine
learning problems, including automatic speech recognition, document classification, object
recognition, and semantic segmentation. An alumnus of the University of North Carolina
and New York University, he lives in Brooklyn with his wife and cat.

I would like to thank my wife, Hallie, and the scikti-learn community.

 About the Reviewer
Oleg Okun is a machine learning expert and an author/editor of four books, numerous
journal articles, and conference papers. His career spans more than a quarter of a century.
He was employed in both academia and industry in his motherland, Belarus, and abroad
(Finland, Sweden, and Germany). His work experience includes document image analysis,
fingerprint biometrics, bioinformatics, online/offline marketing analytics, credit scoring
analytics, and text analytics.

He is interested in all aspects of distributed machine learning and the Internet of Things.
Oleg currently lives and works in Hamburg, Germany.

I would like to express my deepest gratitude to my parents for everything that they have
done for me.
 www.PacktPub.com
For support files and downloads related to your book, please visitXXX1BDLU1VCDPN.

Did you know that Packt offers eBook versions of every book published, with PDF and
ePub files available? You can upgrade to the eBook version at XXX1BDLU1VCDPNand as a
print book customer, you are entitled to a discount on the eBook copy. Get in touch with us
at TFSWJDF!QBDLUQVCDPN for more details.

At XXX1BDLU1VCDPN, you can also read a collection of free technical articles, sign up for a
range of free newsletters and receive exclusive discounts and offers on Packt books and
eBooks.

IUUQTXXXQBDLUQVCDPNNBQU
                       

Get the most in-demand software skills with Mapt. Mapt gives you full access to all Packt
books and video courses, as well as industry-leading tools to help you plan your personal
development and advance your career.

Why subscribe?
Fully searchable across every book published by Packt
Copy and paste, print, and bookmark content
On demand and accessible via a web browser
 Customer Feedback
Thanks for purchasing this Packt book. At Packt, quality is at the heart of our editorial
process. To help us improve, please leave us an honest review on this book's Amazon page
at IUUQTXXXBNB[PODPNEQ.
                    

If you'd like to join our team of regular reviewers, you can e-mail us at
DVTUPNFSSFWJFXT!QBDLUQVCDPN. We award our regular reviewers with free eBooks and
videos in exchange for their valuable feedback. Help us be relentless in improving our
products!
 Table of Contents
Preface 1
Chapter 1: The Fundamentals of Machine Learning 6
Defining machine learning 6
Learning from experience 8
Machine learning tasks 9
Training data, testing data, and validation data 10
Bias and variance 13
An introduction to scikit-learn 15
Installing scikit-learn 16
Installing using pip 17
Installing on Windows 17
Installing on Ubuntu 16.04 17
Installing on Mac OS 17
Installing Anaconda 18
Verifying the installation 18
Installing pandas, Pillow, NLTK, and matplotlib 18
Summary 19
Chapter 2: Simple Linear Regression 20
Simple linear regression 20
Evaluating the fitness of the model with a cost function 25
Solving OLS for simple linear regression 27
Evaluating the model 29
Summary 31
Chapter 3: Classification and Regression with k-Nearest Neighbors 32
K-Nearest Neighbors 32
Lazy learning and non-parametric models 33
Classification with KNN 34
Regression with KNN 42
Scaling features 44
Summary 47
Chapter 4: Feature Extraction 48
Extracting features from categorical variables 48
Standardizing features 49
 Extracting features from text 50
The bag-of-words model 50
Stop word filtering 53
Stemming and lemmatization 54
Extending bag-of-words with tf-idf weights 57
Space-efficient feature vectorizing with the hashing trick 59
Word embeddings 61
Extracting features from images 64
Extracting features from pixel intensities 65
Using convolutional neural network activations as features 66
Summary 68
Chapter 5: From Simple Linear Regression to Multiple Linear
Regression 70
Multiple linear regression 70
Polynomial regression 74
Regularization 79
Applying linear regression 80
Exploring the data 81
Fitting and evaluating the model 84
Gradient descent 86
Summary 90
Chapter 6: From Linear Regression to Logistic Regression 91
Binary classification with logistic regression 92
Spam filtering 94
Binary classification performance metrics 95
Accuracy 97
Precision and recall 98
Calculating the F1 measure 99
ROC AUC 100
Tuning models with grid search 102
Multi-class classification 104
Multi-class classification performance metrics 107
Multi-label classification and problem transformation 108
Multi-label classification performance metrics 113
Summary 114
Chapter 7: Naive Bayes 115
Bayes' theorem 115
Generative and discriminative models 117

[ ii ]
 Naive Bayes 118
Assumptions of Naive Bayes 119
Naive Bayes with scikit-learn 120
Summary 124
Chapter 8: Nonlinear Classification and Regression with Decision
Trees 125
Decision trees 125
Training decision trees 127
Selecting the questions 128
Information gain 131
Gini impurity 136
Decision trees with scikit-learn 137
Advantages and disadvantages of decision trees 139
Summary 140
Chapter 9: From Decision Trees to Random Forests and Other
Ensemble Methods 141
Bagging 141
Boosting 144
Stacking 146
Summary 148
Chapter 10: The Perceptron 149
The perceptron 149
Activation functions 150
The perceptron learning algorithm 152
Binary classification with the perceptron 153
Document classification with the perceptron 161
Limitations of the perceptron 162
Summary 163
Chapter 11: From the Perceptron to Support Vector Machines 164
Kernels and the kernel trick 165
Maximum margin classification and support vectors 169
Classifying characters in scikit-learn 172
Classifying handwritten digits 172
Classifying characters in natural images 175
Summary 177
Chapter 12: From the Perceptron to Artificial Neural Networks 178
Nonlinear decision boundaries 179

[ iii ]
 Feed-forward and feedback ANNs 180
Multi-layer perceptrons 181
Training multi-layer perceptrons 183
Backpropagation 184
Training a multi-layer perceptron to approximate XOR 189
Training a multi-layer perceptron to classify handwritten digits 192
Summary 193
Chapter 13: K-means 194
Clustering 194
K-means 197
Local optima 203
Selecting K with the elbow method 204
Evaluating clusters 207
Image quantization 209
Clustering to learn features 211
Summary 214
Chapter 14: Dimensionality Reduction with Principal Component
Analysis 215
Principal component analysis 215
Variance, covariance, and covariance matrices 220
Eigenvectors and eigenvalues 222
Performing PCA 224
Visualizing high-dimensional data with PCA 227
Face recognition with PCA 228
Summary 231
Index 233

[ iv ]
 Preface
In recent years, popular imagination has become fascinated by machine learning. The
discipline has found a variety of applications. Some of these applications, such as spam
filtering, are ubiquitous and have been rendered mundane by their successes. Many other
applications have only recently been conceived, and hint at machine learning's potential.

In this book, we will examine several machine learning models and learning algorithms. We
will discuss tasks that machine learning is commonly applied to, and we will learn to
measure the performance of machine learning systems. We will work with a popular library
for the Python programming language called scikit-learn, which has assembled state-of-the-
art implementations of many machine learning algorithms under an intuitive and versatile
API.

What this book covers

$IBQUFS, The Fundamentals of Machine Learning, defines machine learning as the study and
design of programs that improve their performance of a task by learning from experience.
This definition guides the other chapters; in each, we will examine a machine learning
model, apply it to a task, and measure its performance.

$IBQUFS, Simple Linear Regression, discusses a model that relates a single feature to a
continuous response variable. We will learn about cost functions and use the normal
equation to optimize the model.

$IBQUFS, Classification and Regression with K-Nearest Neighbors, introduces a simple,

nonlinear model for classification and regression tasks.

$IBQUFS, Feature Extraction, describes methods for representing text, images, and
categorical variables as features that can be used in machine learning models.

Chapter 5, From Simple Linear Regression to Multiple Linear Regression, discusses a

generalization of simple linear regression that regresses a continuous response variable
onto multiple features.

$IBQUFS, From Linear Regression to Logistic Regression, further generalizes multiple linear
regression and introduces a model for binary classification tasks.
Preface

$IBQUFS, Naive Bayes, discusses Bayes’ theorem and the Naive Bayes family of classifiers,
and compares generative and discriminative models.

$IBQUFS, Nonlinear Classification and Regression with Decision Trees, introduces the decision
tree, a simple, nonlinear model for classification and regression tasks.

$IBQUFS, From Decision Trees to Random Forests and other Ensemble Methods, discusses three
methods for combining models called bagging, boosting, and stacking.

$IBQUFS, The Perceptron, introduces a simple online model for binary classification.

$IBQUFS, From the Perceptron to Support Vector Machines, discusses a powerful,

discriminative model for classification and regression called the support vector machine,
and a technique for efficiently projecting features to higher dimensional spaces.

$IBQUFS, From the Perceptron to Artificial Neural Networks, introduces powerful nonlinear
models for classification and regression built from graphs of artificial neurons.

$IBQUFS, K-means, discusses an algorithm that can be used to find structures in

unlabeled data.

$IBQUFS, Dimensionality Reduction with Principal Component Analysis, describes a method

for reducing the dimensions of data that can mitigate the curse of dimensionality.

What you need for this book

The examples in this book require Python >= 2.7 or >= 3.3 and pip, the PyPA recommended
tool for installing Python packages. The examples are intended to be executed in a Jupyter
notebook or an IPython interpreter. $IBQUFS, The Fundamentals of Machine Learning shows
how to install scikit-learn 0.18.1, its dependencies, and other libraries on Ubuntu, Mac OS,
and Windows.

Who this book is for

This book is intended for software engineers who want to understand how common
machine learning algorithms work and develop an intuition for how to use them. It is also
for data scientists who want to learn about the scikit-learn API. Familiarity with machine
learning fundamentals and Python is helpful but not required.

[2]
Preface

Conventions
In this book, you will find a number of text styles that distinguish between different kinds
of information. Here are some examples of these styles and an explanation of their meaning.

Code words in text, database table names, folder names, filenames, file extensions,
pathnames, dummy URLs, user input, and Twitter handles are shown as follows: "The
package is named TLMFBSO because scikit-learn is not a valid Python package name."
*O<>
JNQPSUTLMFBSO
TLMFBSO@@WFSTJPO@@

0VU<>


New terms and important words are shown in bold.

Warnings or important notes appear like this.

Tips and tricks appear like this.

Reader feedback
Feedback from our readers is always welcome. Let us know what you thought about this
book-what you liked or disliked. Reader feedback is important for us as it helps us to
develop titles that you will really get the most out of. To send us general feedback, simply
email GFFECBDL!QBDLUQVCDPN, and mention the book's title in the subject of your
message. If there is a topic that you have expertise in and you are interested in either
writing or contributing to a book, see our author guide at XXXQBDLUQVCDPNBVUIPST.

[3]
Preface

Customer support
Now that you are the proud owner of a Packt book, we have a number of things to help you
to get the most from your purchase.

Downloading the example code

You can download the example code files for this book from your account at IUUQXXXQ        

BDLUQVCDPN. If you purchased this book elsewhere, you can visit IUUQXXXQBDLUQVCD
                           

PNTVQQPSUand register to have the files e-mailed directly to you. You can download the
        

code files by following these steps:

1. Log in or register to our website using your e-mail address and password.
2. Hover the mouse pointer on the SUPPORT tab at the top.
3. Click on Code Downloads & Errata.
4. Enter the name of the book in the Search box.
5. Select the book for which you're looking to download the code files.
6. Choose from the drop-down menu where you purchased this book from.
7. Click on Code Download.

Once the file is downloaded, please make sure that you unzip or extract the folder using the
latest version of:

WinRAR / 7-Zip for Windows

Zipeg / iZip / UnRarX for Mac
7-Zip / PeaZip for Linux

The code bundle for the book is also hosted on GitHub at IUUQTHJUIVCDPN1BDLU1VCM                       

JTIJOH.BTUFSJOH.BDIJOF-FBSOJOHXJUITDJLJUMFBSO4FDPOE&EJUJPO. We also
                                                                

have other code bundles from our rich catalog of books and videos available at IUUQTHJ      

UIVCDPN1BDLU1VCMJTIJOH. Check them out!

                      

[4]
Preface

Errata
Although we have taken care to ensure the accuracy of our content, mistakes do happen. If
you find a mistake in one of our books-maybe a mistake in the text or the code-we would be
grateful if you could report this to us. By doing so, you can save other readers from
frustration and help us to improve subsequent versions of this book. If you find any errata,
please report them by visiting IUUQXXXQBDLUQVCDPNTVCNJUFSSBUB, selecting your
                                

book, clicking on the Errata Submission Form link, and entering the details of your errata.
Once your errata are verified, your submission will be accepted and the errata will be
uploaded to our website or added to any list of existing errata under the Errata section of
that title. To view the previously submitted errata, go to IUUQTXXXQBDLUQVCDPNCPPL                        

TDPOUFOUTVQQPSUand enter the name of the book in the search field. The required
              

information will appear under the Errata section.

Piracy
Piracy of copyrighted material on the Internet is an ongoing problem across all media. At
Packt, we take the protection of our copyright and licenses very seriously. If you come
across any illegal copies of our works in any form on the Internet, please provide us with
the location address or website name immediately. Please contact us at
DPQZSJHIU!QBDLUQVCDPN with a link to the suspected pirated material. We appreciate
your help in protecting our authors and our ability to bring you valuable content.

Questions
If you have a problem with any aspects of this book, you can contact us at
RVFTUJPOT!QBDLUQVCDPN, and we will do our best to address it.

[5]
 The Fundamentals of Machine
1
Learning
In this chapter, we will review fundamental concepts in machine learning. We will compare
supervised and unsupervised learning; discuss the uses of training, testing, and validation
data; and describe applications of machine learning. Finally, we will introduce scikit-learn,
and install the tools required in subsequent chapters.

Defining machine learning

Our imaginations have long been captivated by visions of machines that can learn and
imitate human intelligence. While machines capable of general artificial intelligence-like
Arthur C. Clarke's HAL and Isaac Asimov's Sonny-have yet to be realized, software
programs that can acquire new knowledge and skills through experience are becoming
increasingly common. We use such machine learning programs to discover new music that
we might enjoy, and to find exactly the shoes we want to purchase online. Machine learning
programs allow us to dictate commands to our smart phones, and allow our thermostats to
set their own temperatures. Machine learning programs can decipher sloppily-written
mailing addresses better than humans, and can guard credit cards from fraud more
vigilantly. From investigating new medicines to estimating the page views for versions of a
headline, machine learning software is becoming central to many industries. Machine
learning has even encroached on activities that have long been considered uniquely human,
such as writing the sports column recapping the Duke basketball team's loss to UNC.
The Fundamentals of Machine Learning

Machine learning is the design and study of software artifacts that use past experience to
inform future decisions; machine learning is the study of programs that learn from data.
The fundamental goal of machine learning is to generalize, or to induce an unknown rule
from examples of the rule's application. The canonical example of machine learning is spam
filtering. By observing thousands of emails that have been previously labeled as either spam
or ham, spam filters learn to classify new messages. Arthur Samuel, a computer scientist
who pioneered the study of artificial intelligence, said that machine learning is the "study
that gives computers the ability to learn without being explicitly programmed". Throughout
the 1950s and 1960s, Samuel developed programs that played checkers. While the rules of
checkers are simple, complex strategies are required to defeat skilled opponents. Samuel
never explicitly programmed these strategies, but through the experience of playing
thousands of games, the program learned complex behaviors that allowed it to beat many
human opponents.

A popular quote from computer scientist Tom Mitchell defines machine learning more
formally: "A program can be said to learn from experience 'E' with respect to some class of
tasks 'T' and performance measure 'P', if its performance at tasks in 'T', as measured by 'P',
improves with experience 'E'." For example, assume that you have a collection of pictures.
Each picture depicts either a dog or a cat. A task could be sorting the pictures into separate
collections of dog and cat photos. A program could learn to perform this task by observing
pictures that have already been sorted, and it could evaluate its performance by calculating
the percentage of correctly classified pictures.

We will use Mitchell's definition of machine learning to organize this chapter. First, we will
discuss types of experience, including supervised learning and unsupervised learning.
Next, we will discuss common tasks that can be performed by machine learning systems.
Finally, we will discuss performance measures that can be used to assess machine learning
systems.

[7]
The Fundamentals of Machine Learning

Learning from experience

Machine learning systems are often described as learning from experience either with or
without supervision from humans. In supervised learning problems, a program predicts an
output for an input by learning from pairs of labeled inputs and outputs. That is, the
program learns from examples of the "right answers". In unsupervised learning, a program
does not learn from labeled data. Instead, it attempts to discover patterns in data. For
example, assume that you have collected data describing the heights and weights of people.
An example of an unsupervised learning problem is dividing the data points into groups. A
program might produce groups that correspond to men and women, or children and adults.
Now assume that the data is also labeled with the person's sex. An example of a supervised
learning problem is to induce a rule for predicting whether a person is male or female based
on his or her height and weight. We will discuss algorithms and examples of supervised
and unsupervised learning in the following chapters.

Supervised learning and unsupervised learning can be thought of as occupying opposite

ends of a spectrum. Some types of problem, called semi-supervised learning problems,
make use of both supervised and unsupervised data; these problems are located on the
spectrum between supervised and unsupervised learning. Reinforcement learning is
located near the supervised end of the spectrum. Unlike supervised learning, reinforcement
learning programs do not learn from labeled pairs of inputs and outputs. Instead, they
receive feedback for their decisions, but errors are not explicitly corrected. For example, a
reinforcement learning program that is learning to play a side-scrolling video game like
Super Mario Bros may receive a reward when it completes a level or exceeds a certain score,
and a punishment when it loses a life. However, this supervised feedback is not associated
with specific decisions to run, avoid Goombas, or pick up fire flowers. We will focus
primarily on supervised and unsupervised learning, as these categories include most
common machine learning problems. In the next sections, we will review supervised and
unsupervised learning in more detail.

A supervised learning program learns from labeled examples of the outputs that should be
produced for an input. There are many names for the output of a machine learning
program. Several disciplines converge in machine learning, and many of those disciplines
use their own terminology. In this book, we will refer to the output as the response
variable. Other names for response variables include "dependent variables", "regressands",
"criterion variables", "measured variables", "responding variables", "explained variables",
"outcome variables", "experimental variables", "labels", and "output variables". Similarly, the
input variables have several names. In this book, we will refer to inputs as features, and the
phenomena they represent as explanatory variables. Other names for explanatory variables
include "predictors", "regressors", "controlled variables", and "exposure variables". Response
variables and explanatory variables may take real or discrete values.

[8]
The Fundamentals of Machine Learning

The collection of examples that comprise supervised experience is called a training set. A
collection of examples that is used to assess the performance of a program is called a test
set. The response variable can be thought of as the answer to the question posed by the
explanatory variables; supervised learning problems learn from a collection of answers to
different questions. That is, supervised learning programs are provided with the correct
answers and must learn to respond correctly to unseen, but similar, questions.

Machine learning tasks

Two of the most common supervised machine learning tasks are classification and
regression. In classification tasks, the program must learn to predict discrete values for one
or more response variables from one or more features. That is, the program must predict the
most probable category, class, or label for new observations. Applications of classification
include predicting whether a stock's price will rise or fall, or deciding whether a news
article belongs to the politics or leisure sections. In regression problems, the program must
predict the values of one more or continuous response variables from one or more features.
Examples of regression problems include predicting the sales revenue for a new product, or
predicting the salary for a job based on its description. Like classification, regression
problems require supervised learning.

A common unsupervised learning task is to discover groups of related observations, called

clusters, within the dataset. This task, called clustering or cluster analysis, assigns
observations into groups such that observations within a groups are more similar to each
other based on some similarity measure than they are to observations in other groups.
Clustering is often used to explore a dataset. For example, given a collection of movie
reviews, a clustering algorithm might discover the sets of positive and negative reviews.
The system will not be able to label the clusters as positive or negative; without
supervision, it will only have knowledge that the grouped observations are similar to each
other by some measure. A common application of clustering is discovering segments of
customers within a market for a product. By understanding what attributes are common to
particular groups of customers, marketers can decide what aspects of their campaigns to
emphasize. Clustering is also used by internet radio services; given a collection of songs, a
clustering algorithm might be able to group the songs according to their genres. Using
different similarity measures, the same clustering algorithm might group the songs by their
keys, or by the instruments they contain.

[9]
The Fundamentals of Machine Learning

Dimensionality reduction is another task that is commonly accomplished using

unsupervised learning. Some problems may contain thousands or millions of features,
which can be computationally costly to work with. Additionally, the program's ability to
generalize may be reduced if some of the features capture noise or are irrelevant to the
underlying relationship. Dimensionality reduction is the process of discovering the features
that account for the greatest changes in the response variable. Dimensionality reduction can
also be used to visualize data. It is easy to visualize a regression problem such as predicting
the price of a home from its size; the size of the home can be plotted on the graph's x axis,
and the price of the home can be plotted on the y axis. It is similarly easy to visualize the
housing price regression problem when a second feature is added; the number of
bathrooms in the house could be plotted on the z axis, for instance. A problem with
thousands of features, however, becomes impossible to visualize.

Training data, testing data, and validation

data
As mentioned previously, a training set is a collection of observations. These observations
comprise the experience that the algorithm uses to learn. In supervised learning problems,
each observation consists of an observed response variable and features of one or more
observed explanatory variables. The test set is a similar collection of observations. The test
set is used to evaluate the performance of the model using some performance metric. It is
important that no observations from the training set are included in the test set. If the test
set does contain examples from the training set, it will be difficult to assess whether the
algorithm has learned to generalize from the training set or has simply memorized it. A
program that generalizes well will be able to effectively perform a task with new data. In
contrast, a program that memorizes the training data by learning an overly-complex model
could predict the values of the response variable for the training set accurately, but will fail
to predict the value of the response variable for new examples. Memorizing the training set
is called overfitting. A program that memorizes its observations may not perform its task
well, as it could memorize relations and structure that are coincidental in the training data.
Balancing generalization and memorization is a problem common to many machine
learning algorithms. In later chapters we will discuss regularization, which can be applied
to many models to reduce over-fitting.

[ 10 ]
The Fundamentals of Machine Learning

In addition to the training and test data, a third set of observations, called a validation or
hold-out set, is sometimes required. The validation set is used to tune variables called
hyperparameters that control how the algorithm learns from the training data. The
program is still evaluated on the test set to provide an estimate of its performance in the real
world. The validation set should not be used to estimate real-world performance because
the program has been tuned to learn from the training data in a way that optimizes its score
on the validation data; the program will not have this advantage in the real world.

It is common to partition a single set of supervised observations into training, validation,

and test sets. There are no requirements for the sizes of the partitions, and they may vary
according to the amount of data available. It is common to allocate between fifty and
seventy-five percent of the data to the training set, ten to twenty-five percent of the data to
the test set, and the remainder to the validation set.

Some training sets may contain only a few hundred observations; others may include
millions. Inexpensive storage, increased network connectivity, and the ubiquity of sensor-
packed smartphones have contributed to the contemporary state of big data, or training sets
with millions or billions of examples. While this book will not work with datasets that
require parallel processing on tens or hundreds of computers, the predictive power of many
machine learning algorithms improves as the amount of training data increases. However,
machine learning algorithms also follow the maxim "garbage in, garbage out". A student
who studies for a test by reading a large, confusing textbook that contains many errors
likely will not score better than a student who reads a short but well-written textbook.
Similarly, an algorithm trained on a large collection of noisy, irrelevant, or incorrectly-
labeled data will not perform better than an algorithm trained on a smaller set of data that is
more representative of the problem in the real-world.

Many supervised training sets are prepared manually or by semi-automated processes.

Creating a large collection of supervised data can be costly in some domains. Fortunately,
several datasets are bundled with scikit-learn, allowing developers to focus on
experimenting with models instead. During development, and particularly when training
data is scarce, a practice called cross-validation can be used to train and validate a model
on the same data. In cross-validation, the training data is partitioned. The model is trained
using all but one of the partitions, and tested on the remaining partition. The partitions are
then rotated several times so that the model is trained and evaluated on all of the data. The
mean of the model's scores on each of the partitions is a better estimate of performance in
the real world than an evaluation using a single training/testing split. The following
diagram depicts cross validation with five partitions, or folds.

[ 11 ]
The Fundamentals of Machine Learning

The original dataset is partitioned into five subsets of equal size labeled A through E.
Initially the model is trained on partitions B through E, and tested on partition A. In the
next iteration, the model is trained on partitions A, C, D, and E, and tested on partition B.
The partitions are rotated until models have been trained and tested on all of the partitions.
Cross-validation provides a more accurate estimate of the model's performance than testing
a single partition of the data.

[ 12 ]
The Fundamentals of Machine Learning

Bias and variance

Many metrics can be used to measure whether or not a program is learning to perform its
task more effectively. For supervised learning problems, many performance metrics
measure the amount of prediction error. There are two fundamental causes of prediction
error: a model's bias, and its variance. Assume that you have many training sets that are all
unique, but equally representative of the population. A model with high bias will produce
similar errors for an input regardless of the training set it used to learn; the model biases its
own assumptions about the real relationship over the relationship demonstrated in the
training data. A model with high variance, conversely, will produce different errors for an
input depending on the training set that it used to learn. A model with high bias is
inflexible, but a model with high variance may be so flexible that it models the noise in the
training set. That is, a model with high variance over-fits the training data, while a model
with high bias under-fits the training data. It can be helpful to visualize bias and variance as
darts thrown at a dartboard. Each dart is analogous to a prediction, and is thrown by a
model trained on a different dataset every time. A model with high bias but low variance
will throw darts that will be tightly clustered, but could be far from the bulls-eye. A model
with high bias and high variance will throw darts all over the board; the darts are far from
the bulls-eye and from each other. A model with low bias and high variance will throw
darts that could be poorly clustered but close to the bulls-eye. Finally, a model with low
bias and low variance will throw darts that are tightly clustered around the bulls-eye.

[ 13 ]
The Fundamentals of Machine Learning

Ideally, a model will have both low bias and variance, but efforts to decrease one will
frequently increase the other. This is known as the bias-variance trade-off. We will discuss
the biases and variances of many of the models introduced in this book.

Unsupervised learning problems do not have an error signal to measure; instead,

performance metrics for unsupervised learning problems measure some attribute of the
structure discovered in the data, such as the distances within and between clusters.

Most performance measures can only be calculated for a specific type of task, like
classification or regression. Machine learning systems should be evaluated using
performance measures that represent the costs associated with making errors in the real
world. While this may seem obvious, the following example describes this using a
performance measure that is appropriate for the task in general but not for its specific
application.

Consider a classification task in which a machine learning system observes tumors and
must predict whether they are malignant or benign. Accuracy, or the fraction of instances
that were classified correctly, is an intuitive measure of the program's performance. While
accuracy does measure the program's performance, it does not differentiate between
malignant tumors that were classified as being benign, and benign tumors that were
classified as being malignant. In some applications, the costs associated with all types of
errors may be the same. In this problem, however, failing to identify malignant tumors is
likely a more severe error than mistakenly classifying benign tumors as being malignant.

We can measure each of the possible prediction outcomes to create different views of the
classifier's performance. When the system correctly classifies a tumor as being malignant,
the prediction is called a true positive. When the system incorrectly classifies a benign
tumor as being malignant, the prediction is a false positive. Similarly, a false negative is an
incorrect prediction that the tumor is benign, and a true negative is a correct prediction that
a tumor is benign. Note that positive and negative are used only as binary labels, and are
not meant to judge the phenomena they signify. In this example, it does not matter whether
malignant tumors are coded as positive or negative, so long as they are coded consistently.
True and false positives and negatives can be used to calculate several common measures of
classification performance, including accuracy, precision and recall.

Accuracy is calculated with the following formula, where TP is the number of true
positives, TN is the number of true negatives, FP is the number of false positives, and FN is
the number of false negatives:

[ 14 ]
The Fundamentals of Machine Learning

Precision is the fraction of the tumors that were predicted to be malignant that are actually
malignant. Precision is calculated with the following formula:

Recall is the fraction of malignant tumors that the system identified. Recall is calculated
with the following formula:

In this example, precision measures the fraction of tumors that were predicted to be
malignant that are actually malignant. Recall measures the fraction of truly malignant
tumors that were detected.

The precision and recall measures could reveal that a classifier with impressive accuracy
actually fails to detect most of the malignant tumors. If most tumors in the testing set are
benign, even a classifier that never predicts malignancy could have high accuracy. A
different classifier with lower accuracy and higher recall might be better suited to the task,
since it will detect more of the malignant tumors.

Many other performance measures for classification can be used. We will discuss more
metrics, including metrics for multi-label classification problems, in later chapters. In the
next chapter we will discuss some common performance measures for regression tasks.
Performance on unsupervised tasks can also be assessed; we will discuss some performance
measures for cluster analysis later in the book.

An introduction to scikit-learn
Since its release in 2007, scikit-learn has become one of the most popular machine learning
libraries. scikit-learn provides algorithms for machine learning tasks including
classification, regression, dimensionality reduction, and clustering. It also provides modules
for pre-processing data, extracting features, optimizing hyperparameters, and evaluating
models.

scikit-learn is built on the popular Python libraries NumPy and SciPy. NumPy extends
Python to support efficient operations on large arrays and multi-dimensional matrices.
SciPy provides modules for scientific computing. The visualization library matplotlib is
often used in conjunction with scikit-learn.

[ 15 ]
The Fundamentals of Machine Learning

scikit-learn is popular for academic research because its API is well-documented, easy-to-
use, and versatile. Developers can use scikit-learn to experiment with different algorithms
by changing only a few lines of code. scikit-learn wraps some popular implementations of
machine learning algorithms, such as LIBSVM and LIBLINEAR. Other Python libraries,
including NLTK, include wrappers for scikit-learn. scikit-learn also includes a variety of
datasets, allowing developers to focus on algorithms rather than obtaining and cleaning
data.

Licensed under the permissive BSD license, scikit-learn can be used in commercial
applications without restrictions. Many of scikit-learn's algorithms are fast and scalable to
all but massive datasets. Finally, scikit-learn is noted for its reliability; much of the library is
covered by automated tests.

Installing scikit-learn
This book was written for version 0.18.1 of scikit-learn; use this version to ensure that the
examples run correctly. If you have previously installed scikit-learn, you can retrieve the
version number by executing the following in a notebook or Python interpreter:
*O<>
JNQPSUTLMFBSO
TLMFBSO@@WFSTJPO@@

0VU<>


The package is named TLMFBSO because scikit-learn is not a valid Python

package name.

If you have not previously installed scikit-learn, you may install it from a package manager
or build it from source. We will review the installation processes for Ubuntu 16.04, Max OS,
and Windows 10 in the following sections, but refer to IUUQTDJLJUMFBSOPSHTUBCMF                         

JOTUBMMIUNMfor the latest instructions. The following instructions assume only that you
           

have installed Python >= 2.6 or Python >= 3.3. See IUUQXXXQZUIPOPSHEPXOMPBE for
                         

instructions on installing Python.

[ 16 ]
The Fundamentals of Machine Learning

Installing using pip

The easiest way to install scikit-learn is to use QJQ, the PyPA-recommended tool for
installing Python packages. Install scikit-learn using QJQ as follows:
$ pip install -U scikit-learn

If pip is not available on your system, consult the following sections for installation
instructions for various platforms.

Installing on Windows
scikit-learn requires setuptools, a third-party package that supports packaging and
installing software for Python. Setuptools can be installed on Windows by running the
bootstrap script at IUUQTCJUCVDLFUPSHQZQBTFUVQUPPMTSBXCPPUTUSBQF[@TFUVQ                                                    

QZ.
 

Windows binaries for the 32-bit and 64-bit versions of scikit-learn are also available. If you
cannot determine which version you need, install the 32-bit version. Both versions depend
on NumPy 1.3 or newer. The 32-bit version of NumPy can be downloaded from IUUQTPV      

SDFGPSHFOFUQSPKFDUTOVNQZGJMFT/VN1Z. The 64-bit version can be downloaded

                                  

from IUUQXXXMGEVDJFEV_HPIMLFQZUIPOMJCTTDJLJUMFBSO.
                                              

A Windows installer for the 32-bit version of scikit-learn can be downloaded from IUUQT    

PVSDFGPSHFOFUQSPKFDUTTDJLJUMFBSOGJMFT. An installer for the 64-bit version of

                                      

scikit-learn can be downloaded from IUUQXXXMGEVDJFEV_HPIMLFQZUIPOMJCTTD                                    

JLJUMFBSO.
         

Installing on Ubuntu 16.04

scikit-learn can be installed on Ubuntu 16.04 using BQU.
$ sudo apt install python-scikits-learn

Installing on Mac OS
scikit-learn can be installed on OS X using Macports.
$ sudo port install py27-sklearn

[ 17 ]
The Fundamentals of Machine Learning

Installing Anaconda
Anaconda is a free collection of more than 720 open source data science packages for
Python including scikit-learn, NumPy, SciPy, pandas, and matplotlib. Anaconda is
platform-agnostic and simple to install. See IUUQTEPDTDPOUJOVVNJPBOBDPOEBJOTUB
                                 

MM for instructions for your operating system.

 

Verifying the installation

To verify that scikit-learn has been installed correctly, open a Python console and execute
the following:
*O<>
JNQPSUTLMFBSO
TLMFBSO@@WFSTJPO@@

0VU<>


To run scikit-learn's unit tests, first install the nose Python library. Then execute the
following in a terminal emulator:
$ nosetest sklearn -exe

Congratulations! You've successfully installed scikit-learn.

Installing pandas, Pillow, NLTK, and

matplotlib
pandas is an open source library that provides data structures and analysis tools for Python.
pandas is a powerful library, and several books describe how to use pandas for data
analysis. We will use a few of pandas's convenient tools for importing data and calculating
summary statistics. Pillow is a fork of the Python Imaging Library, which provides a variety
of image processing features. NLTK is a library for working with human language. As for
scikit-learn, QJQ is the preferred installation method for pandas, Pillow, and NLTK. Execute
the following command in a terminal emulator:
$ pip install pandas pillow nltk

[ 18 ]
The Fundamentals of Machine Learning

Matplotlib is a library for easily creating plots, histograms, and other charts with Python.
We will use it to visualize training data and models. Matplotlib has several dependencies.
Like pandas, matplotlib depends on NumPy, which should already be installed. On Ubuntu
16.04, matplotlib and its dependencies can be installed with:
$ sudo apt install python-matplotlib

Binaries for Mac OS and Windows 10 can be downloaded from IUUQNBUQMPUMJCPSHE

                 

PXOMPBETIUNM.
            

Summary
In this chapter, we defined machine learning as the design of programs that can improve
their performance at a task by learning from experience. We discussed the spectrum of
supervision in experience. At one end is supervised learning, in which a program learns
from inputs that are labeled with their corresponding outputs. Unsupervised learning, in
which the program must discover structure in only unlabeled inputs, is at the opposite end
of the spectrum. Semi-supervised approaches make use of both labeled and unlabeled
training data.

Next we discussed common types of machine learning tasks and reviewed examples of
each. In classification tasks the program predict the value of a discrete response variable
from the observed explanatory variables. In regression tasks the program must predict the
value of a continuous response variable from the explanatory variables. Unsupervised
learning tasks include clustering, in which observations are organized into groups
according to some similarity measure, and dimensionality reduction, which reduces a set of
explanatory variables to a smaller set of synthetic features that retain as much information
as possible. We also reviewed the bias-variance trade-off and discussed common
performance measures for different machine learning tasks.

In this chapter we discussed the history, goals, and advantages of scikit-learn. Finally, we
prepared our development environment by installing scikit-learn and other libraries that
are commonly used in conjunction with it. In the next chapter we will discuss a simple
model for regression tasks, and build our first machine learning model with scikit-learn.

[ 19 ]
 Simple Linear Regression
2
In this chapter, we will introduce our first model, simple linear regression. Simple linear
regression models the relationship between one response variable and one feature of an
explanatory variable. We will discuss how to fit our model, and we will work through a toy
problem. While simple linear regression is rarely applicable to real-world problems,
understanding it is essential to understanding many other models. In subsequent chapters,
we will learn about generalizations of simple linear regression and apply them to real-
world datasets.

Simple linear regression

In the previous chapter, we learned that training data is used to estimate the parameters of
a model in supervised learning problems. Observations of explanatory variables and their
corresponding response variables comprise training data. The model can be used to predict
the value of the response variable for values of the explanatory variable that have not been
previously observed. Recall that the goal in regression problems is to predict the value of a
continuous response variable. In this chapter, we will examine simple linear regression,
which can be used to model a linear relationship between one response variable and one
feature representing an explanatory variable.

Suppose you wish to know the price of a pizza. You might simply look at a menu. This,
however, is a machine learning book, so instead we will use simple linear regression to
predict the price of a pizza based on an attribute of the pizza that we can observe, or an
explanatory variable. Let's model the relationship between the size of a pizza and its price.
First, we will write a program with scikit-learn that can predict the price of a pizza given its
size. Then we will discuss how simple linear regression works and how it can be
generalized to work with other types of problems.
Simple Linear Regression

Let's assume that you have recorded the diameters and prices of pizzas that you have
previously eaten in your pizza journal. These observations comprise our training data:

Training instance Diameter in inches Price in dollars

  
  
  
  
  

We can visualize our training data by plotting it on a graph using NBUQMPUMJC:

*O<>
JNQPSUOVNQZBTOQOQBOEQMUBSFDPNNPOBMJBTFTGPS/VN1ZBOE
.BUQMPUMJCSFTQFDUJWFMZ
JNQPSUNBUQMPUMJCQZQMPUBTQMU

9SFQSFTFOUTUIFGFBUVSFTPGPVSUSBJOJOHEBUBUIFEJBNFUFSTPGUIF
QJ[[BT
"TDJLJUMFBSODPOWFOUJPOJTUPOBNFUIFNBUSJYPGGFBUVSFWFDUPST9
6QQFSDBTFMFUUFSTJOEJDBUFNBUSJDFTBOEMPXFSDBTFMFUUFSTJOEJDBUF
WFDUPST
9OQBSSBZ <<><><><><>> SFTIBQF 
Z<>ZJTBWFDUPSSFQSFTFOUJOHUIFQSJDFTPG
UIFQJ[[BT

QMUGJHVSF
QMUUJUMF 1J[[BQSJDFQMPUUFEBHBJOTUEJBNFUFS
QMUYMBCFM %JBNFUFSJOJODIFT
QMUZMBCFM 1SJDFJOEPMMBST
QMUQMPU 9Z L
QMUBYJT <>
QMUHSJE 5SVF
QMUTIPX

[ 21 ]
Simple Linear Regression

The comments in the script state that 9 represents a matrix of pizza diameters, and Z
represents a vector of pizza prices. The reasons for this decision will become clear in the
next chapter. This script produces the following plot. The diameters of the pizzas are
plotted on the x axis, and the prices are plotted on the y axis:

We can see from the plot of the training data that there is a positive relationship between
the diameter of a pizza and its price, which should be corroborated by our own pizza-eating
experience. As the diameter of a pizza increases, its price generally increases. The following
pizza price predictor program models this relationship using simple linear regression. Let's
review the program and discuss how simple linear regression works:
*O<>
GSPNTLMFBSOMJOFBS@NPEFMJNQPSU-JOFBS3FHSFTTJPO
NPEFM-JOFBS3FHSFTTJPO $SFBUFBOJOTUBODFPGUIFFTUJNBUPS
NPEFMGJU 9Z 'JUUIFNPEFMPOUIFUSBJOJOHEBUB

1SFEJDUUIFQSJDFPGBQJ[[BXJUIBEJBNFUFSUIBUIBTOFWFSCFFOTFFO
CFGPSF
UFTU@QJ[[BOQBSSBZ <<>>
QSFEJDUFE@QSJDFNPEFMQSFEJDU UFTU@QJ[[B <>
QSJOU "QJ[[BTIPVMEDPTUG QSFEJDUFE@QSJDF

[ 22 ]
Simple Linear Regression

0VU<>
"QJ[[BTIPVMEDPTU

Simple linear regression assumes that a linear relationship exists between the response
variable and the explanatory variable; it models this relationship with a linear surface called
a hyperplane. A hyperplane is a subspace that has one dimension less than the ambient
space that contains it. In simple linear regression, there is one dimension for the response
variable and another dimension for the explanatory variable, for a total of two dimensions.
The regression hyperplane thus has one dimension; a hyperplane with one dimension is a
line.

The -JOFBS3FHSFTTJPO class is an estimator. Estimators predict a value based on

observed data. In scikit-learn, all estimators implement the GJU methods and QSFEJDU. The
former method is used to learn the parameters of a model, and the latter method is used to
predict the value of a response variable for an explanatory variable using the learned
parameters. It is easy to experiment with different models using scikit-learn because all
estimators implement the GJU and QSFEJDU methods; trying new models can be as simple
as changing one line of code. The GJU method of -JOFBS3FHSFTTJPO learns the parameters
of the following model for simple linear regression:

In the preceding formula, y is the predicted value of the response variable; in this example,
it is the predicted price of the pizza. x is the explanatory variable. The intercept term α and
the coefficient β are parameters of the model that are learned by the learning algorithm. The
hyperplane plotted in the following figure models the relationship between the size of a
pizza and its price. Using this model, we would expect the price of an 8" pizza to be about
$7.33 and the price of a 20" pizza to be $18.75.

[ 23 ]
Simple Linear Regression

Using training data to learn the values of the parameters for simple linear regression that
produce the best fitting model is called ordinary least squares (OLS) or linear least
squares. In this chapter, we will discuss a method for analytically solving the values of the
model's parameters. In subsequent chapters, we will learn approaches for approximating
the values of parameters that are suitable for larger datasets. First, however, we must define
what it means for a model to fit the training data.

[ 24 ]
Simple Linear Regression

Evaluating the fitness of the model with a cost

function
Regression lines produced by several sets of parameter values are plotted in the following
figure. How can we assess which parameters produced the best-fitting regression line?

[ 25 ]
Simple Linear Regression

A cost function, also called a loss function, is used to define and measure the error of a
model. The differences between the prices predicted by the model and the observed prices
of the pizzas in the training set are called residuals, or training errors. Later, we will
evaluate the model on a separate set of test data. The differences between the predicted and
observed values in the test data are called prediction errors, or test errors. The residuals for
our model are indicated by vertical lines between the points for the training instances and
the regression hyperplane in the following plot:

We can produce the best pizza-price predictor by minimizing the sum of the residuals. That
is, our model fits if the values it predicts for the response variable are close to the observed
values for all of the training examples. This measure of the model's fitness is called the
residual sum of squares (RSS) cost function. Formally, this function assesses the fitness of a
model by summing the squared residuals for all of our training examples. The RSS is
calculated with the formula in the following equation, where yi is the observed value and
f(xi) is the predicted value:

[ 26 ]
Simple Linear Regression

Let's compute the RSS for our model by adding the following two lines to the previous
script:
QSJOU 3FTJEVBMTVNPGTRVBSFTG OQNFBO NPEFMQSFEJDU 9
Z  
3FTJEVBMTVNPGTRVBSFT

Now that we have a cost function, we can find the values of the model's parameters that
minimize it.

Solving OLS for simple linear regression

In this section, we will work through solving OLS for simple linear regression. Recall that
simple linear regression is given by the equation y = α + βx and that our goal is to solve for
the values of β and α to minimize the cost function. We will solve for β first. To do so, we
will calculate the variance of x and the covariance of x and y. Variance is a measure of how
far a set of values are spread out. If all the numbers in the set are equal, the variance of the
set is zero. A small variance indicates that the numbers are near the mean of the set, while a
set containing numbers that are far from the mean and from each other will have a large
variance. Variance can be calculated using the following equation:

is the mean of x, xi is the value of x for the ith training instance, and n is the number of
training instances. Let's calculate WBSJBODF of the pizza diameters in our training set:
*O<>
JNQPSUOVNQZBTOQ

9OQBSSBZ <<><><><><>> SFTIBQF 

Y@CBS9NFBO
QSJOU Y@CBS

/PUFUIBUXFTVCUSBDUPOFGSPNUIFOVNCFSPGUSBJOJOHJOTUBODFTXIFO
DBMDVMBUJOHUIFTBNQMFWBSJBODF
5IJTUFDIOJRVFJTDBMMFE#FTTFM TDPSSFDUJPO*UDPSSFDUTUIFCJBTJOUIF
FTUJNBUJPOPGUIFQPQVMBUJPOWBSJBODF
GSPNBTBNQMF
WBSJBODF 9Y@CBS  TVN  9TIBQF<>
QSJOU WBSJBODF

0VU<>

[ 27 ]
Simple Linear Regression




NumPy also provides the method WBS for calculating variance. The keyword parameter
EEPG can be used to set Bessel's correction to calculate the sample variance:

*O<>
QSJOU OQWBS 9EEPG

0VU<>


Covariance is a measure of how much two variables change together. If the variables
increase together, their covariance is positive. If one variable tends to increase while the
other decreases, their covariance is negative. If there is no linear relationship between the
two variables, their covariance will be equal to zero; they are linearly uncorrelated but not
necessarily independent. Covariance can be calculated using the following formula:

th
As with variance, xi is the diameter of the i training instance, is the mean of the
diameters, is the mean of the prices, yi is the price of the ith training instance, and n is the
number of training instances. Let's calculate DPWBSJBODF of the diameters and prices of the
pizzas in the training set:
*O<>
8FQSFWJPVTMZVTFEB-JTUUPSFQSFTFOUZ
)FSFXFTXJUDIUPB/VN1ZOEBSSBZXIJDIQSPWJEFTBNFUIPEUPDBMVMDBUF
UIFTBNQMFNFBO
ZOQBSSBZ <>

Z@CBSZNFBO
8FUSBOTQPTF9CFDBVTFCPUIPQFSBOETNVTUCFSPXWFDUPST
DPWBSJBODFOQNVMUJQMZ 9Y@CBS USBOTQPTF ZZ@CBS TVN 
 9TIBQF<>
QSJOU DPWBSJBODF
QSJOU OQDPW 9USBOTQPTF Z <><>

0VU<>



[ 28 ]
Simple Linear Regression

Now that we have calculated the variance of our explanatory variable and the covariance of
the response and explanatory variables, we can solve for β using the following:

Having solved for β, we can solve for α using this formula:

Here, is the mean of y and is the mean of are the coordinates of the centroid, a
point that the model must pass through.

Now that we have solved for the values of the model's parameters that minimize the cost
function, we can plug in the diameters of the pizzas and predict their prices. For instance,
an 11" pizza should be expected to cost about $12.70, and an 18" pizza should be expected
to cost $19.54. Congratulations! You used simple linear regression to predict the price of a
pizza.

Evaluating the model

We have used a learning algorithm to estimate a model's parameters from training data.
How can we assess whether our model is a good representation of the real relationship?
Let's assume that you have found another page in your pizza journal. We will use this
page's entries as a test set to measure the performance of our model. We have added a
fourth column; it contains the prices predicted by our model.

Test instance Diameter in inches Observed price in dollars Predicted price in dollars
   
   
   
   

[ 29 ]
Simple Linear Regression

   

Several measures can be used to assess our model's predictive capability. We will evaluate
our pizza price predictor using a measure called R-squared. Also known as the coefficient
of determination, R-squared measures how close the data are to a regression line. There are
several methods for calculating R-squared. In the case of simple linear regression, R-
squared is equal to the square of the Pearson product-moment correlation coefficient
(PPMCC), or Pearson's r. Using this method, R-squared must be a positive number between
zero and one. This method is intuitive; if R-squared describes the proportion of variance in
the response variable that is explained by the model, it cannot be greater than one or less
than zero. Other methods, including the method used by scikit-learn, do not calculate R-
squared as the square of Pearson's r. Using these methods, R-squared can be negative if the
model performs extremely poorly. It is important to note the limitations of performance
metrics. R-squared in particular is sensitive to outliers, and can spuriously increase when
features are added to the model.

We will follow the method used by scikit-learn to calculate R-squared for our pizza price
predictor. First we must measure the total sum of squares. yi is the observed value of the
th
response variable for the i test instance, and is the mean of the observed values of the
response variable.

Next we must find the RSS. Recall that this is also our cost function.

[ 30 ]
Simple Linear Regression

Finally, we can find R-squared using the following:

The R-squared score of 0.662 indicates that a large proportion of the variance in the test
instances' prices is explained by the model. Now let's confirm our calculation using scikit-
learn. The TDPSF method of -JOFBS3FHSFTTJPO returns the model's R-squared value, as
seen in the following example:

*O<>
JNQPSUOVNQZBTOQ
GSPNTLMFBSOMJOFBS@NPEFMJNQPSU-JOFBS3FHSFTTJPO

9@USBJOOQBSSBZ <> SFTIBQF 

Z@USBJO<>

9@UFTUOQBSSBZ <> SFTIBQF 

Z@UFTU<>

NPEFM-JOFBS3FHSFTTJPO
NPEFMGJU 9@USBJOZ@USBJO
S@TRVBSFENPEFMTDPSF 9@UFTUZ@UFTU
QSJOU S@TRVBSFE

0VU<>


Summary
In this chapter, we introduced simple linear regression, which models the relationship
between a single explanatory variable and a continuous response variable. We worked
through a toy problem to predict the price of a pizza from its diameter. We used the
residual sum of squares cost function to assess the fitness of our model, and analytically
solved the values of our model's parameter that minimized the cost function. We measured
the performance of our model on a test set. Finally, we introduced scikit-learn's estimator
API. In the next chapter, we will compare and contrast simple linear regression with
another simple, ubiquitous model, k-Nearest Neighbors (KNN).

[ 31 ]
 Classification and Regression
3
with k-Nearest Neighbors
In this chapter, we will introduce k-Nearest Neighbors (KNN), a simple algorithm that can
be used for classification and regression tasks. KNN's simplicity belies its power and
usefulness; it is widely used in the real world in a variety of applications, including search
and recommender systems. We will compare and contrast KNN with simple linear
regression and work through toy problems to understand the model.

K-Nearest Neighbors
KNN is a simple model for regression and classification tasks. It is so simple that its name
describes most of its learning algorithm. The titular neighbors are representations of
training instances in a metric space. A metric space is a feature space in which the distances
between all members of a set are defined. In the previous chapter's pizza problem, our
training instances were represented in a metric space because the distances between all the
pizza diameters was defined. These neighbors are used to estimate the value of the response
variable for a test instance. The hyperparameter k specifies how many neighbors can be
used in the estimation. A hyperparameter is a parameter that controls how the algorithm
learns; hyperparameters are not estimated from the training data and are sometimes set
manually. Finally, the k neighbors that are selected are those that are nearest to the test
instance, as measured by some distance function.
Classification and Regression with k-Nearest Neighbors

For classification tasks, a set of tuples of feature vectors and class labels comprise the
training set. KNN is a capable of binary, multi-class, and multi-label classification; we will
define these tasks later, and we will focus on binary classification in this chapter. The
simplest KNN classifiers use the mode of the KNN labels to classify test instances, but other
strategies can be used. The k is often set to an odd number to prevent ties. In regression
tasks, the feature vectors are each associated with a response variable that takes a real-
valued scalar instead of a label. The prediction is the mean or weighted mean of the KNN
response variables.

Lazy learning and non-parametric models

KNN is a lazy learner. Also known as instance-based learners, lazy learners simply store
the training dataset with little or no processing. In contrast to eager learners such as simple
linear regression, KNN does not estimate the parameters of a model that generalizes the
training data during a training phase. Lazy learning has advantages and disadvantages.
Training an eager learner is often computationally costly, but prediction with the resulting
model is often inexpensive. For simple linear regression, prediction consists only of
multiplying the learned coefficient by the feature, and adding the learned intercept
parameter. A lazy learner can predict almost immediately, but making predictions can be
costly. In the simplest implementation of KNN, prediction requires calculating the distances
between a test instance and all training instances.

In contrast to most of the other models that we will discuss, KNN is a non-parametric
model. A parametric model uses a fixed number of parameters, or coefficients, to define the
model that summarizes the data. The number of parameters is independent of the number
of training instances. Non-parametric may seem to be a misnomer, as it does not mean that
the model has no parameters; rather, non-parametric means that the number of parameters
of the model is not fixed, and may grow with the number of training instances.

Non-parametric models can be useful when training data is abundant and you have little
prior knowledge about the relationship between the response and explanatory variables.
KNN makes only one assumption: instances that are near each other are likely to have
similar values of the response variable. The flexibility provided by non-parametric models
is not always desirable; a model that makes assumptions about the relationship can be
useful if training data is scarce or if you already know about the relationship.

[ 33 ]
Exploring the Variety of Random
Documents with Different Content
But apart from this, had the portrait been not anterior, like Swann's
favourite photograph, to the systematisation of Odette's features in
a fresh type, majestic and charming, but subsequent to it, Elstir's
vision would alone have sufficed to disorganise that type. Artistic
genius in its reactions is like those extremely high temperatures
which have the power to disintegrate combinations of atoms which
they proceed to combine afresh in a diametrically opposite order,
following another type. All that artificially harmonious whole into
which a woman has succeeded in bringing her limbs and features,
the persistence of which every day, before going out, she studies in
her glass, changing the angle of her hat, smoothing her hair,
exercising the sprightliness in her eyes, so as to ensure its
continuity, that harmony the keen eye of the great painter instantly
destroys, substituting for it a rearrangement of the woman's features
such as will satisfy a certain pictorial ideal of femininity which he
carries in his head. Similarly it often happens that, after a certain
age, the eye of a great seeker after truth will find everywhere the
elements necessary to establish those relations which alone are of
interest to him. Like those craftsmen, those players who, instead of
making a fuss and asking for what they cannot have, content
themselves with the instrument that comes to their hand, the artist
might say of anything, no matter what, that it would serve his
purpose. Thus a cousin of the Princesse de Luxembourg, a beauty of
the most queenly type, having succumbed to a form of art which
was new at that time, had asked the leading painter of the naturalist
school to do her portrait. At once the artist's eye had found what he
sought everywhere in life. And on his canvas there appeared, in
place of the proud lady, a street-boy, and behind him a vast, sloping,
purple background which made one think of the Place Pigalle. But
even without going so far as that, not only will the portrait of a
woman by a great artist not seek in the least to give satisfaction to
various demands on the woman's part—such as for instance, when
she begins to age, make her have herself photographed in dresses
that are almost those of a young girl, which bring out her still
youthful figure and make her appear like the sister, or even the
daughter of her own daughter, who, if need be, is tricked out for the
occasion as a "perfect fright" by her side—it will, on the contrary,
emphasise those very drawbacks which she seeks to hide, and which
(as for instance a feverish, that is to say a livid complexion) are all
the more tempting to him since they give his picture "character";
they are quite enough, however, to destroy all the illusions of the
ordinary man who, when he sees the picture, sees crumble into dust
the ideal which the woman herself has so proudly sustained for him,
which has placed her in her unique, her unalterable form so far
apart, so far above the rest of humanity. Fallen now, represented
otherwise than in her own type in which she sat unassailably
enthroned, she is become nothing more than just an ordinary
woman, in the legend of whose superiority we have lost all faith. In
this type we are so accustomed to regard as included not only the
beauty of an Odette but her personality, her identity, that standing
before the portrait which has thus transposed her from it we are
inclined to protest not simply "How plain he has made her!" but
"Why, it isn't the least bit like her!" We find it hard to believe that it
can be she. We do not recognise her. And yet there is a person there
on the canvas whom we are quite conscious of having seen before.
But that person is not Odette; the face of the person, her body, her
general appearance seem familiar. They recall to us not this
particular woman who never held herself like that, whose natural
pose had no suggestion of any such strange and teasing arabesque
in its outlines, but other women, all the women whom Elstir has ever
painted, women whom invariably, however they may differ from one
another, he has chosen to plant thus on his canvas facing you, with
an arched foot thrust out from under the skirt, a large round hat in
one hand, symmetrically corresponding at the level of the knee
which it hides to what also appears as a disc, higher up in the
picture, the face. And furthermore, not only does a portrait by the
hand of genius disintegrate and destroy a woman's type, as it has
been defined by her coquetry and her selfish conception of beauty,
but if it is also old, it is not content with ageing the original in the
same way as a photograph ages its sitter, by shewing her dressed in
the fashions of long ago. In a portrait, it is not only the manner the
woman then had of dressing that dates it, there is also the manner
the artist had of painting. And this, Elstir's earliest manner, was the
most damaging of birth certificates for Odette because it not only
established her, as did her photographs of the same period, as the
younger sister of various time-honoured courtesans, but made her
portrait contemporary with the countless portraits that Manet or
Whistler had painted of all those vanished models, models who
already belonged to oblivion or to history.
It was along this train of thought, meditated in silence by the side of
Elstir, as I accompanied him to his door, that I was being led by the
discovery that I had just made of the identity of his model, when
this original discovery caused me to make a second, more disturbing
still, involving the identity of the artist. He had painted the portrait of
Odette de Crécy. Could it possibly be that this man of genius, this
sage, this eremite, this philosopher with his marvellous flow of
conversation, who towered over everyone and everything, was the
foolish, corrupt little painter who had at one time been "taken up" by
the Verdurins? I asked him if he had known them, whether by any
chance it was he that they used to call M. Biche. He answered me in
the affirmative, with no trace of embarrassment, as if my question
referred to a period in his life that was ended and already somewhat
remote, with no suspicion of what a cherished illusion his words
were shattering in me, until looking up he read my disappointment
upon my face. His own assumed an expression of annoyance. And,
as we were now almost at the gate of his house, a man of less
outstanding eminence, in heart and brain, might simply have said
"good-bye" to me, a trifle dryly, and taken care to avoid seeing me
again. This however was not Elstir's way with me; like the master
that he was—and this was, perhaps, from the point of view of sheer
creative genius, his one fault, that he was a master in that sense of
the word, for an artist if he is to live the true life of the spirit in its
full extent, must be alone and not bestow himself with profusion,
even upon disciples—from every circumstance, whether involving
himself or other people, he sought to extract, for the better
edification of the young, the element of truth that it contained. He
chose therefore, rather than say anything that might have avenged
the injury to his pride, to say what he thought would prove
instructive to me. "There is no man," he began, "however wise, who
has not at some period of his youth said things, or lived in a way the
consciousness of which is so unpleasant to him in later life that he
would gladly, if he could, expunge it from his memory. And yet he
ought not entirely to regret it, because he cannot be certain that he
has indeed become a wise man—so far as it is possible for any of us
to be wise—unless he has passed through all the fatuous or
unwholesome incarnations by which that ultimate stage must be
preceded. I know that there are young fellows, the sons and
grandsons of famous men, whose masters have instilled into them
nobility of mind and moral refinement in their schooldays. They
have, perhaps, when they look back upon their past lives, nothing to
retract; they can, if they choose, publish a signed account of
everything they have ever said or done; but they are poor creatures,
feeble descendants of doctrinaires, and their wisdom is negative and
sterile. We are not provided with wisdom, we must discover it for
ourselves, after a journey through the wilderness which no one else
can take for us, an effort which no one can spare us, for our wisdom
is the point of view from which we come at last to regard the world.
The lives that you admire, the attitudes that seem noble to you are
not the result of training at home, by a father, or by masters at
school, they have sprung from beginnings of a very different order,
by reaction from the influence of everything evil or common-place
that prevailed round about them. They represent a struggle and a
victory. I can see that the picture of what we once were, in early
youth, may not be recognisable and cannot, certainly, be pleasing to
contemplate in later life. But we must not deny the truth of it, for it
is evidence that we have really lived, that it is in accordance with the
laws of life and of the mind that we have, from the common
elements of life, of the life of studios, of artistic groups—assuming
that one is a painter—extracted something that goes beyond them."
Meanwhile we had reached his door. I was disappointed at not
having met the girls. But after all there was now the possibility of
meeting them again later on; they had ceased to do no more than
pass beyond a horizon on which I had been ready to suppose that I
should never see them reappear. Around them no longer swirled that
sort of great eddy which had separated me from them, which had
been merely the expression of the perpetually active desire, mobile,
compelling, fed ever on fresh anxieties, which was aroused in me by
their inaccessibility, their flight from me, possibly for ever. My desire
for them, I could now set it at rest, hold it in reserve, among all
those other desires the realisation of which, as soon as I knew it to
be possible, I would cheerfully postpone. I took leave of Elstir; I was
alone once again. Then all of a sudden, despite my recent
disappointment, I saw in my mind's eye all that chain of coincidence
which I had not supposed could possibly come about, that Elstir
should be a friend of those very girls, that they who only that
morning had been to me merely figures in a picture with the sea for
background had seen me, had seen me walking in friendly intimacy
with a great painter, who was now informed of my secret longing
and would no doubt do what he could to assuage it. All this had
been a source of pleasure to me, but that pleasure had remained
hidden; it was one of those visitors who wait before letting us know
that they are in the room until all the rest have gone and we are by
ourselves. Then only do we catch sight of them, and can say to
them, "I am at your service," and listen to what they have to tell us.
Sometimes between the moment at which these pleasures have
entered our consciousness and the moment at which we are free to
entertain them, so many hours have passed, we have in the interval
seen so many people that we are afraid lest they should have grown
tired of waiting. But they are patient, they do not grow tired, and as
soon as the crowd has gone we find them there ready for us.
Sometimes it is then we who are so exhausted that it seems as
though our weary mind will not have the strength left to seize and
retain those memories, those impressions for which our frail self is
the one habitable place, the sole means of realisation. And we
should regret that failure, for existence to us is hardly interesting
save on the days on which the dust of realities is shot with magic
sand, on which some trivial incident of life becomes a spring of
romance. Then a whole promontory of the inaccessible world rises
dear in the light of our dream, and enters into our life, our life in
which, like the sleeper awakened, we actually see the people of
whom we have been so ardently dreaming that we came to believe
that we should never behold them save in our dreams.
The sense of comfort that I drew from the probability of my now
being able to meet the little band whenever I chose was all the more
precious to me because I should not have been able to keep watch
for them during the next few days, which would be taken up with
preparations for Saint-Loup's departure. My grandmother was
anxious to offer my friend some proof of her gratitude for all the
kindnesses that he had shewn to her and myself. I told her that he
was a great admirer of Proudhon, and this put it into her head to
send for a collection of autograph letters by that philosopher which
she had once bought; Saint-Loup came to her room to look at them
on the day of their arrival, which was also his last day at Balbec. He
read them eagerly, fingering each page with reverence, trying to get
the sentences by heart; and then, rising from the table, was
beginning to apologise to my grandmother for having stayed so long,
when he heard her say: "No, no; take them with you; they are for
you to keep; that was why I sent for them, to give them to you."
He was overpowered by a joy which he could no more control than
we can a physical condition that arises without the intervention of
our will. He blushed scarlet as a child who has just been whipped,
and my grandmother was a great deal more touched to see all the
efforts that he was making (without success) to control the joy that
convulsed him than she would have been to hear any words of
thanks that he could have uttered. But he, fearing that he had failed
to shew his gratitude properly, begged me to make his excuses to
her again, next day, leaning from the window of the little train of the
local railway company which was to take him back to his regiment.
The distance was, as a matter of fact, nothing. He had thought of
going, as he had frequently done that summer, when he was to
return the same evening and was not encumbered with luggage, by
road. But this time he would have had, anyhow, to put all his heavy
luggage in the train. And he found it simpler to take the train himself
also, following the advice of the manager who, on being consulted,
replied that "Carriage or train, it was more or less equivocal." He
meant us to understand that they were equivalent (in fact, very
much what Françoise would have expressed as "coming to as near
as made no difference"). "Very well," Saint-Loup had decided, "I will
take the 'little crawler'." I should have taken it too, had I not been
tired, and gone with my friend to Doncières; failing this I kept on
promising, all the time that we waited in the Balbec station—the
time, that is to say, which the driver of the little train spent in
waiting for unpunctual friends, without whom he refused to start,
and also in seeking some refreshment for himself—to go over there
and see him several times a week. As Bloch had come to the station
also—much to Saint-Loup's disgust—the latter, seeing that our
companion could hear him begging me to come to luncheon, to
dinner, to stay altogether at Doncières, finally turned to him and, in
the most forbidding tone, intended to counteract the forced civility of
the invitation and to prevent Bloch from taking it seriously: "If you
ever happen to be passing through Doncières any afternoon when I
am off duty, you might ask for me at the barracks; but I hardly ever
am off duty." Perhaps, also, Robert feared lest, if left to myself, I
might not come, and, thinking that I was more intimate with Bloch
than I made out, was providing me in this way with a travelling
companion, one who would urge me on.
I was afraid that this tone, this way of inviting a person while
warning him not to come, might have wounded Bloch, and felt that
Saint-Loup would have done better, saying nothing. But I was
mistaken, for after the train had gone, while we were walking back
together as far as the cross-roads at which we should have to part,
one road going to the hotel, the other to the Blochs' villa, he never
ceased from asking me on what day we should go to Doncières, for
after "all the civilities that Saint-Loup had shewn" him, it would be
"too unmannerly" on his part not to accept the invitation. I was glad
that he had not noticed, or was so little displeased as to wish to let it
be thought that he had not noticed on how far from pressing, how
barely polite a note the invitation had been sounded. At the same
time I should have liked Bloch, for his own sake, to refrain from
making a fool of himself by going over at once to Doncières. But I
dared not offer a piece of advice which could only have offended
him by hinting that Saint-Loup had been less pressing than himself
impressed. He was a great deal too ready to respond, and even if all
his faults of this nature were atoned for by remarkable qualities
which others, with more reserve than he, would not possess, he
carried indiscretion to a pitch that was almost maddening. The week
must not, to hear him speak, pass without our going to Doncières
(he said "our" for I think that he counted to some extent on my
presence there as an excuse for his own). All the way home,
opposite the gymnasium, in its grove of trees, opposite the lawn-
tennis courts, the mayor's office, the shell-fish stall, he stopped me,
imploring me to fix a day, and, as I did not, left me in a towering
rage, saying: "As your lordship pleases. For my part, I am obliged to
go since he has invited me."
Saint-Loup was still so much afraid of not having thanked my
grandmother properly that he charged me once again to express his
gratitude to her a day or two later in a letter I received from him
from the town in which he was quartered, a town which seemed, on
the envelope where the post-mark had stamped its name, to be
hastening to me across country, to tell me that within its walls, in
the Louis XVI cavalry barracks, he was thinking of me. The paper
was embossed with the arms of Marsantes, in which I could make
out a lion, surmounted by a coronet formed by the cap of a Peer of
France.
"After a journey which," he wrote, "passed pleasantly enough, with a
book I bought at the station, by Arvède Barine (a Russian author, I
fancy; it seemed to me remarkably well written for a foreigner, but
you shall give me your critical opinion, you are bound to know all
about it, you fount of knowledge who have read everything), here I
am again in the thick of this debased existence, where, alas, I feel a
sad exile, not having here what I had to leave at Balbec; this life in
which I cannot discover one affectionate memory, any intellectual
attraction; an environment on which you would probably look with
contempt—and yet it has a certain charm. Everything seems to have
changed since I was last here, for in the interval one of the most
important periods in my life, that from which our friendship dates,
has begun. I hope that it may never come to an end. I have spoken
of our friendship, of you, to one person only, to the friend I told you
of, who has just paid me a surprise visit here. She would like
immensely to know you, and I feel that you would get on well
together, for she too is extremely literary. I, on the other hand, to go
over in my mind all our talk, to live over again those hours which I
never shall forget, have shut myself off from my comrades, excellent
fellows, but altogether incapable of understanding that sort of thing.
This remembrance of moments spent with you I should almost have
preferred, on my first day here, to call up for my own solitary
enjoyment, without writing. But I was afraid lest you, with your
subtle mind and ultra-sensitive heart, might, if you did not hear from
me, needlessly torment yourself, if, that is to say, you still
condescend to occupy your thoughts with this blunt trooper whom
you will have a hard task to polish and refine, and make a little more
subtle and worthier of your company."
On the whole this letter, in its affectionate spirit, was not at all unlike
those which, when I did not yet know Saint-Loup, I had imagined
that he would write to me, in those daydreams from which the
coldness of his first greeting had shaken me by bringing me face to
face with an icy reality which was not, however, to endure. Once I
had received this letter, whenever, at luncheon-time, the post was
brought in, I could tell at once when it was from him that a letter
came, for it had always that second face which a person assumes
when he is absent, in the features of which (the characters of his
script) there is no reason why we should not suppose that we are
tracing an individual soul just as much as in the line of a nose or the
inflexions of a voice.
I would now gladly remain at the table while it was being cleared,
and, if it was not a moment at which the girls of the little band
might be passing, it was no longer solely towards the sea that I
would turn my eyes. Since I had seen such things depicted in water-
colours by Elstir, I sought to find again in reality, I cherished, as
though for their poetic beauty, the broken gestures of the knives still
lying across one another, the swollen convexity of a discarded napkin
upon which the sun would patch a scrap of yellow velvet, the half-
empty glass which thus shewed to greater advantage the noble
sweep of its curved sides, and, in the heart of its translucent crystal,
clear as frozen daylight, a dreg of wine, dusky but sparkling with
reflected lights, the displacement of solid objects, the transmutation
of liquids by the effect of light and shade, the shifting colour of the
plums which passed from green to blue and from blue to golden
yellow in the half-plundered dish, the chairs, like a group of old
ladies, that came twice daily to take their places round the white
cloth spread on the table as on an altar at which were celebrated the
rites of the palate, where in the hollows of oyster-shells a few drops
of lustral water had gathered as in tiny holy water stoups of stone; I
tried to find beauty there where I had never imagined before that it
could exist, in the most ordinary things, in the profundities of "still
life".
When, some days after Saint-Loup's departure, I had succeeded in
persuading Elstir to give a small tea-party, at which I was to meet
Albertine, that freshness of appearance, that smartness of attire,
both (alas) fleeting, which were to be observed in me at the moment
of my starting out from the Grand Hotel, and were due respectively
to a longer rest than usual and to special pains over my toilet, I
regretted my inability to reserve them (and also the credit accruing
from Elstir's friendship) for the captivation of some other, more
interesting person; I regretted having to use them all up on the
simple pleasure of making Albertine's acquaintance. My brain
assessed this pleasure at a very low value now that it was assured
me. But, inside, my will did not for a moment share this illusion, that
will which is the persevering and unalterable servant of our
successive personalities; hiding itself in secret places, despised,
downtrodden, untiringly faithful, toiling without intermission and
with no thought for the variability of the self, its master, if only that
master may never lack what he requires. Whereas at the moment
when we are just about to start on a long-planned and eagerly
awaited holiday, our brain, our nerves begin to ask themselves
whether it is really worth all the trouble involved, the will, knowing
that those lazy masters would at once begin to consider their
journey the most wonderful experience, if it became impossible for
them to take it, the will leaves them explaining their difficulties
outside the station, multiplying their hesitations; but busies itself
with taking the tickets and putting us into the carriage before the
train starts. It is as invariable as brain and nerves are fickle, but as it
is silent, gives no account of its actions, it seems almost non-
existent; it is by its dogged determination that the other constituent
parts of our personality are led, but without seeing it, while they
distinguish clearly all their own uncertainties. My nerves and brain
then started a discussion as to the real value of the pleasure that
there would be in knowing Albertine, while I studied in the glass vain
and perishable attractions which nerves and brain would have
preserved intact for use on some other occasion. But my will would
not let the hour pass at which I must start, and it was Elstir's
address that it called out to the driver. Brain and nerves were at
liberty, now that the die was cast, to think this "a pity." If my will
had given the man a different address, they would have been finely
"sold".
When I arrived at Elstir's, a few minutes later, my first impression
was that Mlle. Simonet was not in the studio. There was certainly a
girl sitting there in a silk frock, bare-headed, but one whose
marvellous hair, her nose, meant nothing to me, in whom I did not
recognise the human entity that I had formed out of a young cyclist
strolling past, in a polo-cap, between myself and the sea. It was
Albertine, nevertheless. But even when I knew it to be her, I gave
her no thought. On entering any social gathering, when we are
young, we lose consciousness of our old self, we become a different
man, every drawing-room being a fresh universe, in which, coming
under the sway of a new moral perspective, we fasten our attention,
as if they were to matter to us for all time, on people, dances, card-
tables, all of which we shall have forgotten by the morning. Obliged
to follow, if I was to arrive at the goal of conversation with Albertine,
a road in no way of my own planning, which first brought me to a
halt at Elstir, passed by other groups of guests to whom I was
presented, then along the table, at which I was offered, and ate a
strawberry tart or two, while I listened, motionless, to the music that
was beginning in another part of the room, I found myself giving to
these various incidents the same importance as to my introduction
to Mlle. Simonet, an introduction which was now nothing more than
one among several such incidents, having entirely forgotten that it
had been, but a few minutes since, my sole object in coming there
that day. But is it not ever thus in the bustle of daily life, with every
true happiness, every great sorrow. In a room full of other people
we receive from her whom we love the answer, propitious or fatal,
which we have been awaiting for the last year. But we must go on
talking, ideas come, one after another, forming a smooth surface
which is pricked, at the very most, now and then by a dull throb
from within of the memory, deep-rooted enough but of very slender
growth, that misfortune has come upon us. If, instead of misfortune,
it is happiness, it may be that not until many years have elapsed will
we recall that the most important event in our sentimental life
occurred without our having time to give it any prolonged attention,
or even to become aware of it almost, at a social gathering, it may
have been, to which we had gone solely in expectation of that event.
When Elstir asked me to come with him so that he might introduce
me to Albertine, who was sitting a little farther down the room, I
first of all finished eating a coffee éclair and, with a show of keen
interest, asked an old gentleman whose acquaintance I had just
made (and thought that I might, perhaps, offer him the rose in my
buttonhole which he had admired) to tell me more about the old
Norman fairs. This is not to say that the introduction which followed
did not give me any pleasure, nor assume a definite importance in
my eyes. But so far as the pleasure was concerned, I was not
conscious of it, naturally, until some time later, when, once more in
the hotel, and in my room alone, I had become myself again.
Pleasure in this respect is like photography. What we take, in the
presence of the beloved object, is merely a negative film; we
develop it later, when we are at home, and have once again found at
our disposal that inner dark-room, the entrance to which is barred to
us so long as we are with other people.
If my consciousness of the pleasure it had brought me was thus
retarded by a few hours, the importance of this introduction I felt
immediately. At such moments of introduction, for all that we feel
ourselves to have been suddenly enriched, to have been furnished
with a pass that will admit us henceforward to pleasures which we
have been pursuing for weeks past, but in vain, we realise only too
clearly that this acquisition puts an end for us not merely to hours of
toilsome search—a relief that could only fill us with joy—but also to
the very existence of a certain person, her whom our imagination
had wildly distorted, our anxious fear that we might never become
known to her enlarged. At the moment when our name sounds on
the lips of the person introducing us, especially if he amplifies it, as
Elstir was now doing, with a flattering account of us—in that
sacramental moment, as when in a fairy tale the magician
commands a person suddenly to become someone else, she to
whose presence we have been longing to attain vanishes; how could
she remain the same when, for one thing—owing to the attention
which the stranger is obliged to pay to the announcement of our
name and the sight of our person—in the eyes that only yesterday
were situated at an infinite distance (where we supposed that our
eyes, wandering, uncontrolled, desperate, divergent, would never
succeed in meeting them) the conscious gaze, the incommunicable
thought which we have been seeking have been miraculously and
quite simply replaced by our own image, painted in them as though
behind the glass of a smiling mirror. If this incarnation of ourself in
the person who seems to differ most from us is what does most to
modify the appearance of the person to whom we have just been
introduced, the form of that person still remains quite vague; and we
are free to ask ourself whether she will turn out to be a god, a table
or a basin. But, as nimble as the wax-modellers who will fashion a
bust before our eyes in five minutes, the few words which the
stranger is now going to say to us will substantiate her form, will
give her something positive and final that will exclude all the
hypotheses by which, a moment ago, our desire, our imagination
were being tempted. Doubtless, even before her coming to this
party, Albertine had ceased to be to me simply that sole phantom
worthy to haunt our life which is what remains of a passing stranger,
of whom we know nothing and have caught but the barest glimpse.
Her relation to Mme. Bontemps had already restricted the scope of
those marvellous hypotheses, by stopping one of the channels along
which they might have spread. As I drew closer to the girl, and
began to know her better, my knowledge of her underwent a
process of subtraction, all the factors of imagination and desire
giving place to a notion which was worth infinitely less, a notion to
which, it must be admitted, there was added presently what was
more or less the equivalent, in the domain of real life, of what joint
stock companies give one, after paying interest on one's capital, and
call a bonus. Her name, her family connexions had been the original
limit set to my suppositions. Her friendly greeting while, standing
close beside her, I saw once again the tiny mole on her cheek, below
her eye, marked another stage; last of all, I was surprised to hear
her use the adverb "perfectly" (in place of "quite") of two people
whom she mentioned, saying of one: "She is perfectly mad, but very
nice for all that," and of the other, "He is a perfectly common man, a
perfect bore." However little to be commended this use of "perfectly"
may be, it indicates a degree of civilisation and culture which I could
never have imagined as having been attained by the bacchante with
the bicycle, the frenzied muse of the golf-course. Nor did it mean
that after this first transformation Albertine was not to change again
for me, many times. The good and bad qualities which a person
presents to us, exposed to view on the surface of his or her face,
rearrange themselves in a totally different order if we approach them
from another angle—just as, in a town, buildings that appear strung
irregularly along a single line, from another aspect retire into a
graduated distance, and their relative heights are altered. To begin
with, Albertine now struck me as not implacable so much as almost
frightened; she seemed to me rather respectably than ill-bred,
judging by the description, "bad style," "a comic manner" which she
applied to each in turn of the girls of whom I spoke to her; finally,
she presented as a target for my line of sight a temple that was
distinctly flushed and hardly attractive to the eye, and no longer the
curious gaze which I had always connected with her until then. But
this was merely a second impression and there were doubtless
others through which I was successively to pass. Thus it can be only
after one has recognised, not without having had to feel one's way,
the optical illusions of one's first impression that one can arrive at an
exact knowledge of another person, supposing such knowledge to
be ever possible. But it is not; for while our original impression of
him undergoes correction, the person himself, not being an
inanimate object, changes in himself, we think that we have caught
him, he moves, and, when we imagine that at last we are seeing
him clearly, it is only the old impressions which we had already
formed of him that we have succeeded in making clearer, when they
no longer represent him.
And yet, whatever the inevitable disappointments that it must bring
in its train, this movement towards what we have only half seen,
what we have been free to dwell upon and imagine at our leisure,
this movement is the only one that is wholesome for the senses, that
whets the appetite. How dreary a monotony must pervade those
people's lives who, from indolence or timidity, drive in their carriages
straight to the doors of friends whom they have got to know without
having first dreamed of knowing them, without ever daring, on the
way, to stop and examine what arouses their desire.
I returned home, my mind full of the party, the coffee éclair which I
had finished eating before I let Elstir take me up to Albertine, the
rose which I had given the old gentleman, all the details selected
without our knowledge by the circumstances of the occasion, which
compose in a special and quite fortuitous order the picture that we
retain of a first meeting. But this picture, I had the impression that I
was seeing it from a fresh point of view, a long way remote from
myself, realising that it had not existed only for me, when some
months later, to my great surprise, on my speaking to Albertine on
the day on which I had first met her, she reminded me of the éclair,
the flower that I had given away, all those things which I had
supposed to have been—I will not say of importance only to myself
but—perceived only by myself, and which I now found thus
transcribed, in a version the existence of which I had never
suspected, in the mind of Albertine. On this first day itself, when, on
my return to the hotel, I was able to visualise the memory which I
had brought away with me, I realised the consummate adroitness
with which the sleight of hand had been performed, and how I had
talked for a moment or two with a person who, thanks to the skill of
the conjurer, without actually embodying anything of that other
person whom I had for so long been following as she paced beside
the sea, had been effectively substituted for her. I might, for that
matter, have guessed as much in advance, since the girl of the
beach was a fabrication invented by myself. In spite of which, as I
had, in my conversations with Elstir, identified her with this other
girl, I felt myself in honour bound to fulfil to the real the promises of
love made to the imagined Albertine. We betroth ourselves by proxy,
and think ourselves obliged, in the sequel, to marry the person who
has intervened. Moreover, if there had disappeared, provisionally at
any rate, from my life, an anguish that found adequate consolation
in the memory of polite manners, of that expression "perfectly
common" and of the glowing temple, that memory awakened in me
desire of another kind which, for all that it was placid and not at all
painful, resembling rather brotherly love, might in the long run
become fully as dangerous by making me feel at every moment a
compelling need to kiss this new person, whose charming ways, her
shyness, her unlooked-for accessibility, arrested the futile process of
my imagination but gave birth to a sentimental gratitude. And then,
since memory begins at once to record photographs independent of
one another, eliminates every link, any kind of sequence from
between the scenes portrayed in the collection which it exposes to
our view, the most recent does not necessarily destroy or cancel
those that came before. Confronted with the common-place though
appealing Albertine to whom I had spoken that afternoon, I still saw
the other, mysterious Albertine outlined against the sea. These were
now memories, that is to say pictures neither of which now seemed
to me any more true than the other. But, to make an end of this first
afternoon of my introduction to Albertine, when trying to recapture
that little mole on her cheek, just under the eye, I remembered that,
looking from Elstir's window, when Albertine had gone by, I had seen
the mole on her chin. In fact, whenever I saw her I noticed that she
had a mole, but my inaccurate memory made it wander about the
face of Albertine, fixing it now in one place, now in another.
Whatever my disappointment in finding in Mlle. Simonet a girl so
little different from those that I knew already, just as my rude
awakening when I saw Balbec church did not prevent me from
wishing still to go to Quimperlé, Pont-Aven and Venice, I comforted
myself with the thought that through Albertine at any rate, even if
she herself was not all that I had hoped, I might make the
acquaintance of her comrades of the little band.
I thought at first that I should fail. As she was to be staying (and I
too) for a long time still at Balbec, I had decided that the best thing
was not to make my efforts to meet her too apparent, but to wait for
an accidental encounter. But should this occur every day, even, it
was greatly to be feared that she would confine herself to
acknowledging my bow from a distance, and such meetings,
repeated day after day throughout the whole season, would benefit
me not at all.
Shortly after this, one morning when it had been raining and was
almost cold, I was accosted on the "front" by a girl wearing a close-
fitting toque and carrying a muff, so different from the girl whom I
had met at Elstir's party that to recognise in her the same person
seemed an operation beyond the power of the human mind; mine
was, nevertheless, successful in performing it, but after a
momentary surprise which did not, I think, escape Albertine's notice.
On the other hand, when I instinctively recalled the good-breeding
which had so impressed me before, she filled me with a converse
astonishment by her rude tone and manners typical of the "little
band". Apart from these, her temple had ceased to be the optical
centre, on which the eye might comfortably rest, of her face, either
because I was now on her other side, or because her toque hid it, or
else possibly because its inflammation was not a constant thing.
"What weather!" she began. "Really the perpetual summer of Balbec
is all stuff and nonsense. You don't go in for anything special here,
do you? We don't ever see you playing golf, or dancing at the
Casino. You don't ride, either. You must be bored stiff. You don't find
it too deadly, staying about on the beach all day. I see, you just bask
in the sun like a lizard; you enjoy that. You must have plenty of time
on your hands. I can see you're not like me; I simply adore all
sports. You weren't at the Sogne races! We went in the 'tram', and I
can quite believe you don't see the fun of going in an old 'tin-pot'
like that. It took us two whole hours! I could have gone there and
back three times on my bike." I, who had been lost in admiration of
Saint-Loup when he, in the most natural manner in the world, called
the little local train the "crawler", because of the ceaseless windings
of its line, was positively alarmed by the glibness with which
Albertine spoke of the "tram", and called it a "tin-pot". I could feel
her mastery of a form of speech in which I was afraid of her
detecting and scorning my inferiority. And yet the full wealth of the
synonyms that the little band possessed to denote this railway had
not yet been revealed to me. In speaking, Albertine kept her head
motionless, her nostrils closed, allowing only the corners of her lips
to move. The result of this was a drawling, nasal sound, into the
composition of which there entered perhaps a provincial descent, a
juvenile affectation of British phlegm, the teaching of a foreign
governess and a congestive hypertrophy of the mucus of the nose.
This enunciation which, as it happened, soon disappeared when she
knew people better, giving place to a natural girlish tone, might have
been thought unpleasant. But it was peculiar to herself, and
delighted me. Whenever I had gone for several days without seeing
her, I would refresh my spirit by repeating to myself: "We don't ever
see you playing golf," with the nasal intonation in which she had
uttered the words, point blank, without moving a muscle of her face.
And I thought then that there could be no one in the world so
desirable.
We formed that morning one of those couples who dotted the
"front" here and there with their conjunction, their stopping together
for time enough just to exchange a few words before breaking apart,
each to resume separately his or her divergent stroll. I seized the
opportunity, while she stood still, to look again and discover once
and for all where exactly the little mole was placed. Then, just as a
phrase of Vinteuil which had delighted me in the sonata, and which
my recollection allowed to wander from the andante to the finale,
until the day when, having the score in my hands, I was able to find
it, and to fix it in my memory in its proper place, in the scherzo, so
this mole, which I had visualised now on her cheek, now on her
chin, came to rest for ever on her upper lip, just below her nose. In
the same way, too, do we not come with amazement upon lines that
we know by heart in a poem in which we never dreamed that they
were to be found.
At that moment, as if in order that against the sea there might
multiply in freedom, in the variety of its forms, all the rich decorative
whole which was the lovely unfolding of the train of maidens, at
once golden and rosy, baked by sun and wind, Albertine's friends,
with their shapely limbs, their supple figures, but so different one
from another, came into sight in a cluster that expanded as it
approached, advancing towards us, but keeping closer to the sea,
along a parallel line. I asked Albertine's permission to walk for a little
way with her. Unfortunately, all she did was to wave her hand to
them in greeting. "But your friends will be disappointed if you don't
go with them," I hinted, hoping that we might all walk together. A
young man with regular features, carrying a bag of golf-clubs,
sauntered up to us. It was the baccarat-player, whose fast ways so
enraged the chief magistrate's wife. In a frigid, impassive tone,
which he evidently regarded as an indication of the highest
refinement, he bade Albertine good day. "Been playing golf,
Octave?" she asked. "How did the game go? Were you in form?"
"Oh, it's too sickening; I can't play for nuts," he replied. "Was
Andrée playing?" "Yes, she went round in seventy-seven." "Why,
that's a record!" "I went round in eighty-two yesterday." He was the
son of an immensely rich manufacturer who was to take an
important part in the organisation of the coming World's Fair. I was
struck by the extreme degree to which, in this young man and in the
other by no means numerous male friends of the band of girls, the
knowledge of everything that pertained to clothes and how to wear
them, cigars, English drinks, horses, a knowledge which he
possessed in its minutest details with a haughty infallibility that
approached the reticent modesty of the true expert, had been
developed in complete isolation, unaccompanied by the least trace of
any intellectual culture. He had no hesitation as to the right time and
place for dinner jacket or pyjamas, but neither had he any suspicion
of the circumstances in which one might or might not employ this or
that word, or even of the simplest rules of grammar. This disparity
between the two forms of culture must have existed also in his
father, the President of the Syndicate that "ran" Balbec, for, in an
open letter to the electors which he had recently had posted on all
the walls, he announced: "I desired to see the Mayor, to speak to
him of the matter; he would not listen to my righteous plaint."
Octave, at the Casino, took prizes in all the dancing competitions, for
bostons, tangos and what-not, an accomplishment that would entitle
him, if he chose, to make a fine marriage in that seaside society
where it is not figuratively but in sober earnest that the young
women "marry their dancing-partners". He lighted a cigar with a
"D'you mind?" to Albertine, as one who asks permission to finish,
while going on talking, an urgent piece of work. For he was one of
those people who can never be "doing nothing", although there was
nothing, for that matter, that he could ever be said to do. And as
complete inactivity has the same effect on us, in the end, as
prolonged overwork, and on the character as much as on the life of
body and muscles, the unimpaired nullity of intellect that was
enshrined behind Octave's meditative brow had ended by giving him,
despite his air of unruffled calm, ineffectual longings to think which
kept him awake at night, for all the world like an overwrought
philosopher.
Supposing that if I knew their male friends I should have more
opportunities of seeing the girls, I had been on the point of asking
for an introduction to Octave. I told Albertine this, as soon as he had
left us, still muttering, "I couldn't play for nuts!" I thought I would
thus put into her head the idea of doing it next time. "But I can't,"
she cried, "introduce you to a tame cat like that. This place simply
swarms with them. But what on earth would they have to say to
you? That one plays golf quite well, and that's all there is to it. I
know what I'm talking about; you'ld find he wasn't at all your sort."
"Your friends will be cross with you if you desert them like this," I
repeated, hoping that she would then suggest my joining the party.
"Oh, no, they don't want me." We ran into Bloch, who directed at me
a subtle, insinuating smile, and, embarrassed by the presence of
Albertine, whom he did not know, or, rather, knew "without knowing"
her, bent his head with a stiff, almost irritated jerk. "What's he
called, that Ostrogoth?" Albertine asked. "I can't think why he should
bow to me; he doesn't know me. And I didn't bow to him, either." I
had no time to explain to her, for, bearing straight down upon us,
"Excuse me," he began, "for interrupting you, but I must tell you
that I am going to Doncières to-morrow. I cannot put it off any
longer without discourtesy; indeed, I ask myself, what must de
Saint-Loup-en-Bray think of me. I just came to let you know that I
shall take the two o'clock train. At your service." But I thought now
only of seeing Albertine again, and of trying to get to know her
friends, and Doncières, since they were not going there, and my
going would bring me back too late to see them still on the beach,
seemed to me to be situated at the other end of the world. I told
Bloch that it was impossible. "Oh, very well, I shall go alone. In the
fatuous words of Master Arouet, I shall say to Saint-Loup, to beguile
his clericalism:"

'My duty stands alone, by his in no way bound;

Though he should choose to fail, yet faithful I'll be
found.'
"I admit he's not a bad looking boy," was Albertine's comment, "but
he makes me feel quite sick." I had never thought that Bloch might
be "not a bad looking boy"; and yet, when one came to think of it,
so he was. With his rather prominent brow, very aquiline nose, and
his air of extreme cleverness and of being convinced of his
cleverness, he had a pleasing face. But he could not succeed in
pleasing Albertine. This was perhaps due, to some extent, to her
own disadvantages, the harshness, the want of feeling of the little
band, its rudeness towards everything that was not itself. And later
on, when I introduced them, Albertine's antipathy for him grew no
less. Bloch belonged to a section of society in which, between the
free and easy customs of the "smart set" and the regard for good
manners which a man is supposed to shew who "does not soil his
hands", a sort of special compromise has been reached which differs
from the manners of the world and is nevertheless a peculiarly
unpleasant form of worldliness. When he was introduced to anyone
he would bow with a sceptical smile, and at the same time with an
exaggerated show of respect, and, if it was to a man, would say:
"Pleased to meet you, sir," in a voice which ridiculed the words that
it was uttering, though with a consciousness of belonging to some
one who was no fool. Having sacrificed this first moment to a
custom which he at once followed and derided (just as on the first of
January he would greet you with a "Many happy!") he would adopt
an air of infinite cunning, and would "proffer subtle words" which
were often true enough but "got on" Albertine's nerves. When I told
her on this first day that his name was Bloch, she exclaimed: "I
would have betted anything he was a Jew-boy. Trust them to put
their foot in it!" Moreover, Bloch was destined to give Albertine other
grounds for annoyance later on. Like many intellectuals, he was
incapable of saying a simple thing in a simple way. He would find
some precious qualification for every statement, and would sweep
from particular to general. It vexed Albertine, who was never too
well pleased at other people's shewing an interest in what she was
doing, that when she had sprained her ankle and was keeping quiet,
Bloch said of her: "She is outstretched on her chair, but in her
ubiquity has not ceased to frequent simultaneously vague golf-
courses and dubious tennis-courts." He was simply being "literary",
of course, but this, in view of the difficulties which Albertine felt that
it might create for her with friends whose invitations she had
declined on the plea that she was unable to move, was quite enough
to disgust her with the face, the sound of the voice of the young
man who could say such things about her. We parted, Albertine and
I, after promising to take a walk together later. I had talked to her
without being any more conscious of where my words were falling,
of what became of them, than if I were dropping pebbles into a
bottomless pit. That our words are, as a general rule, filled, by the
person to whom we address them, with a meaning which that
person derives from her own substance, a meaning widely different
from that which we had put into the same words when we uttered
them, is a fact which the daily round of life is perpetually
demonstrating. But if we find ourself as well in the company of a
person whose education (as Albertine's was to me) is inconceivable,
her tastes, her reading, her principles unknown, we cannot tell
whether our words have aroused in her anything that resembles
their meaning, any more than in an animal, although there are
things that even an animal may be made to understand. So that to
attempt any closer friendship with Albertine seemed to me like
placing myself in contact with the unknown, if not the impossible, an
occupation as arduous as breaking a horse, as reposeful as keeping
bees or growing roses.
I had thought, a few hours before, that Albertine would
acknowledge my bow but would not speak to me. We had now
parted, after planning to make some excursion soon together. I
vowed that when I next met Albertine I would treat her with greater
boldness, and I had sketched out in advance a draft of all that I
would say to her, and even (being now quite convinced that she was
not strait-laced) of all the favours that I would demand of her. But
the mind is subject to external influences, as plants are, and cells
and chemical elements, and the medium in which its immersion
alters it is a change of circumstances, or new surroundings. Grown
different by the mere fact of her presence, when I found myself
once again in Albertine's company, what I said to her was not at all
what I had meant to say. Remembering her flushed temple, I asked
myself whether she might not appreciate more keenly a polite
attention which she knew to be disinterested. Besides, I was
embarrassed by certain things in her look, in her smile. They might
equally well signify a laxity of morals and the rather silly merriment
of a girl who though full of spirits was at heart thoroughly
respectable. A single expression, on a face as in speech, is
susceptible of divers interpretations, and I stood hesitating like a
schoolboy faced by the difficulties of a piece of Greek prose.
On this occasion we met almost immediately the tall one, Andrée,
the one who had jumped over the old banker, and Albertine was
obliged to introduce me. Her friend had a pair of eyes of
extraordinary brightness, like, in a dark house, a glimpse through an
open door of a room into which the sun is shining with a greenish
reflexion from the glittering sea.
A party of five were passing, men whom I had come to know very
well by sight during my stay at Balbec. I had often wondered who
they could be. "They're nothing very wonderful," said Albertine with
a sneering laugh. "The little old one with dyed hair and yellow gloves
has a fine touch; he knows how to draw all right, he's the Balbec
dentist; he's a good sort. The fat one is the Mayor, not the tiny little
fat one, you must have seen him before, he's the dancing master;
he's rather a beast, you know; he can't stand us, because we make
such a row at the Casino; we smash his chairs, and want to have the
carpet up when we dance; that's why he never gives us prizes,
though we're the only girls there who can dance a bit. The dentist is
a dear, I would have said how d'ye do to him, just to make the
dancing master swear, but I couldn't because they've got M. de
Sainte-Croix with them; he's on the General Council; he comes of a
very good family, but he's joined the Republicans, to make more
money. No nice people ever speak to him now. He knows my uncle,
because they're both in the Government, but the rest of my family
always cut him. The thin one in the waterproof is the bandmaster.
You know him, of course. You don't? Oh, he plays divinely. You
haven't been to Cavalleria Rusticana? I thought it too lovely! He's
giving a concert this evening, but we can't go because it's to be in
the town hall. In the Casino it wouldn't matter, but in the town hall,
where they've taken down the crucifix, Andrée's mother would have
a fit if we went there. You're going to say that my aunt's husband is
in the Government. But what difference does that make? My aunt is
my aunt. That's not why I'm fond of her. The only thing she has ever
wanted has been to get rid of me. No, the person who has really
been a mother to me, and all the more credit to her because she's
no relation at all, is a friend of mine whom I love just as much as if
she was my mother. I will let you see her 'photo'." We were joined
for a moment by the golf champion and baccarat plunger, Octave. I
thought that I had discovered a bond between us, for I learned in
the course of conversation that he was some sort of relative, and
even more a friend of the Verdurins. But he spoke contemptuously
of the famous Wednesdays, adding that M. Verdurin had never even
heard of a dinner jacket, which made it a horrid bore when one ran
into him in a music-hall, where one would very much rather not be
greeted with "Well, you young rascal," by an old fellow in a frock
coat and black tie, for all the world like a village lawyer. Octave left
us, and soon it was Andrée's turn, when we came to her villa, into
which she vanished without having uttered a single word to me
during the whole of our walk. I regretted her departure, all the more
in that, while I was complaining to Albertine how chilling her friend
had been with me, and was comparing in my mind this difficulty
which Albertine seemed to find in making me know her friends with
the hostility that Elstir, when he might have granted my desire,
seemed to have encountered on that first afternoon, two girls came
by to whom I lifted my hat, the young Ambresacs, whom Albertine
greeted also.
I felt that, in Albertine's eyes, my position would be improved by this
meeting. They were the daughters of a kinswoman of Mme. de
Villeparisis, who was also a friend of Mme. de Luxembourg. M. and
Mme. d'Ambresac, who had a small villa at Balbec and were
immensely rich, led the simplest of lives there, and always went
about dressed he in an unvarying frock coat, she in a dark gown.
Both of them used to make sweeping bows to my grandmother,
which never led to anything further. The daughters, who were very
pretty, were dressed more fashionably, but in a fashion suited rather
to Paris than to the seaside. With their long skirts and large hats,
they had the look of belonging to a different race from Albertine.
She, I discovered, knew all about them.
"Oh, so you know the little d'Ambresacs, do you? Dear me, you have
some swagger friends. After all, they're very simple souls," she went
on as though this might account for it. "They're very nice, but so
well brought up that they aren't allowed near the Casino, for fear of
us—we've such a bad tone. They attract you, do they? Well, it all
depends on what you like. They're just little white rabbits, really.
There may be something in that, of course. If little white rabbits are
what appeals to you, they may supply a long-felt want. It seems,
there must be some attraction, because one of them has got
engaged already to the Marquis de Saint-Loup. Which is a cruel blow
to the younger one, who is madly in love with that young man. I'm
sure, the way they speak to you with their lips shut is quite enough
for me. And then they dress in the most absurd way. Fancy going to
play golf in silk frocks! At their age, they dress more showily than
grown-up women who really know about clothes. Look at Mme.
Elstir; there's a well dressed woman if you like," I answered that she
had struck me as being dressed with the utmost simplicity. Albertine
laughed. "She does put on the simplest things, I admit, but she
dresses wonderfully, and to get what you call simplicity costs her a
fortune." Mme. Elstir's gowns passed unnoticed by any one who had
not a sober and unerring taste in matters of attire. This was lacking
in me. Elstir possessed it in a supreme degree, or so Albertine told
me. I had not suspected this, nor that the beautiful but quite simple
objects which littered his studio were treasures long desired by him
which he had followed from sale room to sale room, knowing all
their history, until he had made enough money to be able to acquire
them. But as to this Albertine, being as ignorant as myself, could not
enlighten me. Whereas when it came to clothes, prompted by a
coquettish instinct, and perhaps by the regretful longing of a
penniless girl who is able to appreciate with greater
disinterestedness, more delicacy of feeling, in other, richer people
the things that she will never be able to afford for herself, she
expressed herself admirably on the refinement of Elstir's taste, so
hard to satisfy that all women appeared to him badly dressed, while,
attaching infinite importance to right proportions and shades of
colour, he would order to be made for his wife, at fabulous prices,
the sunshades, hats and cloaks which he had learned from Albertine
to regard as charming, and which a person wanting in taste would
no more have noticed than myself. Apart from this, Albertine, who
had done a little painting, though without, she confessed, having
any "gift" for it, felt a boundless admiration for Elstir, and, thanks to
his precept and example, shewed a judgment of pictures which was
in marked contrast to her enthusiasm for Cavalleria Rusticana. The
truth was, though as yet it was hardly apparent, that she was highly
intelligent, and that in the things that she said the stupidity was not
her own but that of her environment and age. Elstir's had been a
good but only a partial influence. All the branches of her intelligence
had not reached the same stage of development. The taste for
pictures had almost caught up the taste for clothes and all forms of
smartness, but had not been followed by the taste for music, which
was still a long way behind.
Albertine might know all about the Ambresacs; but as he who can
achieve great things is not necessarily capable of small, I did not
find her, after I had bowed to those young ladies, any better
disposed to make me known to her friends. "It's too good of you to
attach any importance to them. You shouldn't take any notice of
them; they don't count. What on earth can a lot of kids like them
mean to a man like you? Now Andrée, I must say, is remarkably
clever. She is a good girl, that, though she is perfectly fantastic at
times, but the others are really dreadfully stupid." When I had left
Albertine, I felt suddenly a keen regret that Saint-Loup should have
concealed his engagement from me and that he should be doing
anything so improper as to choose a wife before breaking with his
mistress. And then, shortly afterwards, I met Andrée, and as she
went on talking to me for some time I seized the opportunity to tell
her that I would very much like to see her again next day, but she
replied that this was impossible, because her mother was not at all
well, and she would have to stay beside her. The next day but one,
when I was at Elstir's, he told me how greatly Andrée had been
attracted by me; on my protesting: "But it was I who was attracted
by her from the start; I asked her to meet me again yesterday, but
she could not." "Yes, I know; she told me all about that," was his
reply, "she was very sorry, but she had promised to go to a picnic,
somewhere miles from here. They were to drive over in a break, and
it was too late for her to get out of it." Albeit this falsehood (Andrée
knowing me so slightly) was of no real importance, I ought not to
have continued to seek the company of a person who was capable
of uttering it. For what people have once done they will do again
indefinitely, and if you go every year to see a friend who, the first
time, was not able to meet you at the appointed place, or was in
bed with a chill, you will find him in bed with another chill which he
has just caught, you will miss him again at another meeting-place at
which he has failed to appear, for a single and unalterable reason in
place of which he supposes himself to have various reasons, drawn
from the circumstances. One morning, not long after Andrée's telling
me that she would be obliged to stay beside her mother, I was
taking a short stroll with Albertine, whom I had found on the beach
tossing up and catching again on a cord an oddly shaped implement
which gave her a look of Giotto's "Idolatry"; it was called, for that
matter, "Diabolo", and is so fallen into disuse now that, when they
come upon the picture of a girl playing with one, the critics of future
generations will solemnly discuss, as it might be over one of the
allegorical figures in the Arena, what it is that she is holding. A
moment later their friend with the penurious and harsh appearance,
the same one who on that first day had sneered so malevolently: "I
do feel sorry for him, poor old man," when she saw the old
gentlemen's head brushed by the flying feet of Andrée, came up to
Albertine with "Good morning, 'm I disturbing you?" She had taken
off her hat, for comfort, and her hair, like a strange and fascinating
plant, lay over her brow, displaying all the delicate tracery of its
foliation. Albertine, perhaps because she resented seeing the other
bare-headed, made no reply, preserved a frigid silence in spite of
which the girl stayed with us, kept apart from myself by Albertine,
who arranged at one moment to be alone with her, at another to
walk with me leaving her to follow. I was obliged, to secure an
introduction, to ask for it in the girl's hearing. Then, as Albertine was
uttering my name, on the face and in the blue eyes of this girl,
whose expression I had thought so cruel when I heard her say:
"Poor old man, I do feel so sorry for him", I saw gather and gleam a
cordial, friendly smile, and she held out her hand. Her hair was
golden, and not her hair only; for if her cheeks were pink and her
eyes blue it was like the still roseate morning sky which sparkles
everywhere with dazzling points of gold.
At once kindled by her flame, I said to myself that this was a child
who when in love grew shy, that it was for my sake, from love for
me that she had remained with us, despite Albertine's rebuffs, and
that she must have rejoiced in the opportunity to confess to me at
last, by that smiling, friendly gaze, that she would be as kind to me
as she was terrible to other people. Doubtless she had noticed me
on the beach, when I still knew nothing of her, and had been
thinking of me ever since; perhaps it had been to win my admiration
that she mocked at the old gentleman, and because she had not
succeeded in getting to know me that on the following days she
appeared so morose. From the hotel I had often seen her, in the
evenings, walking by herself on the beach. Probably in the hope of
meeting me. And now, hindered as much by Albertine's presence as
she would have been by that of the whole band, she had evidently
attached herself to us, braving the increasing coldness of her friend's
attitude, only in the hope of outstaying her, of being left alone with
me, when she might make an appointment with me for some time
when she would find an excuse to slip away without either her
family's or her friends' knowing that she had gone, and would meet
me in some safe place before church or after golf. It was all the
more difficult to see her because Andrée had quarrelled with her and
now detested her. "I have put up far too long with her terrible
dishonesty," she explained to me, "her baseness; I can't tell you all
the vile insults she has heaped on me. I have stood it all because of
the others. But her latest effort was really too much!" And she told
me of some foolish thing that this girl had done, which might indeed
have injurious consequences to Andrée herself.
But those private words promised me by Gisèle's confiding eyes for
the moment when Albertine should have left us by ourselves, were
destined never to be spoken, because after Albertine, stubbornly
planted between us, had answered with increasing curtness, and
finally had ceased to respond at all to her friend's remarks, Gisèle at
length abandoned the attempt and turned back. I found fault with
Albertine for having been so disagreeable. "It will teach her to be
more careful how she behaves. She's not a bad kid, but she'ld talk
the head off a donkey. She's no business, either, to go poking her
nose into everything. Why should she fasten herself on to us without
being asked? In another minute I'ld have told her to go to blazes.
Besides I can't stand her going about with her hair like that; it's such
bad form." I gazed at Albertine's cheeks as she spoke, and asked
myself what might be the perfume, the taste of them: this time they
were not cool, but glowed with a uniform pink, violet-tinted, creamy,
like certain roses whose petals have a waxy gloss. I felt a passionate
longing for them such as one feels sometimes for a particular flower.
"I hadn't noticed it," was all that I said. "You stared at her hard
enough; anyone would have said you wanted to paint her portrait,"
she scolded, not at all softened by the fact that it was at herself that
I was now staring so fixedly. "I don't believe you would care for her,
all the same. She's not in the least a flirt. You like little girls who flirt
with you, I know. Anyhow, she won't have another chance of
fastening on to us and being sent about her business; she's going
off to-day to Paris." "Are the rest of your friends going too?" "No;
only she and 'Miss', because she's got an exam, coming; she's got to
stay at home and swot for it, poor kid. It's not much fun for her, I