Visit https://ebookfinal.

com to download the full version and

explore more ebooks

Learning OpenCV 5 Computer Vision with Python

Fourth Edition Joseph Howse & Joe Minichino

_____ Click the link below to download _____

https://ebookfinal.com/download/learning-
opencv-5-computer-vision-with-python-fourth-edition-
joseph-howse-joe-minichino/

Explore and download more ebooks at ebookfinal.com

Here are some suggested products you might be interested in.
Click the link to download

Learning OpenCV Computer Vision with the OpenCV Library

1st Edition Gary Bradski

https://ebookfinal.com/download/learning-opencv-computer-vision-with-
the-opencv-library-1st-edition-gary-bradski/

Industrial Vision Systems with Raspberry Pi Build and

Design Vision products Using Python and OpenCV 1st Edition
Mohaideen A
https://ebookfinal.com/download/industrial-vision-systems-with-
raspberry-pi-build-and-design-vision-products-using-python-and-
opencv-1st-edition-mohaideen-a/

Learning Image Processing with OpenCV 1st Edition Gloria

Bueno García Oscar Deniz Suarez

https://ebookfinal.com/download/learning-image-processing-with-
opencv-1st-edition-gloria-bueno-garcia-oscar-deniz-suarez/

Deep Learning with Python 2nd Edition François Chollet

https://ebookfinal.com/download/deep-learning-with-python-2nd-edition-
francois-chollet/
Architectures for computer vision from algorithm to chip
with Verilog Jeong

https://ebookfinal.com/download/architectures-for-computer-vision-
from-algorithm-to-chip-with-verilog-jeong/

Thoughtful Machine Learning with Python Early Release

Matthew Kirk

https://ebookfinal.com/download/thoughtful-machine-learning-with-
python-early-release-matthew-kirk/

Hacking Secret Ciphers with Python A beginner s guide to

cryptography and computer programming with Python 1st
Edition Al Sweigart
https://ebookfinal.com/download/hacking-secret-ciphers-with-python-a-
beginner-s-guide-to-cryptography-and-computer-programming-with-
python-1st-edition-al-sweigart/

Learning Joomla 1 5 Extension Development Creating Modules

Components and Plugins with PHP 1st Edition Joseph L.
Leblanc
https://ebookfinal.com/download/learning-joomla-1-5-extension-
development-creating-modules-components-and-plugins-with-php-1st-
edition-joseph-l-leblanc/

Fish Quality Control by Computer Vision First Edition Pau

https://ebookfinal.com/download/fish-quality-control-by-computer-
vision-first-edition-pau/
Learning OpenCV 5 Computer Vision with Python
Fourth Edition Joseph Howse & Joe Minichino Digital
Instant Download
Author(s): Joseph Howse & Joe Minichino
ISBN(s): 9781803230221, 1803230223
File Details: PDF, 18.83 MB
Year: 2023
Language: english
Learning OpenCV 5 Computer
Vision with Python
Copyright © 2022 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a

retrieval system, or transmitted in any form or by any means, without the
prior written permission of the publisher, except in the case of brief
quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the
accuracy of the information presented. However, the information contained
in this book is sold without warranty, either express or implied. Neither the
author, nor Packt Publishing, and its dealers and distributors will be held
liable for any damages caused or alleged to be caused directly or indirectly
by this book.

Packt Publishing has endeavored to provide trademark information about all

of the companies and products mentioned in this book by the appropriate
use of capitals. However, Packt Publishing cannot guarantee the accuracy of
this information.

Early Access Publication: Learning OpenCV 5 Computer Vision with

Python

Early Access Production Reference: B18454

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham
B3 2PB, UK

ISBN: 978-1-80323-022-1

www.packt.com
Table of Contents
1. Learning OpenCV 5 Computer Vision with Python, Fourth Edition:
Tackle tools, techniques, and algorithms for computer vision and
machine learning
2. 5 Detecting and Recognizing Faces
I. Technical requirements
II. Conceptualizing Haar cascades
III. Getting Haar cascade data
IV. Using OpenCV to perform face detection
i. Performing face detection on a still image
ii. Performing face detection on a video
V. Performing face recognition
i. Generating the data for face recognition
ii. Choosing a face recognition algorithm
iii. Loading the training data for face recognition
iv. Performing face recognition with Eigenfaces
v. Performing face recognition with Fisherfaces
vi. Performing face recognition with LBPH
vii. Discarding results based on the confidence score
VI. Swapping faces in infrared
i. Modifying the application's loop
ii. Masking a copy operation
VII. Summary
VIII. Join our book community on Discord
3. 6 Retrieving Images and Searching Using Image Descriptors
I. Technical requirements
II. Understanding types of feature detection and matching
i. Defining features
III. Detecting Harris corners
IV. Detecting DoG features and extracting SIFT descriptors
i. Anatomy of a keypoint
V. Detecting Fast Hessian features and extracting SURF descriptors
VI. Using ORB with FAST features and BRIEF descriptors
i. FAST
 ii. BRIEF
iii. Brute-force matching
iv. Matching a logo in two images
VII. Filtering matches using K-Nearest Neighbors and the ratio test
VIII. Matching with FLANN
IX. Finding homography with FLANN-based matches
X. A sample application – tattoo forensics
i. Saving image descriptors to file
ii. Scanning for matches
XI. Summary
XII. Join our book community on Discord
4. 7 Building Custom Object Detectors
I. Technical requirements
II. Understanding HOG descriptors
III. Visualizing HOG
IV. Using HOG to describe regions of an image
V. Understanding NMS
VI. Understanding SVMs
VII. Detecting people with HOG descriptors
VIII. Creating and training an object detector
i. Understanding BoW
ii. Applying BoW to computer vision
IX. Detecting cars
i. Combining an SVM with a sliding window
X. Summary
XI. Join our book community on Discord
Learning OpenCV 5 Computer
Vision with Python, Fourth Edition:
Tackle tools, techniques, and
algorithms for computer vision
and machine learning
Welcome to Packt Early Access. We’re giving you an exclusive preview
of this book before it goes on sale. It can take many months to write a book,
but our authors have cutting-edge information to share with you today.
Early Access gives you an insight into the latest developments by making
chapter drafts available. The chapters may be a little rough around the edges
right now, but our authors will update them over time. You can dip in and
out of this book or follow along from start to finish; Early Access is designed
to be flexible. We hope you enjoy getting to know more about the process
of writing a Packt book.

1. Chapter 1: Setting Up OpenCV

2. Chapter 2: Handling Files, Cameras, and GUIs
3. Chapter 3: Processing Images with OpenCV
4. Chapter 4: Depth Estimation and Segmentation
5. Chapter 5: Detecting and Recognizing Faces
6. Chapter 6: Retrieving Images and Searching Using Image Descriptors
7. Chapter 7: Building Custom Object Detector
8. Chapter 8: Tracking Objects
9. Chapter 9: Camera Models and Augmented Reality
10. Chapter 10: 3D Reconstruction and Navigation
11. Chapter 11: NeuraNetworks with OpenCV - an Introduction
12. Chapter 12: OpenCV Applications at Scale
5 Detecting and Recognizing Faces
Computer vision makes many futuristic-sounding tasks a reality. Two such tasks are face detection (locating faces
in an image) and face recognition (identifying a face as belonging to a specific person). OpenCV implements
several algorithms for face detection and recognition. These have applications in all sorts of real-world contexts,
from security to entertainment.This chapter introduces some of OpenCV's face detection and recognition
functionality, along with data files that define particular types of trackable objects. Specifically, in this chapter,
we’ll look at Haar cascade classifiers, which analyze the contrast between adjacent image regions to determine
whether or not a given image or sub-image matches a known type. We consider how to combine multiple Haar
cascade classifiers in a hierarchy so that one classifier identifies a parent region (for our purposes, a face) and other
classifiers identify child regions (such as eyes).We also take a detour into the humble but important subject of
rectangles. By drawing, copying, and resizing rectangular image regions, we can perform simple manipulations on
image regions that we are tracking.We will cover the following topics:

Understanding Haar cascades

Finding the pre-trained Haar cascades that come with OpenCV. These include several face detectors
Using Haar cascades to detect faces in still images and videos
Gathering images to train and test a face recognizer
Using several different face recognition algorithms: Eigenfaces, Fisherfaces, and Local Binary Pattern
Histograms (LBPHs)
Copying rectangular regions from one image to another, with or without a mask
Using a depth camera to distinguish between a face and the background based on depth
Swapping two people's faces in an interactive application

While this chapter focuses on classic approaches to face detection and recognition, we will go on to explore
advanced new models by using OpenCV with a variety of other libraries in Chapter 11, Neutral Networks
with OpenCV – an Introduction.

By the end of this chapter, we will have integrated face tracking and rectangle manipulations into Cameo, the
interactive application that we have developed in previous chapters. Finally, we will have some face-to-face
interaction!

Technical requirements
This chapter uses Python, OpenCV, and NumPy. As part of OpenCV, it uses the optional opencv_contrib
modules, which include functionality for face recognition. Some parts of this chapter use OpenCV's optional
support for OpenNI 2 to capture images from depth cameras. Please refer back to Chapter 1, Setting Up OpenCV,
for installation instructions.The complete code for this chapter can be found in this book's GitHub repository,
https://github.com/PacktPublishing/Learning-OpenCV-5-Computer-Vision-with-Python-Fourth-Edition, in the
chapter05 folder. Sample images are in the repository images folder.A subset of the chapter’s sample code can
be edited and run interactively in Google Colab at
https://colab.research.google.com/github/PacktPublishing/Learning-OpenCV-5-Computer-Vision-with-Python-
Fourth-Edition/blob/main/chapter05/chapter05.ipynb.

Conceptualizing Haar cascades

When we talk about classifying objects and tracking their location, what exactly are we hoping to pinpoint? What
constitutes a recognizable part of an object?Photographic images, even from a webcam, may contain a lot of detail
for our (human) viewing pleasure. However, image detail tends to be unstable with respect to variations in lighting,
viewing angle, viewing distance, camera shake, and digital noise. Moreover, even real differences in physical
detail might not interest us for classification. Joseph Howse, one of this book's authors, was taught in school that
no two snowflakes look alike under a microscope. Fortunately, as a Canadian child, he had already learned how to
recognize snowflakes without a microscope, as the similarities are more obvious in bulk.Hence, having some
means of abstracting image detail is useful in producing stable classification and tracking results. The abstractions
are called features, which are said to be extracted from the image data. There should be far fewer features than
pixels, though any pixel might influence multiple features. A set of features is represented as a vector
(conceptually, a set of coordinates in a multidimensional space), and the level of similarity between two images
can be evaluated based on some measure of the distance between the images' corresponding feature vectors.

Later, in Chapter 6, Retrieving Images and Searching Using Image Descriptors, we will explore several kinds
of features, as well as advanced ways of describing and matching sets of features.

Haar-like features are one type of feature that is often applied to real-time face detection. They were first used for
this purpose in the paper Robust Real-Time Face Detection, by Paul Viola and Michael Jones (International
Journal of Computer Vision 57(2), 137–154, Kluwer Academic Publishers, 2001). An electronic version of this
paper is available at http://comp3204.ecs.soton.ac.uk/cw/viola04ijcv.pdf.Each Haar-like feature describes the
pattern of contrast among adjacent image regions. For example, edges, vertices, and thin lines each generate a kind
of feature. Some features are distinctive in the sense that they typically occur in a certain class of object (such as a
face) but not in other objects. These distinctive features can be organized into a hierarchy, called a cascade, in
which the highest layers contain features of greatest distinctiveness, enabling a classifier to quickly reject subjects
that lack these features. If a subject is a good match for the higher-layer features, then the classifier considers the
lower-layer features too in order to weed out more false positives.For any given subject, the features may vary
depending on the scale of the image and the size of the neighborhood (the region of nearby pixels) in which
contrast is being evaluated. The neighborhood’s size is called the window size. To make a Haar cascade classifier
scale-invariant or, in other words, robust to changes in scale, the window size is kept constant but images are
rescaled a number of times; hence, at some level of rescaling, the size of an object (such as a face) may match the
window size. Together, the original image and the rescaled images are called an image pyramid, and each
successive level in this pyramid is a smaller rescaled image. OpenCV provides a scale-invariant classifier that can
load a Haar cascade from an XML file in a particular format. Internally, this classifier converts any given image
into an image pyramid.Haar cascades, as implemented in OpenCV, are not robust to changes in rotation or
perspective. For example, an upside-down face is not considered similar to an upright face and a face viewed in
profile is not considered similar to a face viewed from the front. A more complex and resource-intensive
implementation could improve a Haar cascade’s robustness to rotation by considering multiple transformations of
images as well as multiple window sizes. However, we will confine ourselves to the implementation in OpenCV.

Getting Haar cascade data

Your installation of OpenCV 5 should contain a subfolder called data . The path to this folder is stored in an
OpenCV variable called cv2.data.haarcascades .The data folder contains XML files that can be loaded by an
OpenCV class called cv2.CascadeClassifier . An instance of this class interprets a given XML file as a Haar
cascade, which provides a detection model for a type of object such as a face. cv2.CascadeClassifier can detect
this type of object in any image. As usual, we could obtain a still image from a file, or we could obtain a series of
frames from a video file or a video camera.From the data folder, we will use the following cascade files:
haarcascade_frontalface_default.xml
haarcascade_eye.xml

As their names suggest, these cascades are for detecting faces and eyes. They require a frontal, upright view of the
subject. We will use them later when building a face detector.

If you are curious about how these cascade files are generated, you can find more information in Joseph
Howse's book, OpenCV 4 for Secret Agents (Packt Publishing, 2019), specifically in Chapter 3, Training a
Smart Alarm to Recognize the Villain and His Cat. With a lot of patience and a reasonably powerful computer,
you can make your own cascades and train them for various types of objects.

Using OpenCV to perform face detection

With cv2.CascadeClassifier , it makes little difference whether we perform face detection on a still image or a
video feed. The latter is just a sequential version of the former: face detection on a video is simply face detection
applied to each frame. Naturally, with more advanced techniques, it would be possible to track a detected face
continuously across multiple frames and determine that the face is the same one in each frame. However, it is good
to know that a basic sequential approach also works.Let's go ahead and detect some faces.

Performing face detection on a still image

The first and most basic way to perform face detection is to load an image and detect faces in it. To make the result
visually meaningful, we will draw rectangles around faces in the original image. Remembering that the face
detector is designed for upright, frontal faces, we will use an image of a row of people, specifically woodcutters,
standing shoulder to shoulder and facing the photographer or viewer.Let's go ahead and create the following basic
script to perform face detection:
import cv2face_cascade = cv2.CascadeClassifier(
f'{cv2.data.haarcascades}haarcascade_frontalface_default.xml')
img = cv2.imread('../images/woodcutters.jpg')
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
faces = face_cascade.detectMultiScale(gray, 1.08, 5)
for (x, y, w, h) in faces:
img = cv2.rectangle(img, (x, y), (x+w, y+h), (255, 255255, 0), 2)

cv2.namedWindow('Woodcutters Detected!')
cv2.imshow('Woodcutters Detected!', img)
cv2.imwrite('./woodcutters_detected.pngpng', img)
cv2.waitKey(0)

Let's walk through the preceding code in small steps. First, we use the obligatory cv2 import that you will find in
every script in this book. Then, we declare a face_cascade variable, which is a CascadeClassifier object that
loads a cascade for face detection:
face_cascade = cv2.CascadeClassifier(
f'{cv2.data.haarcascades}haarcascade_frontalface_default.xml')

We then load our image file with cv2.imread and convert it into grayscale because CascadeClassifier , like
many of OpenCV’s classifiers, expects grayscale images. (If we try to use a color image, CascadeClassifier will
internally convert it to grayscale anyway.) The next step, face_cascade.detectMultiScale , is where we perform
the actual face detection:
img = cv2.imread('../images/woodcutters.jpg')
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
faces = face_cascade.detectMultiScale(gray, 1.08, 5)

The parameters of detectMultiScale include scaleFactor and minNeighbors . The scaleFactor argument,
which should be greater than 1.0, determines the downscaling ratio of the image at each iteration of the face
detection process. As we discussed earlier in the Conceptualizing Haar cascades section, this downscaling is
intended to achieve scale invariance by matching various faces to the window size. The minNeighbors argument
is the minimum number of overlapping detections that are required in order to retain a detection result. Normally,
we expect that a face may be detected in multiple overlapping windows, and a greater number of overlapping
detections makes us more confident that the detected face is truly a face.The value returned from the detection
operation is a list of tuples that represent the face rectangles. OpenCV's cv2.rectangle function allows us to
draw rectangles at the specified coordinates. x and y represent the left and top coordinates, while w and h
represent the width and height of the face rectangle. We draw cyan rectangles around all of the faces we find by
looping through the faces variable, making sure we use the original image for drawing, not the gray version:
for (x, y, w, h) in faces:
cv2.rectangle(img, (x, y), (x+w, y+h), (255, 255255, 0), 2)

Lastly, we call cv2.imshow to display the resulting processed image and we call cv2.imwrite to save it. As
usual, to prevent the image window from closing automatically, we insert a call to waitKey , which returns when
the user presses any key:
cv2.imshow('Woodcutters Detected!', img)
cv2.imwrite('./woodcutters_detected.pngpng', img)
cv2.waitKey(0)

And there we go, three members of the band of woodcutters have been detected in our image, as shown in the
following screenshot:

Figure 5.1: Face detection results of a photograph of woodcutters (Image credit: Prokudin-Gorsky)

The photograph in this example is the work of Sergey Prokudin-Gorsky (1863-1944), a pioneer of color
photography. Tsar Nicholas II sponsored Prokudin-Gorsky to photograph people and places throughout the
Russian Empire as a vast documentary project. Prokudin-Gorsky photographed these woodcutters near the
Svir river, in northwestern Russia, in 1909.

Here, we have no false positive detections; all three rectangles really are woodcutters’ faces. However, we do have
two false negatives (woodcutters whose faces were not detected). Try adjusting the parameters of
face_cascade.detectMultiScale to see how the results change. Then, let’s proceed to a more interactive
example.
Performing face detection on a video

We now understand how to perform face detection on a still image. As mentioned previously, we can repeat the
process of face detection on each frame of a video (be it a camera feed or a pre-recorded video file).The next script
will open a camera feed, read a frame, examine that frame for faces, and scan for eyes within the detected faces.
Finally, it will draw blue rectangles around the faces and green rectangles around the eyes. Here is the script in its
entirety:
import cv2
face_cascade = cv2.CascadeClassifier(
f'{cv2.data.haarcascades}haarcascade_frontalface_default.xml')
eye_cascade = cv2.CascadeClassifier(
f'{cv2.data.haarcascades}haarcascade_eye.xml')
camera = cv2.VideoCapture(0)
while (cv2.waitKey(1) == -1):
success, frame = camera.read()
if success:
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
faces = face_cascade.detectMultiScale(
gray, 1.3, 5, minSize=(120, 120))
for (x, y, w, h) in faces:
cv2.rectangle(frame, (x, y), (x+w, y+h), (255, 0, 0), 2)
roi_gray = gray[y:y+h, x:x+w]
eyes = eye_cascade.detectMultiScale(
roi_gray, 1.11, 5, minSize=(40, 40))
for (ex, ey, ew, eh) in eyes:
cv2.rectangle(frame, (x+ex, y+ey),
(x+ex+ew, y+ey+eh), (0, 255, 0), 2)
cv2.imshow('Face Detection', frame)

Let's break up the preceding sample into smaller, more digestible chunks:

1. As usual, we import the cv2 module. After that, we initialize two CascadeClassifier objects, one for faces
and another for eyes:
face_cascade = cv2.CascadeClassifier( f'{cv2.data.haarcascades}haarcascade_frontalface_default
eye_cascade = cv2.CascadeClassifier(
f'{cv2.data.haarcascades}haarcascade_eye.xml')

1. As in most of our interactive scripts, we open a camera feed and start iterating over frames. We continue until
the user presses any key. Whenever we successfully capture a frame, we convert it into grayscale as our first
step in processing it:
camera = cv2.VideoCapture(0)
while (cv2.waitKey(1) == -1):
success, frame = camera.read()
if success:
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

1. We detect faces with the detectMultiScale method of our face detector. As we have previously done, we
use the scaleFactor and minNeighbors arguments. We also use the minSize argument to specify a
minimum size of a face, specifically 120x120. No attempt will be made to detect faces smaller than this.
(Assuming that our user is sitting close to the camera, it is safe to say that the user's face will be larger than
120x120 pixels.) Here is the call to detectMultiScale :
faces = face_cascade.detectMultiScale(
gray, 1.3, 5, minSize=(120, 120))

1. We iterate over the rectangles of the detected faces. We draw a blue border around each rectangle in the
original color image. Then, within the same rectangular region of the grayscale image, we perform eye
detection:
for (x, y, w, h) in faces:
cv2.rectangle(frame, (x, y), (x+w, y+h), (255, 0, 0), 2)
 roi_gray = gray[y:y+h, x:x+w]
eyes = eye_cascade.detectMultiScale(
roi_gray, 1.1, 5, minSize=(40, 40))

The eye detector is a bit less accurate than the face detector. You might see shadows, parts of the frames of
glasses, or other regions of the face falsely detected as eyes. To improve the results, you could try defining
roi_gray as a smaller region of the face, since we can make a good guess about the eyes' location in an
upright face. You could also try using a maxSize argument to avoid false positives that are too large to be
eyes. Also, you could adjust minSize and maxSize so that the dimensions are proportional to w and h , the
size of the detected face. As an exercise, feel free to experiment with changes to these and other parameters.

1. We loop through the resulting eye rectangles and draw green outlines around them:
for (ex, ey, ew, eh) in eyes:
cv2.rectangle(frame, (x+ex, y+ey),
(x+ex+ew, y+ey+eh), (0, 255, 0), 2)

1. Finally, we show the resulting frame in the window:

cv2.imshow('Face Detection', frame)

Run the script. If our detectors produce accurate results, and if any face is within the field of view of the camera,
you should see a blue rectangle around the face and a green rectangle around each eye, as shown in this screenshot:

Figure 5.2: The resulting detection of a face and eyes in a photograph

Experiment with this script to see how the face and eye detectors perform under various conditions. Try a brighter
or darker room. If you wear glasses, try removing them. Try various people's faces and various expressions. Adjust
the detection parameters in the script to see how they affect the results. When you are satisfied, let's consider what
else we can do with faces in OpenCV.

Performing face recognition

Detecting faces is a fantastic feature of OpenCV and one that constitutes the basis for a more advanced operation:
face recognition. What is face recognition? It is the ability of a program, given an image or a video feed containing
a person's face, to identify that person. One of the ways to achieve this (and the approach adopted by OpenCV) is
to train the program by feeding it a set of classified pictures (a facial database) and perform recognition based on
the features of those pictures.Another important feature of OpenCV's face recognition module is that each
recognition has a confidence score, which allows us to set thresholds in real-life applications to limit the incidence
of false identifications.Let's start from the very beginning; to perform face recognition, we need faces to recognize.
We fulfill this requirement in two ways: supply the images ourselves or obtain freely available face databases. A
large directory of face databases is available online at http://www.face-rec.org/databases/. Here are a few notable
examples from the directory:

Yale Face Database (Yalefaces): http://vision.ucsd.edu/content/yale-face-database

Extended Yale Face Database B: http://vision.ucsd.edu/content/extended-yale-face-database-b-b
Database of Faces (from AT&T Laboratories Cambridge): https://cam-orl.co.uk/facedatabase.html

If we trained a face recognizer on these samples, we would then have to run face recognition on an image that
contains the face of one of the sampled people. This process might be educational, but perhaps not as satisfying as
providing images of our own. You probably had the same thought that many computer vision learners have had: I
wonder if I can write a program that recognizes my face with a certain degree of confidence. Indeed, you can and
soon you will!

Generating the data for face recognition

Let's go ahead and write a script that will generate those images for us. A few images containing different
expressions are all that we need, but it is preferable that the training images are square and are all the same size.
Our sample script uses a size of 200x200, but most freely available datasets have smaller images than this.Here is
the script itself:
import cv2
import os
output_folder = '../data/at/jm'
if not os.path.exists(output_folder):
os.makedirs(output_folder)
face_cascade = cv2.CascadeClassifier(
f'{cv2.data.haarcascades}haarcascade_frontalface_default.xml')
camera = cv2.VideoCapture(0)
count = 0
while (cv2.waitKey(1) == -1):
success, frame = camera.read()
if success:
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
faces = face_cascade.detectMultiScale(
gray, 1.3, 5, minSize=(120, 120))
for (x, y, w, h) in faces:
cv2.rectangle(frame, (x, y), (x+w, y+h), (255, 0, 0), 2)
face_img = cv2.resize(gray[y:y+h, x:x+w], (200, 200))
face_filename = '%s/%d.pgm' % (output_folder, count)
cv2.imwrite(face_filename, face_img)
count += 1
cv2.imshow('Capturing Faces...', frame)

Here, we are generating sample images by building on our newfound knowledge of how to detect a face in a video
feed. We are detecting a face, cropping that region of the grayscale-converted frame, resizing it to be 200x200
pixels, and saving it as a PGM file with a name in a particular folder (in this case, jm , one of the authors’ initials;
you can use your own initials). Like many of our windowed applications, this one runs until the user presses any
key.The count variable is present because we needed progressive names for the images. Run the script for a few
seconds, change your facial expression a few times, and check the destination folder you specified in the script.
You will find a number of images of your face, grayed, resized, and named with the format <count>.pgm .Modify
the output_folder variable to make it match your name. For example, you might choose
'../data/at/my_name' . Run the script, wait for it to detect your face in a number of frames (say, 20 or more),
and then press any key to quit. Now, modify the output_folder variable again to make it match the name of a
friend whom you also want to recognize. For example, you might choose '../data/at/name_of_my_friend' . Do
not change the base part of the folder (in this case, '../data/at' ) because later, in the Loading the training data
for face recognition section, we will write code that loads the training images from all of the subfolders of this
base folder. Ask your friend to sit in front of the camera, run the script again, let it detect your friend's face in a
number of frames, and then quit. Repeat this process for any additional people you might want to recognize.Let's
now move on to try and recognize the user's face in a video feed. This should be fun!

Choosing a face recognition algorithm

OpenCV 5 implements three different algorithms for recognizing faces: Eigenfaces, Fisherfaces, and Local
Binary Pattern Histograms (LBPHs). Eigenfaces and Fisherfaces are derived from a more general-purpose
algorithm called Principal Component Analysis (PCA). For a detailed description of the algorithms, refer to the
following links:

PCA: A Tutorial on Principal Component Analysis (2013), by Jonathon Shlens, is available at

http://arxiv.org/pdf/1404.1100v1.pdf. This algorithm was invented in 1901 by Karl Pearson, and the original
paper, On Lines and Planes of Closest Fit to Systems of Points in Space, is available at
http://pca.narod.ru/pearson1901.pdf.
Eigenfaces: The paper Eigenfaces for Recognition (1991), by Matthew Turk and Alex Pentland, is available
at http://www.cs.ucsb.edu/~mturk/Papers/jcn.pdf.
Fisherfaces: The seminal paper The Use of Multiple Measurements in Taxonomic Problems (1936), by R. A.
Fisher, is available at http://onlinelibrary.wiley.com/doi/10.1111/j.1469-1809.1936.tb02137.x/pdf.
Local Binary Pattern: The first paper describing this algorithm is Performance evaluation of texture
measures with classification based on Kullback discrimination of distributions (1994), by T. Ojala, M.
Pietikainen, and D. Harwood. It is available at https://ieeexplore.ieee.org/document/576366.

For this book's purposes, let's just take a high-level overview of the algorithms. First and foremost, they all follow
a similar process; they take a set of classified observations (our face database, containing numerous samples per
individual), train a model based on it, perform an analysis of face images (which may be face regions that we
detected in an image or video), and determine two things: the subject's identity and a measure of confidence that
this identification is correct. The latter is commonly known as the confidence score.Eigenfaces performs PCA,
which identifies principal components of a certain set of observations (again, your face database), calculates the
divergence of the current observation (the face being detected in an image or frame) compared to the dataset, and
produces a value. The smaller the value, the smaller the difference between the face database and the detected face;
hence, a value of 0 is an exact match.Fisherfaces also derives from PCA and evolves the concept, applying more
complex logic. While computationally more intensive, it tends to yield more accurate results than
Eigenfaces.LBPH instead divides a detected face into small cells and, for each cell, builds a histogram that
describes whether the brightness of the image is increasing when comparing neighboring pixels in a given
direction. This cell's histogram can be compared to the histogram of the corresponding cell in the model, producing
a measure of similarity. Of the face recognizers in OpenCV, the implementation of LBPH is the only one that
allows the model sample faces and the detected faces to be of a different shape and size. Hence, it is a convenient
option, and the authors of this book find that its accuracy compares favorably to the other two options.Despite the
algorithms’ differences, OpenCV provides a similar interface for all three, as we shall soon see.

Loading the training data for face recognition

Regardless of our choice of face recognition algorithm, we can load the training images in the same way. Earlier,
in the Generating the data for face recognition section, we generated training images and saved them in folders
that were organized according to people's names or initials. For example, the following folder structure could
contain sample face images of this book's authors, Joseph Howse ( jh ) and Joe Minichino ( jm ):
../
data/
at/
jh/
jm/

Let's write a script that loads these images and labels them in a way that OpenCV's face recognizers will
understand. To work with the filesystem and the data, we will use the Python standard library's os module, as well
as the cv2 and numpy modules. Let's create a script that starts with the following import statements:
import os
import cv2
import numpy

Let's add the following read_images function, which walks through a directory's subdirectories, loads the images
and shows each of them for user information purposes, resizes them to a specified size, and puts the resized images
in a list. The order of this list has no significance in itself but, at the same time, our function builds two
corresponding lists that describe the images: first, a corresponding list of people's names or initials (based on the
subfolder names), and second, a corresponding list of labels or numeric IDs associated with the loaded images. For
example, jh could be a name and 0 could be the label for all images that were loaded from the jh subfolder.
Finally, the function converts the lists of images and labels into NumPy arrays, and it returns three variables: the
list of names, the NumPy array of images, and the NumPy array of labels. Here is the function's implementation:
def read_images(path, image_size):
names = []
training_images, training_labels = [], []
label = 0
for dirname, subdirnames, filenames in os.walk(path):
for subdirname in subdirnames:
names.append(subdirname)
subject_path = os.path.join(dirname, subdirname)
for filename in os.listdir(subject_path):
img = cv2.imread(os.path.join(subject_path, filename),
cv2.IMREAD_GRAYSCALE)
if img is None:
# The file cannot be loaded as an image.
# Skip it.
continue
img = cv2.resize(img, image_size)
cv2.imshow('training', img)
cv2.waitKey(5)
training_images.append(img)
training_labels.append(label)
label += 1
training_images = numpy.asarray(training_images, numpy.uint8)
training_labels = numpy.asarray(training_labels, numpy.int32)
return names, training_images, training_labels

Let's call our read_images function by adding code such as the following:
path_to_training_images = '../data/at'
training_image_size = (200, 200)
names, training_images, training_labels = read_images(
path_to_training_images, training_image_size)

Edit the path_to_training_images variable in the preceding code block to ensure that it matches the base
folder of the output_folder variables you defined earlier in the code for the section Generating the data for
face recognition.

So far, we have training data in a useful format but we have not yet created a face recognizer or performed any
training. We will do so in the next section, where we continue the implementation of the same script.
Performing face recognition with Eigenfaces

Now that we have an array of training images and an array of their labels, we can create and train a face recognizer
with just two more lines of code:
model = cv2.face.EigenFaceRecognizer_create()
model.train(training_images, training_labels)

What have we done here? We created the Eigenfaces face recognizer with OpenCV's
cv2.EigenFaceRecognizer_create function, and we trained the recognizer by passing the arrays of images and
labels (numeric IDs). Optionally, we could have passed two arguments to cv2.EigenFaceRecognizer_create :

num_components : This is the number of components to keep for the PCA.

threshold : This is a floating-point value specifying a confidence threshold.Faces with a confidence score
below the threshold will be discarded. By default, the threshold is the maximum floating-point value so that
no faces are discarded.

Having trained this model, we could save it to a file using code such as model.write('my_model.xml') .
Later (for example, in another script), we could skip the training step and just load the pre-training model
from the file using code such as model.read('my_model.xml') . However, for the sake of our simple demo,
we will just train and test the model in one script without saving the model to a file.

To test this recognizer, let's use a face detector and a video feed from a camera. As we have done in previous
scripts, we can use the following line of code to initialize the face detector:
face_cascade = cv2.CascadeClassifier(
f'{cv2.data.haarcascades}haarcascade_frontalface_default.xml')

The following code initializes the camera feed, iterates over frames (until the user presses any key), and performs
face detection and recognition on each frame:
camera = cv2.VideoCapture(0)
while (cv2.waitKey(1) == -1):
success, frame = camera.read()
if success:
faces = face_cascade.detectMultiScale(frame, 1.3, 5)
for (x, y, w, h) in faces:
cv2.rectangle(frame, (x, y), (x+w, y+h), (255, 0, 0), 2)
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
roi_gray = gray[x:x+w, y:y+h]
if roi_gray.size == 0:
# The ROI is empty. Maybe the face is at the image edge.
# Skip it.
continue
roi_gray = cv2.resize(roi_gray, training_image_size)
label, confidence = model.predict(roi_gray)
text = '%s, confidence=%.2f' % (names[label], confidence)
cv2.putText(frame, text, (x, y - 20),
cv2.FONT_HERSHEY_SIMPLEX, 1, (255, 0, 0), 2)
cv2.imshow('Face Recognition', frame)

Let's walk through the most important functionality of the preceding block of code. For each detected face, we
convert and resize it so that we have a grayscale version that matches the expected size (in this case, 200x200
pixels as defined by the training_image_size variable in the previous section, Loading the training data for face
recognition). Then, we pass the resized, grayscale face to the face recognizer's predict function. This returns a
label and confidence score. We look up the person's name corresponding to the numeric label of that face.
(Remember that we created the names array in the previous section, Loading the training data for face
recognition.) We draw the name and confidence score in blue text above the recognized face. After iterating over
all detected faces, we display the annotated image.

We have taken a simple approach to face detection and recognition, and it serves the purpose of enabling you
to have a basic application running and understand the process of face recognition in OpenCV 5. To improve
 upon this approach and make it more robust, you could take further steps such as correctly aligning and
rotating detected faces so that the accuracy of the recognition is maximized.

When you run the script, you should see something similar to the following screenshot:

Figure 5.3: A face recognition result using live camera input

Next, let's consider how we would adapt these script to replace Eigenfaces with another face recognition
algorithm.

Performing face recognition with Fisherfaces

What about Fisherfaces? The process does not change much; we simply need to instantiate a different algorithm.
With default arguments, the declaration of our model variable would look like this:
model = cv2.face.FisherFaceRecognizer_create()

cv2.face.FisherFaceRecognizer_create takes the same two optional arguments as

cv2.createEigenFaceRecognizer_create : the number of principal components to keep and the confidence
threshold. As you might guess, these types of *_create functions are quite common in OpenCV for situations
where several different algorithms share a common interface. Let’s look at one more example.

Performing face recognition with LBPH

For the LBPH algorithm, again, the process is similar. However, the algorithm factory takes the following optional
parameters (in order):

radius : The pixel distance between the neighbors that are used to calculate a cell's histogram (by default, 1)
neighbors : The number of neighbors used to calculate a cell's histogram (by default, 8)
grid_x : The number of cells into which the face is divided horizontally (by default, 8)
grid_y : The number of cells into which the face is divided vertically (by default, 8)
 confidence : The confidence threshold (by default, the highest possible floating-point value so that no results
are discarded)

With default arguments, the model declaration would look like this:
model = cv2.face.LBPHFaceRecognizer_create()

Note that, with LBPH, we do not need to resize images as the division into grids allows a comparison of
patterns identified in each cell.

Having looked at the available face recognition algorithms, let’s now consider how to evaluate a recognition result.

Discarding results based on the confidence score

The predict method returns a tuple, in which the first element is the label of the recognized individual and the
second is the confidence score. All algorithms come with the option of setting a confidence score threshold, which
measures the distance of the recognized face from the original model; therefore, a score of 0 signifies an exact
match.There may be cases in which you would rather retain all recognitions and then apply further processing, so
you can come up with your own algorithms to estimate the confidence score of a recognition. For example, if you
are trying to identify people in a video, you may want to analyze the confidence score in subsequent frames to
establish whether the recognition was successful or not. In this case, you can inspect the confidence score obtained
by the algorithm and draw your own conclusions.

The typical range of the confidence score depends on the algorithm. Eigenfaces and Fisherfaces produce
values (roughly) in the range of 0 to 20,000, with any score below 4,000-5,000 being a quite confident
recognition. For LBPH, a good recognition scores (roughly) below 50, and any value above 80 is considered a
poor confidence score.

A normal custom approach would be to hold off drawing a rectangle around a recognized face until we have a
number of frames with a good confidence score (where “good” is an arbitrary threshold we must choose, based on
our algorithm and use case), but you have total freedom to use OpenCV's face recognition module to tailor your
application to your needs. Next, let’s see how face detection and recognition work in a specialized use case.

Swapping faces in infrared

Face detection and recognition are not limited to the visible spectrum of light. With a Near-Infrared (NIR)
camera and NIR light source, face detection and recognition are possible even when a scene appears totally dark to
the human eye. This capability is quite useful in security and surveillance applications.We studied the basic usage
of NIR depth cameras, such as the Asus Xtion PRO, in Chapter 4, Depth Estimation and Segmentation. We
extended the object-oriented code of our interactive application, Cameo. We captured frames from a depth camera.
Based on depth, we segmented each frame into a main layer (such as the user's face) and other layers. We painted
the other layers black. This achieved the effect of hiding the background so that only the main layer (the user's
face) appeared on-screen in the interactive video feed.Now, let's modify Cameo to do something that exercises our
previous skills in depth segmentation and our new skills in face detection. Let's detect faces and, when we detect at
least two faces in a frame, let's swap the faces so that one person's head appears atop another person's body. Rather
than copying all pixels in a detected face rectangle, we will only copy the pixels that are part of the main depth
layer for that rectangle. This should achieve the effect of swapping faces but not the background pixels
surrounding the faces.Once the changes are complete, Cameo will be able to produce output such as the following
screenshot:
 Figure 5.4: The result of swapping two faces in a frame from an infrared camera.

Here, we see the face of Joseph Howse swapped with the face of Janet Howse, his mother. Although Cameo is
copying pixels from rectangular regions (and this is clearly visible at the bottom of the swapped regions, in the
foreground), some of the background pixels within the rectangles are masked based on depth and thus are not
swapped, so we do not see rectangular edges everywhere.You can find all of the relevant changes to the Cameo
source code in this book's repository at https://github.com/PacktPublishing/Learning-OpenCV-5-Computer-Vision-
with-Python-Fourth-Edition, specifically in the chapter05/cameo folder. For brevity, we will not discuss all of the
changes here in this book, but we will cover some of the highlights in the next two subsections, Modifying the
application's loop and Masking a copy operation.

Modifying the application's loop

To support face swapping, the Cameo project has two new modules called rects and trackers . The rects
module contains functions for copying and swapping rectangles, with an optional mask to limit the copy or swap
operation to particular pixels. The trackers module contains a class called FaceTracker , which adapts
OpenCV's face detection functionality to an object-oriented style of programming.As we have covered OpenCV's
face detection functionality earlier in this chapter, and we have demonstrated an object-oriented programming style
in previous chapters, we will not go into the FaceTracker implementation here. Instead, you may look at it in this
book's repository.Let's open cameo.py so that we can walk through the overall changes to the application:

1. Near the top of the file, we need to import our new modules, as highlightedhighlighted in the following
code block:
import cv2
import depth
import filters
from managers import WindowManager, CaptureManager
import rects
from trackers import FaceTracker

1. Now, let's turn our attention to changes in the __init__ method of our CameoDepth class. Our updated
application uses an instance of FaceTracker . As part of its functionality, FaceTracker can draw rectangles
around detected faces. Let's give Cameo's user the option to enable or disable the drawing of face rectangles.
We will keep track of the currently selected option via a Boolean variable. The following code block
highlights the necessary changes to initialize the FaceTracker object and the Boolean variable:

class CameoDepth(Cameo):
def __init__(self):
self._windowManager = WindowManager('Cameo',
self.onKeypress)
#device = cv2.CAP_OPENNI2 # uncomment for Kinect
device = cv2.CAP_OPENNI2_ASUS # uncomment for Xtion
self._captureManager = CaptureManager(
cv2.VideoCapture(device), self._windowManager, True)
self._faceTracker = FaceTracker()
self._shouldDrawDebugRects = False
self._curveFilter = filters.BGRPortraCurveFilter()

We make use of the FaceTracker object in the run method of CameoDepth , which contains the application's
main loop that captures and processes frames. Every time we successfully capture a frame, we call methods of
FaceTracker to update the face detection result and get the latest detected faces. Then, for each face, we create a
mask based on the depth camera's disparity map. (Previously, in Chapter 4, Depth Estimation and Segmentation,
we created such a mask for the entire image instead of a mask for each face rectangle.)

1. Then, we call a function, rects.swapRects , to perform a masked swap of the face rectangles. (We will look
at the implementation of swapRects a little later, in the Masking a copy operation section.) Depending on the
currently selected option, we might tell FaceTracker to draw rectangles around the faces. All of the relevant
changes are highlighted in the following code block:
def run(self):
"""Run the main loop."""
self._windowManager.createWindow()
while self._windowManager.isWindowCreated:
# ... The logic for capturing a frame is unchanged ...
if frame is not None:
self._faceTracker.update(frame)
faces = self._faceTracker.faces
masks = [
depth.createMedianMask(
disparityMap, validDepthMask,
face.faceRect) \
for face in faces
]
rects.swapRects(frame, frame,
[face.faceRect for face in faces],
masks)
if self._captureManager.channel == cv2.CAP_OPENNI_BGR_IMAGE:
# A BGR frame was captured.
# Apply filters to it.
filters.strokeEdges(frame, frame)
self._curveFilter.apply(frame, frame)
if self._shouldDrawDebugRects:
self._faceTracker.drawDebugRects(frame)
self._captureManager.exitFrame()
self._windowManager.processEvents()

1. Finally, let's modify the onKeypress method so that the user can hit the X key to start or stop displaying
rectangles around detected faces. Again, the relevant changes are highlighted in the following code block:
 def onKeypress(self, keycode):
"""Handle a keypress.
space -> Take a screenshot.
tab -> Start/stop recording a screencast.
x -> Start/stop drawing debug rectangles around faces.
escape -> Quit.
"""
if keycode == 32: # space
self._captureManager.writeImage('screenshot.png')
elif keycode == 9: # tab
if not self._captureManager.isWritingVideo:
self._captureManager.startWritingVideo(
'screencast.avi')
else:
self._captureManager.stopWritingVideo()
elif keycode == 120: # x
self._shouldDrawDebugRects = \
not self._shouldDrawDebugRects
elif keycode == 27: # escape
self._windowManager.destroyWindow()

Next, let's look at the implementation of the rects module that we imported earlier in this section.

Masking a copy operation

The rects module is implemented in rects.py . We already saw a call to the rects.swapRects function in the
previous section. However, before we consider the implementation of swapRects , we first need a more basic
copyRect function.As far back as Chapter 2, Handling Files, Cameras, and GUIs, we learned how to copy data
from one rectangular region of interest (ROI) to another using NumPy's slicing syntax. Outside the ROIs, the
source and destination images were unaffected. Now, we want to apply further limits to this copy operation. We
want to use a given mask that has the same dimensions as the source rectangle.We shall copy only those pixels in
the source rectangle where the mask's value is not zero. Other pixels shall retain their old values from the
destination image. This logic, with an array of conditions and two arrays of possible output values, can be
expressed concisely with the numpy.where function.With this approach in mind, let's consider our copyRect
function. As arguments, it takes a source and destination image, a source and destination rectangle, and a mask.
The latter may be None , in which case, we simply resize the content of the source rectangle to match the
destination rectangle and then assign the resulting resized content to the destination rectangle. Otherwise, we next
ensure that the mask and the images have the same number of channels. We assume that the mask has one channel
but the images may have three channels (BGR). We can add duplicate channels to mask using the repeat and
reshape methods of numpy.array . Finally, we perform the copy operation using numpy.where . The complete
implementation is as follows:
def copyRect(src, dst, srcRect, dstRect, mask = None,
interpolation = cv2.INTER_LINEAR):
"""Copy part of the source to part of the destination."""
x0, y0, w0, h0 = srcRect
x1, y1, w1, h1 = dstRect
# Resize the contents of the source sub-rectangle.
# Put the result in the destination sub-rectangle.
if mask is None:
dst[y1:y1+h1, x1:x1+w1] = \
cv2.resize(src[y0:y0+h0, x0:x0+w0], (w1, h1),
interpolation = interpolation)
else:
if not utils.isGray(src):
# Convert the mask to 3 channels, like the image.
mask = mask.repeat(3).reshape(h0, w0, 3)
# Perform the copy, with the mask applied.
dst[y1:y1+h1, x1:x1+w1] = \
numpy.where(cv2.resize(mask, (w1, h1),
interpolation = \
cv2.INTER_NEAREST),
cv2.resize(src[y0:y0+h0, x0:x0+w0], (w1, h1),
interpolation = interpolation),
dst[y1:y1+h1, x1:x1+w1])
We also need to define a swapRects function, which uses copyRect to perform a circular swap of a list of
rectangular regions. swapRects has a masks argument, which is a list of masks whose elements are passed to the
respective copyRect calls. If the value of the masks argument is None , we pass None to every copyRect call.
The following code shows the full implementation of swapRects :
def swapRects(src, dst, rects, masks = None,
interpolation = cv2.INTER_LINEAR):
"""Copy the source with two or more sub-rectangles swapped."""
if dst is not src:
dst[:] = src
numRects = len(rects)
if numRects < 2:
return
if masks is None:
masks = [None] * numRects
# Copy the contents of the last rectangle into temporary storage.
x, y, w, h = rects[numRects - 1]
temp = src[y:y+h, x:x+w].copy()
# Copy the contents of each rectangle into the next.
i = numRects - 2
while i >= 0:
copyRect(src, dst, rects[i], rects[i+1], masks[i],
interpolation)
i -= 1
# Copy the temporarily stored content into the first rectangle.
copyRect(temp, dst, (0, 0, w, h), rects[0], masks[numRects - 1],
interpolation)

Note that the mask argument in copyRect and the masks argument in swapRects both have a default value of
None . If no mask is specified, these functions copy or swap the entire contents of the rectangle or rectangles.

Summary
By now, you should have a good understanding of how face detection and face recognition work and how to
implement them in Python and OpenCV 5.The accuracy of detection and recognition algorithms heavily depends
on the quality of the training data, so make sure you provide your applications with a large number of training
images covering a variety of expressions, poses, and lighting conditions. Later in this book, in Chapter 11, Neutral
Networks with OpenCV – an Introduction, we will look at how to use several robust, pre-trained face detection
models that build atop advanced algorithms and large sets of training data.As human beings, we might be
predisposed to think that human faces are particularly recognizable. We might even be overconfident in our own
face recognition abilities. However, in computer vision, there is nothing very special about human faces, and we
can just as readily use algorithms to find and identify other things. We will begin to do so next in Chapter 6,
Retrieving Images and Searching Using Image Descriptors.
Join our book community on Discord
https://discord.gg/djGjeMECxw
6 Retrieving Images and Searching
Using Image Descriptors
Similar to the human eyes and brain, OpenCV can detect the main features of an
image and extract them into so-called image descriptors. These features can then be
used as a database, enabling image-based searches. Moreover, we can use key points
to stitch images together and compose a bigger image. (Think of putting together
many pictures to form a 360° panorama.)This chapter will show you how to detect
the features of an image with OpenCV and make use of them to match and search
images. Over the course of this chapter, we will take sample images and detect their
main features, and then try to find a region of another image that matches the sample
image. We will also find the homography or spatial relationship between a sample
image and a matching region of another image.More specifically, we will cover the
following tasks:

Detecting keypoints and extracting local descriptors around the keypoints using
any of the following algorithms: Harris corners, SIFT, SURF, or ORB
Matching keypoints using brute-force algorithms or the FLANN algorithm
Filtering out bad matches using KNN and the ratio test
Finding the homography between two sets of matching keypoints
Searching a set of images to determine which one contains the best match for a
reference image

We will finish this chapter by building a proof-of-concept forensic application.

Given a reference image of a tattoo, we will search for a set of images of people in
order to find a person with a matching tattoo.

Technical requirements
This chapter uses Python, OpenCV, and NumPy. In regards to OpenCV, we use the
optional opencv_contrib modules, which include additional algorithms for
keypoint detection and matching. To enable theSURF algorithm (whichis patented
and not free for commercial use), we must configure the opencv_contrib modules
with the OPENCV_ENABLE_NONFREE flag in CMake. Please refer to Chapter 1, Setting
Up OpenCV, for installation instructions. Additionally, if you have not already
installed Matplotlib, install it by running $ pip install matplotlib (or
$ pip3 install matplotlib , depending on your environment).The complete code
for this chapter can be found in this book's GitHub repository,
https://github.com/PacktPublishing/Learning-OpenCV-5-Computer-Vision-with-
Python-Fourth-Edition, in the chapter06 folder. The sample images can be found
in the images folder.A subset of the chapter’s sample code can be edited and run
interactively in Google Colab at
https://colab.research.google.com/github/PacktPublishing/Learning-OpenCV-5-
Computer-Vision-with-Python-Fourth-
Edition/blob/main/chapter06/chapter06.ipynb.

Understanding types of feature detection and

matching
A number of algorithms can be used to detect and describe features, and we will
explore several of them in this section. The most commonly used feature detection
and descriptor extraction algorithms in OpenCV are as follows:

Harris: This algorithm is useful for detecting corners.

SIFT: This algorithm is useful for detecting blobs.
SURF: This algorithm is useful for detecting blobs.
FAST: This algorithm is useful for detecting corners.
BRIEF: This algorithm is useful for detecting blobs.
ORB: This algorithm stands for Oriented FAST and Rotated BRIEF. It is
useful for detecting a combination of corners and blobs.

Matching features can be performed with the following methods:

Brute-force matching
FLANN-based matching

Spatial verification can then be performed with homography.We have just

introduced a lot of new terminology and algorithms. Now, we will go over their
basic definitions.

Defining features

What is a feature, exactly? Why is a particular area of an image classifiable as a

feature, while others are not? Broadly speaking, a feature is an area of interest in the
image that is unique or easily recognizable. Corners and regions with a high density
of textural detail are good features, while patterns that repeat themselves a lot and
low-density regions (such as a blue sky) are not. Edges are good features as they
tend to divide two regions with abrupt changes in the intensity (gray or color) values
of an image. A blob (a region of an image that greatly differs from its surrounding
areas) is also an interesting feature.Most feature detection algorithms revolve around
the identification of corners, edges, and blobs, with some also focusing on the
concept of a ridge, which you can conceptualize as the axis of symmetry of an
elongated object. (Think, for example, about identifying a road in an image.)Some
algorithms are better at identifying and extracting features of a certain type, so it is
important to know what your input image is so that you can utilize the best tool in
your OpenCV belt.

Detecting Harris corners

Let's start by finding corners using the Harris corner detection algorithm. We will do
this by implementing an example. If you continue to study OpenCV beyond this
book, you will find that chessboards are a common subject of analysis in computer
vision, partly because a checkered pattern is suited to many types of feature
detection, and partly because chess is a popular pastime, especially in Russia, which
is the country of origin of many of OpenCV's developers.Here is our sample image
of a chessboard and chess pieces:
 Figure 6.1: A chessboard, to be used for corner detection

OpenCV has a handy function called cv2.cornerHarris , which detects corners in

an image. We can see this function at work in the following basic example:
import cv2
img = cv2.imread('../images/chess_board.png')
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
dst = cv2.cornerHarris(gray, 2, 23, 0.04)
img[dst > 0.01 * dst.max()] = [0, 0, 255]
cv2.imshow('corners', img)
cv2.waitKey()
Let's analyze the code. After the usual imports, we load the chessboard image and
convert it into grayscale. Then, we call the cornerHarris function:
dst = cv2.cornerHarris(gray, 2, 23, 0.04)

The most important parameter here is the third one, which defines the aperture or
kernel size of the Sobel operator. The Sobel operator detects edges by measuring
horizontal and vertical differences between pixel values in a neighborhood, and it
does this using a kernel. The cv2.cornerHarris function uses a Sobel operator
whose aperture is defined by this parameter. In plain English, the parameters define
how sensitive corner detection is. It must be between 3 and 31 and be an odd value.
With a low (highly sensitive) value of 3 , all those diagonal lines in the black
squares of the chessboard will register as corners when they touch the border of the
square. For a higher (less sensitive) value of 23 , only the corners of each square
will be detected as corners. cv2.cornerHarris returns an image in floating-point
format. Each value in this image represents a score for the corresponding pixel in
the source image. A moderate or high score indicates that the pixel is likely to be a
corner. Conversely, we can treat pixels with the lowest scores as non-corners.
Consider the following line:
img[dst > 0.01 * dst.max()] = [0, 0, 255]

Here, we select pixels with scores that are at least 1% of the highest score, and we
color these pixels red in the original image. Here is the result:
 Figure 6.2: Corners detections in a chessboard

Great! Nearly all the detected corners are marked in red. The marked points include
nearly all the corners of the chessboard's squares.

If we tweak the second parameter in cv2.cornerHarris , we will see that

smaller regions (for a smaller parameter value) or larger regions (for a larger
parameter value) will be detected as corners. This parameter is called the block
size.
Detecting DoG features and extracting SIFT
descriptors
The preceding technique, which uses cv2.cornerHarris , is great for detecting
corners and has a distinct advantage because even if the image is rotated corners are
still the corners. However, if we scale an image to a smaller or larger size, some
parts of the image may lose or even gain a corner quality.For example, take a look at
the following corner detections in an image of the F1 Italian Grand Prix track:

Figure 6.3: Corner detections in an image of the F1 Italian Grand Prix track

Here is the corner detection result with a smaller version of the same image:
Figure 6.4: Corner detections in a smaller image of the F1 Italian Grand Prix track

You will notice how the corners are a lot more condensed; however, even though we
gained some corners, we lost others! In particular, let's examine the Variante Ascari
chicane, which looks like a squiggle at the end of the part of the track that runs
straight from northwest to southeast. In the larger version of the image, both the
entrance and the apex of the double bend were detected as corners. In the smaller
image, the apex is not detected as such. If we further reduce the image, at some
scale, we will lose the entrance to that chicane too.This loss of features raises an
issue; we need an algorithm that works regardless of the scale of the image. Enter
Scale-Invariant Feature Transform (SIFT). While the name may sound a bit
mysterious, now that we know what problem we are trying to solve, it actually
makes sense. We need a function (a transform) that will detect features (a feature
transform) and will not output different results depending on the scale of the image
(a scale-invariant feature transform). Note that SIFT does not detect keypoints.
Keypoint detection is done with the Difference of Gaussians (DoG), while SIFT
describes the region surrounding the keypoints by means of a feature vector.A quick
introduction to the DoG is in order. Previously, in Chapter 3, Processing Images
with OpenCV, we talked about low pass filters and blurring operations, and
specifically the cv2.GaussianBlur function. DoG is the result of applying different
Gaussian filters to the same image. Previously, we applied this type of technique for
edge detection, and the idea is the same here. The final result of a DoG operation
contains areas of interest (keypoints), which are then going to be described through
SIFT.Let's see how DoG and SIFT behave in the following image, which is full of
corners and features:

Figure 6.5: SIFT descriptors for an image of Varese, Lombardy, Italy

Here, the beautiful panorama of Varese (in Lombardy, Italy) gains a new type of
fame as a subject of computer vision. Here is the code that produces this processed
image:
import cv2
img = cv2.imread('../images/varese.jpg')
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
sift = cv2.SIFT_create()
keypoints, descriptors = sift.detectAndCompute(gray, None)
cv2.drawKeypoints(img, keypoints, img, (51, 163, 236),
cv2.DRAW_MATCHES_FLAGS_DRAW_RICH_KEYPOINTS)
cv2.imshow('sift_keypoints', img)
cv2.waitKey()

After the usual imports, we load the image we want to process. Then, we convert the
image into grayscale. By now, you may have gathered that many methods in
OpenCV expect a grayscale image as input. The next step is to create a SIFT
detection object and compute the features and descriptors of the grayscale image:
sift = cv2.SIFT_create()
keypoints, descriptors = sift.detectAndCompute(gray, None)
Behind the scenes, these simple lines of code carry out an elaborate process; we
create a cv2.SIFT object, which uses DoG to detect keypoints and then computes a
feature vector for the surrounding region of each keypoint. As the name of the
detectAndCompute method clearly suggests, two main operations are performed:
feature detection and the computation of descriptors. The return value of the
operation is a tuple containing a list of keypoints and another list of the keypoints'
descriptors.Finally, we process this image by drawing the keypoints on it with the
cv2.drawKeypoints function and then displaying it with the usual cv2.imshow
function. As one of its arguments, the cv2.drawKeypoints function accepts a flag
that specifies the type of visualization we want. Here, we specify
cv2.DRAW_MATCHES_FLAGS_DRAW_RICH_KEYPOINT in order to draw a visualization of
the scale and orientation of each keypoint.

Anatomy of a keypoint

Each keypoint is an instance of the cv2.KeyPoint class, which has the following
properties:

The pt (point) property contains the x and y coordinates of the keypoint in the
image.
The size property indicates the diameter of the feature.
The angle property indicates the orientation of the feature, as shown by the
radial lines in the preceding processed image.
The response property indicates the strength of the keypoint. Some features
are classified by SIFT as stronger than others, and response is the property
you would check to evaluate the strength of a feature.
The octave property indicates the layer in the image pyramid where the
feature was found. Let's briefly review the concept of an image pyramid, which
we discussed previously in Chapter 5, Detecting and Recognizing Faces, in the
Conceptualizing Haar cascades section. The SIFT algorithm operates in a
similar fashion to face detection algorithms in that it processes the same image
iteratively but alters the input at each iteration. In particular, the scale of the
image is a parameter that changes at each iteration ( octave ) of the algorithm.
Thus, the octave property is related to the image scale at which the keypoint
was detected.
Finally, the class_id property can be used to assign a custom identifier to a
keypoint or a group of keypoints.

Detecting Fast Hessian features and extracting SURF

descriptors
Computer vision is a relatively young branch of computer science, so many famous
algorithms and techniques have only been invented recently. SIFT is, in fact, only 23
years old, having been published by David Lowe in 1999.SURF is a feature
detection algorithm that was published in 2006 by Herbert Bay. SURF is several
times faster than SIFT, and it is partially inspired by it.

Note that SURF is a patented algorithm and, for this reason, is made available
only in builds of opencv_contrib where the OPENCV_ENABLE_NONFREE
CMake flag is used. SIFT was formerly a patented algorithm but its patent
expired and now SIFT is available in standard builds of OpenCV.

It is not particularly relevant to this book to understand how SURF works under the
hood, inasmuch as we can use it in our applications and make the best of it. What is
important to understand is that cv2.SURF is an OpenCV class that performs
keypoint detection with the Fast Hessian algorithm and descriptor extraction with
SURF, much like the cv2.SIFT class performs keypoint detection with DoG and
descriptor extraction with SIFT.Also, the good news is that OpenCV provides a
standardized API for all its supported feature detection and descriptor extraction
algorithms. Thus, with only trivial changes, we can adapt our previous code sample
to use SURF instead of SIFT. Here is the modified code, with the changes
highlighted :

import cv2
img = cv2.imread('../images/varese.jpg')
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
surf = cv2.xfeatures2d.SURF_create(8000)
keypoints, descriptor = surf.detectAndCompute(gray, None)
cv2.drawKeypoints(img, keypoints, img, (51, 163, 236),
cv2.DRAW_MATCHES_FLAGS_DRAW_RICH_KEYPOINTS)
cv2.imshow('surf_keypoints', img)
cv2.waitKey()

The parameter to cv2.xfeatures2d.SURF_create is a threshold for the Fast

Hessian algorithm. By increasing the threshold, we can reduce the number of
features that will be retained. With a threshold of 8000 , we get the following result:
 Figure 6.6: SURF descriptors for an image of Varese, Lombardy, Italy

Try adjusting the threshold to see how it affects the result. As an exercise, you may
want to build a GUI application with a slider that controls the value of the threshold.
This way, a user can adjust the threshold and see the number of features increase and
decrease in an inversely proportional fashion. We built a GUI application with
sliders in Chapter 4, Depth Estimation and Segmentation, in the Depth estimation
with a normal camera section, so you may want to refer back to that section as a
guide.Next, we'll examine the FAST corner detector, the BRIEF keypoint descriptor,
and ORB (which uses FAST and BRIEF together).

Using ORB with FAST features and BRIEF descriptors

If SIFT is young, and SURF younger, ORB is in its infancy. ORB was first
published in 2011 as a fast alternative to SIFT and SURF.The algorithm was
published in the paper ORB: an efficient alternative to SIFT or SURF, available in
PDF format at http://www.willowgarage.com/sites/default/files/orb_final.pdf.ORB
mixes the techniques used in the FAST keypoint detector and the BRIEF keypoint
descriptor, so it is worth taking a quick look at FAST and BRIEF first. Then, we will
talk about brute-force matching – an algorithm used for feature matching – and look
at an example of feature matching.

FAST
The Features from Accelerated Segment Test (FAST) algorithm works by
analyzing circular neighborhoods of 16 pixels. It marks each pixel in a
neighborhood as brighter or darker than a particular threshold, which is defined
relative to the center of the circle. A neighborhood is deemed to be a corner if it
contains a number of contiguous pixels marked as brighter or darker.FAST also uses
a high-speed test, which can sometimes determine that a neighborhood is not a
corner by checking just 2 or 4 pixels instead of 16. To understand how this test
works, let's take a look at the following diagram, taken from the OpenCV
documentation:

Figure 6.7: How the FAST algorithm analyzes a neighbourhood around a corner

Here, we can see a 16-pixel neighborhood at two different magnifications. The

pixels at positions 1, 5, 9, and 13 correspond to the four cardinal points at the edge
of the circular neighborhood. If the neighborhood is a corner, we expect that out of
these four pixels, exactly three or exactly one will be brighter than the threshold.
(Another way of saying this is that exactly one or exactly three of them will be
darker than the threshold.) If exactly two of them are brighter than the threshold,
then we have an edge, not a corner. If exactly four or exactly zero of them are
brighter than the threshold, then we have a relatively uniform neighborhood that is
neither a corner nor an edge.FAST is a clever algorithm, but it's not devoid of
weaknesses, and to compensate for these weaknesses, developers analyzing images
can implement a machine learning approach in order to feed a set of images
(relevant to a given application) to the algorithm so that parameters such as the
threshold are optimized. Whether the developer specifies parameters directly or
provides a training set for a machine learning approach, FAST is an algorithm that is
sensitive to the developer's input, perhaps more so than SIFT.

BRIEF

Binary Robust Independent Elementary Features (BRIEF), on the other hand, is

not a feature detection algorithm, but a descriptor. Let's delve deeper into the
concept of what a descriptor is, and then look at BRIEF.When we previously
analyzed images with SIFT and SURF, the heart of the entire process was the call to
the detectAndCompute function. This function performs two different steps –
detection and computation – and they return two different results, coupled in a
tuple.The result of detection is a set of keypoints; the result of the computation is a
set of descriptors for those keypoints. This means that OpenCV's cv2.SIFT and
cv2.xfeatures2d.SURF classes implement algorithms for both detection and
description. Remember, though, that the original SIFT and SURF are not feature
detection algorithms. OpenCV's cv2.SIFT implements DoG feature detection plus
SIFT description, while OpenCV's cv2.xfeatures2d.SURF implements Fast
Hessian feature detection plus SURF description.Keypoint descriptors are a
representation of the image that serves as the gateway to feature matching because
you can compare the keypoint descriptors of two images and find
commonalities.BRIEF is one of the fastest descriptors currently available. The
theory behind BRIEF is quite complicated, but suffice it to say that BRIEF adopts a
series of optimizations that make it a very good choice for feature matching.

Brute-force matching

A brute-force matcher is a descriptor matcher that compares two sets of keypoint

descriptors and generates a result that is a list of matches. It is called brute-force
because little optimization is involved in the algorithm. For each keypoint descriptor
in the first set, the matcher makes comparisons to every keypoint descriptor in the
second set. Each comparison produces a distance value and the best match can be
chosen on the basis of least distance.More generally, in computing, the term brute-
force is associated with an approach that prioritizes the exhaustion of all possible
combinations (for example, all the possible combinations of characters to crack a
password of a known length). Conversely, an algorithm that prioritizes speed might
skip some possibilities and try to take a shortcut to the solution that seems the most
plausible.OpenCV provides a cv2.BFMatcher class that supports several
approaches to brute-force feature matching.

Matching a logo in two images

Now that we have a general idea of what FAST and BRIEF are, we can understand
why the team behind ORB (composed of Ethan Rublee, Vincent Rabaud, Kurt
Konolige, and Gary R. Bradski) chose these two algorithms as a foundation for
ORB.In their paper, the authors set out to achieve the following results:

The addition of a fast and accurate orientation component to FAST

The efficient computation of oriented BRIEF features
Analysis of variance and correlation of oriented BRIEF features
A learning method to decorrelate BRIEF features under rotational invariance,
leading to better performance in nearest-neighbor applications

The main points are quite clear: ORB aims to optimize and speed up operations,
including the very important step of utilizing BRIEF in a rotation-aware fashion so
that matching is improved, even in situations where a training image has a very
different rotation to the query image.At this stage, though, perhaps you have had
enough of the theory and want to sink your teeth into some feature matching, so let's
look at some code. The following script attempts to match features in a logo to the
features in a photograph that contains the logo:
import cv2
from matplotlib import pyplot as plt
# Load the images.
img0 = cv2.imread('../images/nasa_logo.png',
cv2.IMREAD_GRAYSCALE)
img1 = cv2.imread('../images/kennedy_space_center.jpg',
cv2.IMREAD_GRAYSCALE)
# Perform ORB feature detection and description.
orb = cv2.ORB_create()
kp0, des0 = orb.detectAndCompute(img0, None)
kp1, des1 = orb.detectAndCompute(img1, None)
# Perform brute-force matching.
bf = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck=True)
matches = bf.match(des0, des1)
# Sort the matches by distance.
matches = sorted(matches, key=lambda x:x.distance)
# Draw the best 25 matches.
img_matches = cv2.drawMatches(
img0, kp0, img1, kp1, matches[:25], img1,
flags=cv2.DRAW_MATCHES_FLAGS_NOT_DRAW_SINGLE_POINTS)
# Show the matches.
plt.imshow(img_matches)
plt.show()

Let's examine this code step by step. After the usual imports, we load two images
(the query image and the scene) in grayscale format. Here is the query image, which
is the NASA logo:
 Figure 6.8: The NASA logo, to be used for keypoint descriptor matching

Here is the photo of the scene, which is the Kennedy Space Center:
 Figure 6.9: The Kennedy Space Center, with a NASA logo on its wall

Now, we proceed to create the ORB feature detector and descriptor:

# Perform ORB feature detection and description.
orb = cv2.ORB_create()
kp0, des0 = orb.detectAndCompute(img0, None)
kp1, des1 = orb.detectAndCompute(img1, None)

In a similar fashion to what we did with SIFT and SURF, we detect and compute the
keypoints and descriptors for both images.From here, the concept is pretty simple:
iterate through the descriptors and determine whether they are a match or not, and
then calculate the quality of this match (distance) and sort the matches so that we
can display the top n matches with a degree of confidence that they are, in fact,
matching features on both images. cv2.BFMatcher does this for us:
# Perform brute-force matching.
bf = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck=True)
matches = bf.match(des0, des1)
# Sort the matches by distance.
matches = sorted(matches, key=lambda x:x.distance)
Exploring the Variety of Random
Documents with Different Content
however, in Scotland where he met any such troubles. With immense
difficulty, he next day crossed Mount Skene by an uneven stony way,
full of bogs, quagmires, and long heath, ‘where a dog with three
legs would outrun a horse with four,’ and came in the evening to
Braemar. This he describes as a large county, full of lofty mountains,
compared with which English hills are but ‘as a liver or a gizzard
below a capon’s wing.’ ‘There I saw Benawne [Ben Aven], with a
furred mist upon his snowy head, instead of a night-cap.’
He here found his friend, Sir William Murray, engaged in Highland
sports, along with the Earl of Mar, the Earl of Enzie (afterwards
second Marquis of Huntly), the Earl of Buchan, and Lord Erskine,
accompanied by their countesses, and a hundred other knights and
squires, with their followers, ‘all in general in one habit, as if
Lycurgus had been there.’ ‘For once in the year, which is the whole
month of August, and sometimes part of September, many of the
nobility and gentry of the kingdom, for their pleasure, come into
these Highland countries to hunt, where they conform to the habit of
the Highlandmen, who for the most part speak nothing but Irish....
Their habit is shoes with but one sole apiece; stockings which they
call short hose, made of a warm stuff of divers colours, which they
call tartan: as for breeches, many of them, nor their forefathers,
never wore any, but a jerkin of the same stuff that their hose is of,
their garters being bands or wreaths of hay or straw, with a plaid
about their shoulders, which is a mantle of divers colours, [of] much
finer and lighter stuff than their hose; with flat blue caps on their
heads, a handkerchief knit with two knots about their neck; and thus
they are attired.... Their weapons are long bows and forked arrows,
swords and targets, harquebusses, muskets, durks, and Lochaber
axes. With these arms, I found many of them armed for the hunting.
As for their attire, any man of what degree soever that comes
amongst them, must not disdain to wear it; for if they do, they will
disdain to hunt, or willingly to bring in their dogs; but if men be kind
to them, and be in their habit, then they are conquered with
kindness, and the sport will be plentiful. This was the reason that I
found so many noblemen and gentlemen in those shapes.’
Taylor allowed himself to be invested by the Earl of
Mar in Highland attire, and then accompanied the 1618.
party for twelve days into a wilderness devoid of
corn and human habitations—probably the district around the skirts
of Ben Muicdhui. He found temporary lodges called lonchards,
designed for the use of the sportsmen, and he himself received a
kind of accommodation in that of Lord Erskine. The kitchen, he tells
us, was ‘always on the side of a bank, many kettles and pots boiling,
and many spits turning and winding, with great variety of cheer, as
venison—baked, sodden, roast, and stewed beef—mutton, goats,
kid, hares, fresh salmon, pigeons, hens, capons, chickens, partridge,
moorcoots, heath-cocks, cappercailzies, and termagants; good ale,
sack, white and claret, tent (or Alicant), with most potent aquavitæ.’
Thus a company of about fourteen hundred persons was most amply
fed.
‘The manner of the hunting is this: five or six hundred men rise early
in the morning, and disperse themselves divers ways, and seven,
eight, or ten miles compass, they bring or chase in the deer in many
herds (two, three, or four hundred in a herd), to such or such a
place, as the noblemen shall appoint them. Then, when day is come,
the lords and gentlemen of their companies ride or go to the said
places, sometimes wading up to the middle, through burns and
rivers; and then they, being come to the place, lie down on the
ground, till those foresaid scouts, who are called the Tinchel-men,
bring down the deer.... After we had stayed there three hours or
thereabouts, we might perceive the deer appear on the hills round
about us (their heads making a show like a wood), which, being
followed close by the Tinchel, are chased down into the valley where
we lay. Then, all the valley on each side being waylaid with a
hundred couple of strong Irish greyhounds, they are let loose, as
occasion serves, upon the herd of deer, [so] that with dogs, guns,
arrows, durks, and daggers, in the space of two hours, fourscore fat
deer were slain, which after are disposed, some one way and some
another, twenty or thirty miles, and more than enough left for us to
make merry withal at our rendezvous’.
After spending some days in this manner in the
Brae of Mar, the party, attended by Taylor, went 1618.
into Badenoch, and renewed the sport there for
three or four days, concluding with a brief visit to Ruthven Castle.
This grand old fortress—anciently the stronghold of the Cumins,
lords of Badenoch—seated on an alluvial promontory jutting into the
haugh beside the Spey, occupying an area of a hundred and twenty
yards long, and consisting of two great towers surrounded by a
fortified wall with an iron gate and portcullis,383 was now the
property of the Gordon family. Here, says Taylor, ‘my Lord of Enzie
and his noble countess (being daughter to the Earl of Argyle) did
give us most noble welcome for three days.’ ‘From thence we went
to a place called Ballo[ch] Castle, a fair and stately house, a worthy
gentleman being the owner of it, called the Laird of Grant.384... Our
cheer was more than sufficient, and yet much less than they could
afford us. There stayed there four days four earls, one lord, divers
knights and gentlemen, and their servants, footmen, and horses;
and every meal four long tables furnished with all varieties; our first
and second course being threescore dishes at one board; and after
that always a banquet; and there, if I had not forsworn wine till I
came to Edinburgh, I think I had there drank my last.’
The Water-poet was afterwards four days at Tarnaway, entertained
in the same hospitable manner by the Earl and Countess of Moray.
He speaks of Morayland as the pleasantest and most plentiful
country in Scotland, ‘being plain land, that a coach may be driven
more than four-and-thirty miles one way in it, alongst the sea-coast,’
He spent a few days with the Marquis of Huntly at the Bog, ‘where
our entertainment was, like himself, free, bountiful, and honourable,’
and then returned by the Cairn-a-mount to Edinburgh.
Here he was again in the midst of plentiful good cheer and good
company for eight days, while recovering from certain bruises he
had got at the Highland hunting. In Leith, at the house of Mr John
Stuart, he found his ‘long approved and assured good friend, Mr
Benjamin Jonson,’ who gave him a piece of gold of the value of
twenty-two shillings, to drink his health in England. ‘So with a
friendly farewell, I left him as well as I hope never to see him in a
worse estate; for he is among noblemen and gentlemen that know
his true worth and their own honours, where with much respective
love he is worthily entertained.’
In short, Taylor, in his progress through Scotland, seems to have
been everywhere feasted sumptuously, and supplied liberally with
money. So much of a virtue comparatively rare in England, and so
much plenty in a country which his own people were accustomed to
think of as the birthplace of famine, seems to have greatly
astonished him. The wonder comes to a climax at Cockburnspath,
near his exit from Scotland, where he was handsomely entertained
at an inn by Master William Arnot and his wife, the owners thereof. ‘I
must explain,’ he says, ‘their bountiful entertainment of guests,
which is this:
‘Suppose ten, fifteen, or twenty men and horses
come to lodge at their house. The men shall have 1618.
flesh, tame and wild fowl, fish, with all variety of
good cheer, good lodging, and welcome, and the horses shall want
neither hay nor provender; and at the morning at their departure the
reckoning is just nothing. This is this worthy gentleman’s use, his
chief delight being to give strangers entertainment gratis! And I am
sure that in Scotland, beyond Edinburgh, I have been at houses like
castles for building; the master of the house’s beaver being his blue
bonnet, one that will wear no other shirts but of the flax that grows
on his own ground, and of his wife’s, daughters’, or servants’
spinning; that hath his stockings, hose, and jerkin of the wool of his
own sheep’s backs; that never by his pride of apparel caused mercer,
draper, silk-man, embroiderer, or haberdasher, to break and turn
bankrupt; and yet this plain home-spun fellow keeps and maintains
thirty, forty, fifty servants, or perhaps more, every day relieving three
or four score poor people at his gate; and besides all this, can give
noble entertainment for four days together to five or six earls and
lords, besides knights, gentlemen, and their followers, if they be
three or four hundred men and horse of them, where they shall not
only feed but feast, and not feast but banquet; this is a man that
desires to know nothing so much as his duty to God and his king,
whose greatest cares are to practise the works of piety, charity, and
hospitality. He never studies the consuming art of fashionless
fashions; he never tries his strength to bear four or five hundred
acres on his back at once; his legs are always at liberty, not being
fettered with golden garters and manacled with artificial roses....
Many of these worthy housekeepers there are in Scotland....

‘There th’ Almighty doth his blessings heap,

In such abundant food for beasts and men,
That I ne’er saw more plenty or more cheap.’385

In the summer of this year, Scotland received a

visit from the famous Ben Jonson. The burly 1618.
laureate walked all the way, notwithstanding a
previous hint from Lord Bacon, that ‘he loved not to see Poesy go on
other feet than poetical Dactylus and Spondæus.’ Among the
motives for a journey then undertaken by few Englishmen, might be
curiosity regarding a country from which he knew that his family was
derived, his grandfather having been one of the Johnstons of
Annandale. He had many friends, too, particularly among the
connections of the Lennox family, whom he might be glad to see at
their own houses. Indeed, his biographer, Gifford, expressly says he
had received from some of these friends an invitation to spend some
time amongst them in the north. In September, he was found by
Taylor the water-poet, residing with one Mr John Stuart in Leith; and
Taylor speaks of him generally as being then ‘among noblemen and
gentlemen who knew his true worth and their own honours, where
with much respective love he is entertained.’
Among those with whom he had amicable intercourse, was William
Drummond, the poet, then in the prime of life, and living as a
bachelor in his romantic mansion of Hawthornden, on the Esk, seven
miles from Edinburgh. It is probable that Drummond and Jonson had
met before in London, and indulged together in the ‘wit-combats’ at
the Mermaid and similar scenes. Indeed, there is a prevalent belief
in Scotland that it was mainly to see Drummond at Hawthornden
that Jonson came so far from home. The story is, that the pilgrim
came first to Hawthornden, and was received by Drummond with
wonted ceremony, under the Covine Tree—that is, company tree—
which still stands on the lawn in front of the house, when their
greetings very appropriately took the form of a couplet:

D. Welcome, welcome, royal Ben!

1618.
J. Thank ye, thank ye, Hawthornden—

the laureate having already learned to address a Scottish laird by

what Scott calls ‘his territorial appellation.’ Be this as it may, we have
no authentic notice of any intercourse between the two poets till the
commencement of the ensuing year, when they undoubtedly spent a
considerable time together, as it was then that they had those
Conversations, which, being noted down by Drummond, and
published many years after in his Works, have furnished so much
food for modern controversy. These memoranda partly relate facts
regarding Jonson himself, but chiefly report his opinions of his
poetical contemporaries, which are generally of a censorious
character. Added to them, however, is Drummond’s report on Jonson
himself, in these terms: ‘Ben Jonson was a great lover and praiser of
himself, a contemner and scorner of others, given rather to lose a
friend than a jest; jealous of every word and action of those about
him, especially after drink, which is one of the elements in which he
lived; a dissembler of the parts which reign in him; a bragger of
some good that he wanted; thinketh nothing well done but what
either he himself or some of his friends have said or done. He is
passionately kind and angry, careless either to gain or keep;
vindictive, but if he be well answered, at himself; interprets best
sayings and deeds often to the worst. He was for any religion, as
being versed in both; oppressed with fancy, which hath
overmastered his reason, a general disease in many poets,’ &c.
Drummond has been severely blamed for stating so much, even
privately, regarding a man whom he treated with all the external
marks of friendship; but the censure seems scarcely just. It is not
necessarily to be supposed that he was the less sensible of the
merits of Jonson, that he also observed his being marked by that
vanity, impulsiveness, and irritability which have been remarked in so
many poets. Neither may he have loved him the less—possibly he
only loved him the more—because of his faults. As to the act of
taking such notes, some allowance may be made for a man of
literary tastes, living at a distance from the centre of the world of
letters, in an age when there were no periodicals to prattle about
literary celebrities, or an opportunity occurring of hearing a famous
poet talk of his poetical brethren, and of seeing that eminent person
revealing his own character.
A letter written by Drummond to Jonson in this very month, and
probably only a few days after their conversation, has been
preserved:386
‘TO HIS WORTHY FRIEND, MR BENJAMIN JONSON.
‘Sir—Here you have that epigram you desired, with
another of the like argument. If there be any other thing
in this country (unto which my power can reach),
command it; there is nothing I wish more than to be in
the calendar of them who love you. I have heard from
court that the late masque was not so much approved of
the king as in former times, and that your absence was
regretted—such applause hath true worth, even of those
who otherwise are not for it. Thus, to the next occasion
taking my leave, I remain
Your loving friend,
William Drummond.

January 17, 1619.’

From a consideration of the whole circumstances,
it is most likely that Jonson had spent some time 1618.
in autumn with his Lennox friends, and thus been introduced to the
romantic beauties of Loch Lomond, and perhaps also to the romantic
sports indulged in by the gentry of its neighbourhood. Certain it is,
from Drummond’s report of his conversations, that he designed ‘to
write a Fisher or Pastoral Play, and make the stage of it on the
Lomond Lake.’ And after he had returned to London, Drummond
sent him ‘a description of Loch Lomond, with a map of Inch
Merinoch,’ which, he says, ‘may by your book be made most famous.’
It is possible, however, that this description of Loch Lomond and
map of Inch Merinoch—one of the many isles in the lake—may have
been intended for a prose work, which he also contemplated writing
—namely, his ‘Foot Pilgrimage’ to Scotland, which, with a feeling
very natural in one who found so much to admire where so little had
been known, he spoke of entitling A Discovery. Unfortunately, this
work, as well as a poem in which he called Edinburgh

‘The Heart of Scotland, Britain’s other eye,’

has not been preserved to us. We can readily see that the work
contemplated must have been of a general character, for Jonson
wrote to Drummond (London, May 10, 1619), not merely for ‘some
things concerning the Loch of Lomond,’ but for copies of ‘the
inscriptions at Pinkie,’ referring probably to the Roman antiquities
which had been found in Queen Mary’s time at Inveresk, and also
bids him ‘urge Mr James Scott,’ and send ‘what else you can procure
for me with all speed.’ The king, he adds, was ‘pleased to hear of the
purpose of my book.’ How much to be regretted that we have not
the Scotland of that day delineated by so vigorous a pen as that of
the author of Sejanus!
The last visit Jonson paid in Scotland was to Drummond at
Hawthornden in the month of April, just before his return to London,
which, as we see, he had reached before the 10th of May. He lived
with Drummond on that occasion three weeks, enjoying, doubtless,
the vernal beauties of that romantic spot, as well as the converse of
his friend, and the more substantial hospitalities for which, if
Drummond be right, he had only too keen a relish. Their parting—
which, by Scottish use and wont, would be under the Covine Tree,
when royal Ben set out on foot as before to return to London—who
but wishes he could picture as it really was! Jonson’s letter of the
10th May, written soon after his arrival in London, and breathing of
the feelings which his excursion had excited, may aptly conclude this
notice:
‘TO MY WORTHY, HONOURED, AND BELOVED
FRIEND, MR W. DRUMMOND. 1618.

‘Most loving and beloved Sir—against which titles I should

most knowingly offend, if I made you not some account of
myself to come even with your friendship—I am arrived
safely, with a most Catholic welcome, and my reports not
unacceptable to his majesty. He professed, thank God,
some joy to see me, and is pleased to hear of the purpose
of my book; to which I most earnestly solicit you for your
promise of the inscriptions at Pinkie, some things
concerning the Loch of Lomond, touching the government
of Edinburgh, to urge Mr James Scott, and what else you
can procure for me with all speed. Though these requests
be full of trouble, I hope they shall neither burthen nor
weary such a friendship, whose commands to me I shall
ever interpret a pleasure. News we have none here, but
what is making against the queen’s funeral, whereof I
have somewhat in hand, that shall look upon you with the
next. Salute the beloved Fentons, the Nisbets, the Scotts,
the Livingstones, and all the honest and honoured names
with you, especially Mr James Wroth, your sister, &c. And
if you forget yourself, you believe not in
Your most true friend and lover,
Ben Jonson.387

London, May 10, 1619.’

At the very time when Ben Jonson and John Taylor
had the pleasant experiences of the Highlands 1618.
which are above described and adverted to, sundry
parts of that province of Scotland were the scene of lawless
violences, which these English poets could not have heard of without
horror. M‘Ronald of Gargarach,388 ‘with a band of lawless limmers
his assisters and accomplices,’ went about ‘committing murders,
slaughters, treasonable fire-raisings, and other insolencies, in all
parts where they had any mastery or command.’ It was necessary to
give his rival and antagonist, Sir Lachlan MacIntosh, a commission
‘to pursue him to the deid with fire and sword.’ At the same time,
several men of the Clan Campbell, taking advantage of the absence
of their chief, the Earl of Argyle, ‘has lately broken louss, and goes
athort the country, in companies, sorning [upon] and oppressing his
majesty’s guid subjects, in all parts where they may be masters.’ The
earl, it appeared, had nominated none in his absence ‘to have the
charge and burden to retein his clan and country under obedience.’
To supply this defect, the Council (July 9, 1618) summoned the chief
men of the name—the Lairds of Glenurchy, Caddell, Lawers,
Auchinbreck, and Glenlyon—to give information about the
delinquents, and surety for the future peace of the country.
Contemporaneously with these outrages, were others of the like
nature committed by ‘a number of vagabonds and broken men of
the Clan Donochie (Robertsons),’ particularly in the lands of Simon
Lord Fraser of Lovat.—P. C. R.
It may be remarked that the Earl of Argyle had gone abroad, either
from dissatisfaction with the rewards assigned him for his putting
down of the Macgregors and MacConnel of Kintyre,389 or because of
troubles on account of his debts.390 To the surprise of his
countrymen, he entered the service of Philip III. against the people
of the Low Countries, and, by the persuasion of his wife, joined the
Church of Rome. The poet Craig thus satirised him:

‘Now, Earl of Guile and Lord Forlorn, thou goes,

 Quitting thy prince to serve his Spanish foes:
No faith in plaids, no trust in Highland trews,
Chameleon-like, they change so many hues.’
The king felt as indignant as any, and on the 4th of February 1619,
the Privy Council had this ‘shameful defection’ under its notice. It
was alleged that the apostate earl not only goes openly to mass, but
has associated with ‘auld M‘Ranald’ and sundry other exiled rebels.
Sundry charges against him having been slighted, the Council now
‘do repute him as a traitor,’ and ordain him to be pursued as such.
The earl, who, before leaving his country, had handed over his
estates to his son, did not return till he was enabled to do so by the
grace of Charles I. in 1638.

Thomas Ross, a man of good family, formerly

minister of Cargill, in Perthshire, had gone to Aug. 20.
Oxford to study; but in a moment of partial lunacy
—for the act can be accounted for in no other way 1618.
—he wrote a libel on the Scottish nation, and
affixed it to the door of St Mary’s Church. It spoke in scurrilous
terms of the people of Scotland, excepting only the king and a few
others, and counselled that the king should banish all Scots from his
court. James took the matter in high dudgeon, and had Ross sent
down to Scotland, with strict orders to bring him to condign
punishment. He was accordingly tried at the date marginally noted,
for an act of which the design was assumed to be to have stirred up
the people of England ‘to the cruel, barbarous, and unmerciful
murdering, massacring, and assassine of the haill Scots people, as
well noblemen and councillors as others, attendants about his
highness’s royal person in court.’ The unfortunate satirist professed
penitence, and craved mercy, averring that he had been inops
mentis at the time of his committing the act. Nothing could avail to
save him. His right hand was first struck off, and then he was
beheaded and quartered, his head being fixed on a prick at the
Nether Bow Port, and his hand at the West Port.—Pit.

Mr John Guthrie, minister of Perth, ‘on ane Sunday

after the afternoon’s sermon, married the Master Aug. 29.
of Sanquhar with Sir Robert Swift’s daughter, ane
English knight in Yorkshire. Neither of the parties exceeded thirteen
years of age.’—Chron. Perth.

‘About the midst of November, there appeared a

prodigious comet in the morning, in the north-east, Nov. 18.
broad, and stretching with a large tail towards the
north-west. It appeared fine and clear some few days in the
beginning, and after became more dim and obscure, and vanished
away at last in the north. This comet by appearance portended the
wars of Germany, which began not long after, and continueth yet to
this hour.’—Cal.
Dr Bembridge, ‘a very profound and learned mathematician,’ obliged
the king with an account of this comet. He told him it was as far
above the moon as the moon is above the earth, and not less than
2,300,000 English miles! Rushworth speaks of it as followed by, first,
the Bohemian wars, then the German and Swedish, &c. ‘Dr
Bembridge observed it to be vertical to London, and to pass over it
in the morning; so it gave England and Scotland in their civil wars a
sad wipe with its tail.’—Foun. Hist. Ob.
This notable comet was observed in Silesia, Rome, and Ispahan.
From Skipton’s observations, Halley afterwards computed its orbit. It
passed its perihelion on the 8th of November, at little more than a
third of the earth’s distance from the sun. On the 9th of December,
its tail was 70° in length, being, according to Kepler, the longest that
had been seen for a hundred and fifty years.
This comet is also remarkable as the only one,
besides another in 1607, which was observable by 1618. Dec. 23.
the naked eye in the first half of the seventeenth
century; whereas in other spaces of time of the same extent, as
many as thirteen have been detected. The comet of 1607, which is
the same with that seen in 1682, 1759, and 1835, and usually
known as Halley’s comet, is not mentioned by any of our
contemporary chroniclers as having been visible in Scottish skies.
Christmas was observed in Edinburgh at the command of the king,
and two churches opened for service; but the attendance was scant.
‘The Great Kirk was not half filled, notwithstanding the provost,
bailies, and council’s travels.... The dogs were playing in the flure of
the Little Kirk, for rarity of people, and these were of the meaner
sort.... Mr Patrick [Galloway] denounced judgments ... famine of the
word, deafness, blindness, lameness, inability to come to the kirk to
hear and see, to fall upon those who came not to his Christmas
sermon.’—Cal.
A few weeks afterwards, Richard Lawson, James Cathkin, and John
Mean, merchants, were obliged to appear before the Court of High
Commission, accused of ‘not coming to the kirk on Christmas-day,
for opening of their booth-doors, walking before them in time of
sermon, dissuading others from going to the kirk, and reasoning
against preaching on that day. They answered they did nothing of
contempt; they reasoned to receive instruction, and to try what
warrant others had. They were dismissed, with an admonition to be
modest in their speeches and behaviour in time coming.’—Cal.
On Christmas-day 1621, there was service in the Old Kirk in St Giles,
which the magistrates and state officials attended; but no other
church was open, and Calderwood informs us that ‘one hundred and
six booth-doors or thereby stood open’—a proof of the general
disregard of the festival.
Patrick Anderson, doctor of physic, and who is
usually said to have been physician to Charles I., 1618.
published a tract on the ‘Cold spring of Kinghorn,’
its admirable properties for the cure of sundry diseases. He took
care to draw a distinction between the simply natural efficacy of this
well and ‘the superstitious mud-earth wells of Menteith, or our Lady
Well of Strathern, and our Lady Well of Ruthven, with a number of
others in this country, all tapestried about with old rags, as certain
signs and sacraments, wherewith they arle the devil.’391 He further
assured the public that the ‘clear and delicate cauld water’ of this
spring, being drunk in great quantity, ‘is never for all that felt in the
belly.’ Modern physiologists, it may be remarked, admit the rapid
absorption of saline waters by the stomach; and the drinking of nine
tumblers before breakfast is at this day not uncommon at Airthry.
Dr Patrick Anderson was the inventor of a pill of aloetic character,
which long had a great celebrity in Scotland, and is still in such
repute that an agency-office for its sale may be found in both
Edinburgh and London. He is the more entitled to some notice here,
as our work has been somewhat indebted to a History of Scotland by
him, to be found in manuscript in the Advocates’ Library.

At this time, one Thomas Milne was a maker of virginals in Aberdeen

—a calling, however, ‘but lately put in practice in the burgh.’ The
trade must have been tolerably encouraged, as John Davidson, who
had served an apprenticeship under Milne, now proposed to set up
for himself. On his exhibiting a pair of virginals of his own making as
his ‘master-stick’ before the Council, they gave him the freedom of
the burgh without a fee, which he was too poor to pay.—Ab. C. R.
Died, ‘Mr William Cowper, Bishop of Galloway, a
very holy and good [man], if he had not been 1619. Feb. 15.
corrupted with superior powers and warldly cares
of a bishopric and other such things. He was buried at the south
door of the new kirk callit the South Kirk, in the Greyfriars’ Yard, or
common burial-place of Edinburgh, whilk kirk was newly completed,
and at the funeral sermon consecrated by Mr John Spottiswoode,
archbishop of Sanctandrews.’—Jo. H.
Cowper was an eloquent and able man, and had
been conspicuous for his zeal against bishops, 1619.
‘appearing to all men to hate very much that lordly
dignity in a kirkman, comparing them and their godless followers to
snuffs of candles, whilk not only is destitute of light, but also casts
out a filthy flewrish stink in man’s noses.’ To a former friend, who
had accepted a bishopric, he wrote a despiteful letter, telling him he
had fallen away and apostatised, and while he still loved himself, he
hated his way. Afterwards, ‘perceiving the courses of the bishops
daily going forward, and being a proud ambitious man, glorying in
his gifts, he began first privately to be social and homely with the
bishops, and then, after the Golden Assembly at Glasgow in 1610,
perceiving that the bishops had gotten all their intent, he also
embraced a bishopric, and (1612) was created Bishop of Galloway.’
Peeling that his conduct had been inconsistent, Cowper wrote an
apology, which mainly came to this, that he had got more light than
he had before. ‘One answered merrily: “It is true; for now he has
upon his table two great candles, whereas before he had but one
small candle—other more light I know none.”’
In the end, he announced from the pulpit, he would give full
satisfaction to all who would come and confer with him. ‘Upon whilk
invitation, so many came to him, both in the fields and in his own
house, that he was wearied with them.’ [According to the Chronicle
of Perth—‘The wives of Edinburgh came in to him, and shewed to
him his awn books against friers’ books.’] One person went so far as
to charge him with apostasy, and call upon him to prepare an
answer shortly to the Judge of all the world. It would appear from
what followed that the bishop was by this time out of health. ‘Within
a day or two after, being at his pastime [golf?] in the Links of Leith,
he was terrified with a vision, or an apprehension; for he said to his
playfellows, after he had in an affrighted and commoved way cast
away his play-instruments: “I vow to be about with these two men,
who have now come upon me with drawn swords!” When his
playfellows replied: “My lord, it is a dream: we saw no such thing,”
he was silent, went home trembling, took bed instantly, and died.’—
Row.

It had been a custom of the congregations in

Edinburgh to hold a meeting on the Tuesday Mar. 23.
before the administration of the communion. ‘If
anything was amiss in the lives, doctrines, or any part of the office
of their pastors, every man had liberty to shew wherein they were
offended; and if anything was found amiss, the pastors promised to
amend it. If they had anything likewise to object against the
congregation, it was likewise heard, and amendment was promised.
If there was any variance among neighbours, pains were taken to
make reconciliation, that so both pastors and people might
communicate in love at the banquet of love.’ On the present
occasion, the affair had much the character of a modern public
meeting, and the people stood boldly up to their pastors, arguing
against the innovations of worship now about to be introduced,
particularly kneeling at the sacrament.—Cal.
‘At various times in the year 1621, there were
private meetings of ministers and other good 1619.
Christians in Edinburgh, setting apart days for
fasting, praying, and humiliation, crying to God for help in such a
needful time; whilk exercises, joined with handling of Scripture,
resolving of questions, clearing doubts, and tossing of cases of
conscience, were very comfortable.... Thir meetings the bishops and
their followers, enemies still to the power of godliness and life of
religion, hated to the death; and sundry ministers of Edinburgh
inveighed against them, under the name of unlawful conventicles,
candle-light congregations (because sometimes they continued their
exercises for a great part of the night), persecuting them with
odious names of Puritans, Separatists, Brownists, &c.’—Row.
One of the Edinburgh clergy ‘sent to Nicolas Balfour, daughter of
umwhile Mr James Balfour, minister of Edinburgh, to advertise her
that she was to be banished the town, for entertaining such
meetings in her house; and reviled her despitefully, when she came
to confer with him.’—Cal.

This day, being Easter Sunday, the communion

was administered in the Edinburgh churches for Mar. 28.
the first time after it had been arranged that the
people should kneel on receiving the elements. There being a
general disrelish for this new form, the people left the town in great
numbers to communicate at country churches where the order was
not yet appointed. Of those who attended in town, few willingly
knelt besides government officials and pauper dependents on the
church contributions. ‘Some were dashed and kneeled, but with
shedding of tears for grief.’ In some churches throughout the
country, certain persons told the ministers: ‘The dangers, if any be,
light upon your soul, not on ours!’ Some departed, ‘beseeking God to
judge between them and the minister.’ ‘It is not to be passed over,
how that when John Lauder, minister at Cockburnspath, was
reaching the bread to one kneeling, a black dog started up to snatch
it out of his hand.’—Cal.
On next Easter Sunday, the like disinclination to kneeling was shewn.
In the College Kirk, where sixteen hundred people communicated,
only about twenty kneeled, and it was thought that none would have
done so, ‘if they had not brought the poor out of the hospital, to
begin, and give a good example.’ These, ‘being aged, poor, and
ignorant persons, durst not refuse;’ yet even of them, some, ‘when
they were kneeling, knocked on their breasts and lifted up their
hands and eyes.’
While the officers of the government and many others joined
cordially in the new arrangements, the bulk of the people revolted
from them. Whenever they heard of a church in the country where
they might be allowed to communicate sitting, they resorted to it in
great numbers; ‘whereupon the auditory of the kirks of Edinburgh
became rare and thin.’
On Easter Sunday, 1622, at the communion in the
Old Kirk, Edinburgh, ‘among all the two hundred 1619.
and fifty [communicants] there was not a man of
honest countenance but the President, Sir William Oliphant, the
Advocate, Sir Henry Wardlaw, the Provost, the Dean of Guild, Dame
Dick, the Master of Works’ wife, and two bailies, who communicate
not: plaids, gray cloaks, and blue bonnets made the greatest show.’
At the same time, ‘many of the profaner sort were drawn out upon
the sixth of May, to May-games in Gilmerton and Roslin; so profanity
began to accompany superstition and idolatry, as it had done in
former times. Upon the first of May, the weavers in Paul’s Work,
English and Dutch, set up a high May-pole, with their garlands and
bells hanging at them, whereat was great concourse of people.’—
Cal.

John Maxwell of Garrarie—‘ane landed gentleman,

in the rank of ane baron, worth three thousand Apr. 2.
merks of yearly rent, and above’—was, with his
son, George Maxwell, tried for the crime of treason. Garrarie had, in
a crafty manner, possessed himself of the estate and whole worldly
means of John M‘Kie of Glashock, who thus became a miserable
dependent upon him, almost constantly living in his house. At
length, tiring of the company, and probably also of the complaints of
the unfortunate Glashock, Garrarie and his son resolved to be rid of
him. On the 8th of July 1618, when Glashock was coming by night to
the place of Garrarie, the two Maxwells, attended by an armed band
of servants, fell upon him, tied his hands and feet, strangled him,
and flung his body into a peat-moss.
The accused protested their innocence, and it was necessary twice
to postpone proceedings against them, apparently for lack of
evidence. Another remarkable circumstance was, that no fewer than
seventeen gentlemen of the district incurred fines by failing to
appear as jury-men. Notwithstanding these and other impediments,
justice asserted its claims, and Garrarie and his son had their heads
stricken off at the Cross of Edinburgh, with forfeiture of lands and
goods.—Pit.

The period at which we have now arrived, being

one of internal peace, is distinguished as the time 1619. July 22.
when the practice of several of the useful arts was
first introduced into Scotland. Sir George Hay, the Clerk Registrar—
ancestor of the Earls of Kinnoul—a man of talent and intelligence,
had set up, at the village of Wemyss, in Fife, a small glasswork,
being the first known to have existed amongst us. An ironwork, of
what nature we are not informed, was also originated by Sir George.
The Privy Council informed the king that Sir George Hay had
enterprisingly set up works for iron and glass, which for some years
he supported at high charges, in hopes of being remunerated by
profits. ‘But now he has found, by experience, that all the country
dispatch of his glass in ane haill year will not uphold his glassworks
the space of ane month.’ It was entreated that the king would allow
of Sir George’s glass being sold unrestrainedly in England, and at the
same time restrict the exportation of coal into that country. By such
means he admitted he had a hope of thriving.—M. S. P.
It would appear that the native manufacture, after all, prospered; for
in February 1621, the Privy Council appointed a commission,
including Sir George Hay, to meet and confer anent the glassworks,
to examine and try the glass, and see that measures were taken for
the full supply of the country, so as to save the introduction of
foreign glass.
The commission soon after reported that the glassworks at Wemyss
were going on satisfactorily. The cradles or cases contained fifteen
wisps, each wisp having three tables, three-quarters of a Scots ell
and a nail in depth. The glass was fully as good as Danskine glass,
though they could wish it to be ‘thicker and tewcher.’ Being less sure
of the character of the drinking-glasses produced, they
recommended patterns of English glass of that kind to be
established in Edinburgh Castle, for trying the sufficiency of Scots
glass in all time coming. On the strength of this report, the Council
granted the desired monopoly as against foreign glassmakers, on
certain conditions, one of which was, that the price of ‘braid glass’
should not exceed ‘twelve punds the cradle.’—P. C. R.

Before this time, soap was imported into Scotland

from foreign countries, chiefly from Flanders. It Oct. 20.
was estimated that the entire quantity brought in
was about a hundred and twenty lasts. The king 1619.
now gave a patent to Mr Nathaniel Uddart for the
manufacture of soap within the country, and Mr Nathaniel
accordingly raised a goodly work at Leith, furnishing it with all
matters pertaining to the business. Before he had been at work two
years (June 21, 1621), he petitioned the Privy Council that foreign
soap should be prohibited, professing to be able himself to furnish
all that was required for the use of the country people, and thus
save money from being sent out of the country—a piece of false
political economy much in favour, as we have seen, in those days.
The Privy Council, after taking some pains to ascertain the character
of ‘Mr Nathaniel his soap,’ and becoming convinced that he could
furnish the quantity needful, granted the prohibition requested, but
not without fixing down the native manufacturer at a maximum
price. This was decreed to be £24 per barrel for ‘green soap,’ and
£32 per barrel for ‘white soap,’ each barrel to contain sixteen stone.
If we are right in considering the stone at 17·39 pounds avoirdupois,
and the last as containing twelve barrels, the estimated amount of
regularly manufactured soap annually used in Scotland at that time
might be approximately 400,880 pounds. In 1845, taking its
consumption of the same article as one-seventh of that of Great
Britain, Scotland consumed about twenty-seven million pounds!
Matters had not proceeded upon the footing of protection for above
two years, when complaints became rife as to the inconveniences
sustained by the lieges. The merchants of Edinburgh felt it as a
grievance that their traffic for soap with the Low Countries was
interrupted. They also complained of the quality of the article
produced at Leith. The merchants of Dumfries and other distant
ports groaned at being obliged to carry soap a long land-journey
from Leith, when they could have it brought by ships direct to their
doors. In short, it was not to be borne.
The Lords of Council took pains to inform themselves of the whole
matter. They also had a letter from the king testifying his ‘dislike of
this and others the like restraints, as being a mean to overthrow
traffic and to destroy commerce.’ Being now satisfied that Uddart’s
privilege was ‘hurtful to the commonweal,’ and that ‘the subjects has
not been so commodiously furnist with the soap made by the said
Mr Nathaniel as formerly they were with foreign soap,’ they decreed
(July 1623) that the restraint should terminate in a year, or sooner, if
he should produce an inferior or dearer article.—P. C. R.

While the struggle was going on between the Episcopalian and

Presbyterian principles, there was a small group of Edinburgh
citizens, including the booksellers Cathkin and
Lawson, who took a lead in opposing the new 1620. Mar. 30.
practices, and standing up against the dictates of 1620.
the High Commission. Deeply impressed with
evangelical doctrine, and viewing all ceremonies as tending to the
corruption of pure religion, they were disposed to venture a good
way in the course they entered upon. Their wranglings in the kirk-
session against ministers of the court fashion, and their earnest
private exercises, were fully known to the king; but he bore with
them till they began to lend countenance and active help to the few
refractory ministers who fell under the ban of the bishops. He then,
at the date noted, ordered them to be removed as ‘evil weeds’ from
Edinburgh—William Rig, merchant, and James Cathkin, bookseller, to
Caithness; Richard Lawson, to Aberdeen; Robert Meiklejohn, skinner,
to Dunkeld; John Mean, to Wigton in Galloway; and Thomas Inglis,
skinner, to Montrose. This was a great stretch of power in a country
professedly under regular laws; and even the state-officers felt it to
be so. After some dealings, in the course of which the clergy were
eager to negative a suspicion of their having sent the names of the
men to the king, the archbishop of St Andrews said he would
intercede for them, and meanwhile stayed further proceedings. In
the end, the offence of the Edinburgh patriots was passed over for
this occasion.—Cal.
The Edinburgh pulpits were at this time filled with men wholly
devoted to the Episcopalian system, and such of the people as were
strenuous for the Presbyterian model, had to act not merely without
clerical leaders, but in despite of clerical opposition. As a specimen
of the spirit of these metropolitan clergy—‘Mr William Struthers
(January 9, 1619) made a sermon in the Little East Kirk, whilk, by all
the holy divines in Scotland, was judged rather to have been a
discourse of hateful passion nor a sermon of a charitable divine or
loving theologue. For the most part of his haill discourse consisted in
calling Christ’s flock of Edinburgh a pack of cruel people, seeking the
overthrow of their ministry; calling them also the authors of the
Seventeenth of December.392... He also alleged the doings of the
good town of that day till be in all histories a foul blot to them for
ever. He alleged the people were bund to follow him and the rest of
his brethren the ministers, and to do all things that they bade them
do, calling the ministers the heid, and the people the tail, and
whatever the ministers as the heid spak, it was good and savoury,
and whatever the tail or the people spak was unsavoury, adding
thereto that the language of the tail was deir of the hearing.... This
sermon made me and all the hearers thereof tremble for fear to
behold fit untruth spoken in the chair of verity....’—Jo. H.

It has been seen that horse-racing was, from an

early time, practised as a public amusement at May 3.
various places in Scotland. One of these, not
formerly noticed, was Paisley. A silver bell of four 1620.
ounce weight was made in 1608 to serve as a prize
for the Paisley race: such was in those days the accustomed prize at
a race, giving rise to the proverbial expression—‘He bore off the bell.’
It may be remarked, however, that the winner of a silver bell at a
race did not obtain it as permanent property, but only for a year’s
keeping, as is customary with the silver arrows and silver clubs now
played for by archery and golfing societies.
At the date noted, the Town Council of Paisley, under the guidance
of their provost, the Earl of Abercorn, arranged that their annual
horse-race should be run on the 6th of May, ‘to be start at the gray
stane called St Cormel’s Stane, and frae that richt east to the little
house at the causeyend of Renfrew, and frae that the king’s highway
to the Wall-neuk of Paisley; and what horse first comes over the
score at ... Renfrew, sall have ane double angel; and the horse and
master thereof that first comes over the score at the Wall-neuk of
Paisley, sall have the said bell with the said burgh’s arms thereon, for
that year, together with the rest of the gold that sall be given with
the said bell ... except ane double angel that sall be given to the
second horse and his master that comes next over the score to the
foremost....’ The horses and their owners to gather at Paisley in
good time before the race, and the riders to be weighed at the Tron
of the burgh. It was also arranged that there should be ‘an aftershot
race ... frae ane score at the slates of Ellerslie to ane other score at
the causey-head of the burgh of Paisley, by horse of the price of ane
hundred merks ... for ane furnished saddle, whilk sall be presented
by the said bailies of Paisley present and to come at the score of the
said causey-head.’393

Patrick Anderson, a native of Ross-shire, and

nephew of the celebrated Bishop Lesly, had risen 1620.
by learning and talent to be head of the Scots
College at Rome. This situation he left in order to add his exertions
to those which a number of his co-religionists were making, at the
hazard of their lives, for the recovery of Scotland from what they
called the Calvinistic heresy. Dempster speaks of him as ‘moribus
innocens ac fide integer,’ and tells us he had no superior in
mathematics and theology. Such as he was, he threw himself into
this mission with a zeal and gallantry which no generous opponent
could now dispute, but which was regarded in the Scotland of his
own day as only a diabolic mania for the turning of living souls to
death and perdition.
Poor Patrick had not practised long, when he was
apprehended with his mass-clothes, books, and May 18.
papers, and committed to prison as a trafficking
Romish priest. He owned to the fact of his having performed mass
sundry times, but would not tell in whose houses. In the ensuing
October, a brother-missionary, an Irishman, named Edmund Cana,
was apprehended, along with a younger brother, ‘who carried his
mass-clothes, a portable altar, a flagon of wine, and other requisites
necessar for the mass.’—Cal.
Possibly, King James had heard of the merits of Father Anderson as
a man of learning, and felt some sympathy for him; perhaps the
French ambassador made friendly intercession in his behalf.
However it was, after the Father had suffered nine months’
imprisonment, the king came to the resolution to shew him some
mercy. At his command, the Privy Council liberated the Father from
prison, with a suit of good clothes, and some money in his pocket,
on condition that he should leave Scotland, and return no more;
otherwise, he would be liable to capital punishment. It was enjoined
upon the provost and bailies of Edinburgh that they should ‘try and
speir out some ship bown from the port of Leith towards France or
Flanders; and when the ship is ready to lowse, that they tak the said
Patrick Anderson furth of their Tolbooth, carry him to the ship, and
deliver him to the skipper, and see him put aboard of the ship; and
that they give a strait command and direction to the skipper that the
said Anderson be not sufferit to come ashore again till their arrival at
their port in France or Flanders, where they sall put him a-land, and
sall report a certificate from the magistrates of the town or port
where they land, that the said Anderson was set ashore there.’394—
P. C. R.
The Catholic Church was at this time anxiously set
upon the recovery of Scotland; and many were 1620.
they who devoted themselves to the work. We are
now disposed to wonder, not merely how so many men were
induced to risk their lives in this mission, but how they should have
expected to produce conversions in a field so inveterately Protestant.
There were, however, some encouraging precedents. It was but
recently that St Francis of Sales had brought thousands of the Swiss
Calvinists back to the bosom of the church. He and his cousin, Lewis
de Sales, entered a Protestant canton in September 1594, amidst
the tears and remonstrances of their friends, who believed their task
impracticable, as well as dangerous. In the course of a very few
years, says Alban Butler, ‘his patience, zeal, and eminent virtue
wrought upon the most obdurate, and insensibly wore away their
prejudices. It is incredible what fatigues and hardships he underwent
in this mission; with what devotion and tears he daily recommended
the work of God; with what invincible courage he braved the
greatest dangers; with what meekness and patience he bore all
manner of affronts and calumnies. In 1596, he celebrated mass on
Christmas-day in the church of St Hippolytus at Thonon, and had
then made seven or eight hundred converts. In 1598, the public
exercise of the Catholic religion was restored, and Calvinism
banished by the duke’s orders, over all Chablais and the two
bailiwicks of Terni and Guillard.’ At the same time, ‘his extraordinary
sweetness, in conjunction with his eminent piety, reclaimed as many
vicious Catholics as it converted heretics. The Calvinists ascribe
principally to his meekness the wonderful conversions he made
amongst them. They were certainly the most obstinate of people at
that time near Geneva; yet St Francis converted no fewer than
seventy-two thousand of them.’395 Such success in the great
stronghold of Calvinism might well engender hopes regarding
Scotland, whose determined adherence to the reformed faith had
not then been so much tried as we now know it to have been.
The tanning of leather may be said to have been
introduced into Scotland at this time. About a June.
dozen tanners from Durham, Morpeth, and
Chester-le-Street, were brought in, under royal patronage, in order
‘to instruct the tanners and barkers of the kingdom in the true and
perfect form of tanning.’ They were invested with certain privileges,
and distributed to various parts of the kingdom. It was hoped
through this means that much money, which was usually spent on
foreign leather, would now be kept within the kingdom.
Unfortunately for the success of this reformation, a
tax was put upon the leather—four shillings Scots 1620.
per hide for the first twenty-one years, and
thereafter one penny. The consequence was a grievous discontent
among the cordwainers, who everywhere did what in them lay to
thwart his majesty’s design. ‘To steir the people up to exclaim
against it, they have very extraordinarily raised the prices of boots
and shoon, to twenty shillings or thereby the pair of boots, and six
shillings or thereby the pair of shoon, more nor was paid before;’
thus oppressing the whole country, and particularly the poorer sort
of people, besides slandering the king and his Council. In January
1622, the Privy Council dealt with a complaint that many of the
tanners throughout the country, disregarding the obvious benefit to
themselves and the commonwealth from the new modes, continued
the old practice of letting their leather remain but a short time in the
pots, and then bringing it to market in a raw state. By way of a
stimulus to these persons, a certain number of them were
proclaimed rebels.

At this time, the Earl of Sutherland being a minor,

and the family resources much reduced, the Oct.
inhabitants of the district ‘did shew themselves
exceeding loving and thankful to their Master and superior; for not
only did they give a general contribution—every one according to his
estate and ability—for defraying of his sister’s portion, who was now
to be married to the Laird of Pitfoddels, but also they yielded a
voluntary yearly support to the earl and his two brothers’ fitter
maintenance at the university for the space of five years.... So much
did they value and regard the education and good-breeding of him
who was to govern and command them, knowing how much it doth
concern every state and country to have weel-bred and wise
superiors; which good-will and course of theirs was exceedingly weel
thought of by the Earl of Sutherland and his greatest friends.’—G. H.
S.

We find it noted that in this year a pearl was found in the burn of
Kellie, a tributary of the Ythan, Aberdeenshire, so large and beautiful
that it was esteemed the best that had at any time been found in
Scotland. Sir Thomas Menzies, provost of Aberdeen, obtaining this
precious jewel, went to London to present it to the king, who, in
requital, ‘gave him twelve or fourteen chalder of victual about
Dunfermline, and the custom of merchant goods in Aberdeen during
his life.’396 It has been reported that this pearl was inserted in the
apex of the crown of Scotland.
Apparently this circumstance called the king’s
attention to the old repute of certain Scottish rivers 1620.
for the production of pearls. In January 1621, we
find the Privy Council adverting to the fact, that the seeking for
pearls had for many years been left to interlopers, who pursued their
vocation at unseasonable times, and thus damaged the fishery, to
the hurt of his majesty’s interest, he having an undoubted right to all
pearls, as he had to all precious metals found in his dominions.
Being now inclined to take up pearl-seeking on his own account, he
issued a proclamation for the preservation of ‘the waters wherein the
pearls do breed;’ and, took measures to have the fishery conducted
on a regular plan ‘no pearls to be socht or taken but at such times
and seasons of the year when they are at their chief perfection both
of colour and quality, whilk will be in the months of July and August
yearly.’ The Privy Council commissioned three gentlemen to protect
the rivers, and ‘nominat expert and skilful men to fish for pearls at
convenient seasons;’ one gentleman for the rivers of Sutherland,
another for those of Ross, and another (Mr Patrick Maitland of
Auchincroch) for the waters Ythan and Don. The gentleman just
named was further made commissioner ‘for receiving to his
majesty’s use, of the haill pearls that sall be gotten in the waters
within the bounds above written, and who will give reasonable prices
for the same; the best of the whilk pearls for bigness and colour he
sall reserve to his majesty’s awn use.’
Patrick Maitland gave up his commission in July 1622, and it was
then conferred on Robert Buchan, merchant in Aberdeen, who was
reputed to be skilful in fishing for pearls, and ‘hath not only taken
divers of good value, but hath found some to be in divers waters
where none were expected.’—P. C. R.
Among the acts of the first parliament of Charles I. was one for the
‘discharge of Robert Buchan’s patent of the pearl and other
monopolies.’ Since then, there has occasionally been successful
fishing for pearls in this river; it is said that ‘about the middle of the
last century, a gentleman in Aberdeen got £100 for a lot of pearls
found in the Ythan.’ The mouth of the river has a great muscle and
cockle fishery, and is accordingly the haunt of an extraordinary
variety and quantity of sea-fowl. In summer, when the water is low,
school-boys often amuse themselves by going in search of pearls,
feeling with their toes for the shell, which is distinguished by its
curved shape, and griping it when found with a kind of forceps at
the end of a long stick.397

The church historian Calderwood notes the

occurrence of three fires in Edinburgh in one day 1621. Feb. 6.
as being regarded by the people as ‘foretokenings
of some mischief.’ ‘About the same time,’ he adds, ‘there came in a
great whale at Montrose; which was also apprehendit to be a
forerunner of some trouble.’

On a complaint that coal had risen to eight

shillings the load, the Privy Council had interfered Mar. 1.
in the usual rash manner, and dictated a certain
maximum price to be exacted for the article; namely, seven shillings
the load—that is, horse-load; for coal was borne at this time, and for
a long time after, on horseback. Certain coal-proprietors—Alexander,
Master of Elphinstone; Samuel Johnston of Elphinstone; Sir James
Richardson of Smeaton; Robert Richardson of Pencaitland; Jonet
Lawson, Lady Fawside; and David Preston of Whitehall—now
petitioned, setting forth that the cost of mining coal had greatly risen
of late years, and that the dearth of the article to the public was
much owing to the base fellows who act as carriers of coals. It was
represented that some of the proprietors of ‘coal-heughs’ were
£10,000, and some even £20,000 out of pocket. The Master of
Elphinstone’s coal of Little Fawside had been on fire for several
years; another mine of the same owner had caused an outlay of
£8000. The Smeaton pits had been so unproductive for some years
as scarcely to supply the laird’s house. The coal of Elphinstone had
proved for nine years barren, and 20,000 merks had been sunk upon
it, being more than it promised ever to repay. The coal of Mickle
Fawside had undone the late laird’s estate, and ‘made him to sell
ane part of his auld heritage:’ what with fire on the one hand and
water on the other, it was a hopeless case. As for the coal of
Pencaitland, it was wasted and decayed, past hope of recovery, but
at such extraordinary charges as it was not worth having bestowed
upon it. The basis of the evils complained of lay with the coal-
carriers, who dealt fraudulently with the public. Had the particulars
been rightly known, the lords, it was assumed, would never have
given a decreet against the complainers, ‘who are gentlemen of grit
charge and burden,’ overlooking the faults of those base fellows who
carry coals.
The lords appointed a commission to inquire into
the matter, and report what prices they thought 1621.
ought to be fixed for this necessary article. In
consequence of a report soon after given in by this commission, it
was ordained that the price of coal at ‘the hill’ should be 7s. 8d. (7-
2/3d. sterling) per load; and it was at the same time agreed that a
measure for the load and a charge for carriage should afterwards be
appointed.—P. C. R.
On the 23d April 1623, an act of Privy Council was passed in favour
of Samuel Johnston, laird of Elphinstone, in consideration of his
having super-expended 20,000 merks on his coal-heughs, ‘to his
great hurt and apparent wrack.’ It was stated that he had
Welcome to our website – the ideal destination for book lovers and
knowledge seekers. With a mission to inspire endlessly, we offer a
vast collection of books, ranging from classic literary works to
specialized publications, self-development books, and children's
literature. Each book is a new journey of discovery, expanding
knowledge and enriching the soul of the reade

Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.

Let us accompany you on the journey of exploring knowledge and

personal growth!

ebookfinal.com